Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
2.443724408s
FAIL (2025-10-07 07:07:44 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
46 (329h16m46.051786948s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment...
Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment.
Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment
Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments...
Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments...
Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 2.443755596s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
2.548520562s
FAIL (2025-10-07 07:06:58 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
46 (329h16m46.051786948s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment...
Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment.
Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment
Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments...
Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments...
Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 2.548551131s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
2.45636901s
FAIL (2025-10-07 07:05:45 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
46 (329h16m46.051786948s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment...
Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment.
Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment
Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments...
Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments...
Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 2.456397604s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
2.491680815s
FAIL (2025-10-07 07:05:29 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
46 (329h16m46.051786948s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment...
Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment.
Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment
Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments...
Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments...
Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 2.491710838s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
2.344911465s
FAIL (2025-10-07 07:04:41 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
46 (329h16m46.051786948s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment...
Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment.
Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment
Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments...
Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments...
Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 2.344952567s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
3.361908005s
FAIL (2025-10-07 07:03:59 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
46 (329h16m46.051786948s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment...
Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment.
Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment
Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments...
Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments...
Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 3.361936476s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
3.19229839s
FAIL (2025-10-07 07:03:29 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
46 (329h16m46.051786948s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment...
Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment.
Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment
Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments...
Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments...
Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 3.192327955s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
2.769079092s
FAIL (2025-10-07 07:02:19 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
46 (329h16m46.051786948s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment...
Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment.
Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment
Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments...
Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments...
Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 2.769121263s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
2.666834033s
FAIL (2025-10-07 07:01:24 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
46 (329h16m46.051786948s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment...
Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment.
Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment
Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments...
Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments...
Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 2.666865376s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
3.699525779s
FAIL (2025-10-07 07:00:16 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
46 (329h16m46.051786948s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment...
Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment.
Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment
Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments...
Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments...
Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 3.699559193s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
2.564453167s
FAIL (2025-10-07 06:59:23 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
46 (329h16m46.051786948s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment...
Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment.
Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment
Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments...
Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments...
Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 2.564492198s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
2.387495634s
FAIL (2025-10-07 06:58:45 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
46 (329h16m46.051786948s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment...
Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment.
Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment
Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments...
Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments...
Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 2.387527665s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
2.655060176s
FAIL (2025-10-07 06:57:38 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
46 (329h16m46.051786948s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment...
Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment.
Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment
Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments...
Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments...
Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 2.655092058s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
2.749769371s
FAIL (2025-10-07 06:56:53 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
46 (329h16m46.051786948s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment...
Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment.
Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment
Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments...
Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments...
Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 2.749813281s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
2.438314131s
FAIL (2025-10-07 06:56:00 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
46 (329h16m46.051786948s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment...
Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment.
Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment
Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments...
Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments...
Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 2.438356362s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
2.322258781s
FAIL (2025-10-07 06:55:06 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
46 (329h16m46.051786948s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment...
Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment.
Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment
Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments...
Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments...
Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 2.322352115s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
2.405496851s
FAIL (2025-10-07 06:54:39 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
46 (329h16m46.051786948s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment...
Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment.
Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment
Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments...
Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments...
Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 2.405528418s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
2.528206467s
FAIL (2025-10-07 06:54:08 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
46 (329h16m46.051786948s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment...
Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment.
Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment
Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments...
Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments...
Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 2.528241233s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
2.544905129s
FAIL (2025-10-07 06:53:33 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
46 (329h16m46.051786948s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment...
Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment.
Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment
Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments...
Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments...
Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 2.544940556s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
2.343669286s
FAIL (2025-10-07 06:52:23 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
46 (329h16m46.051786948s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment...
Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment.
Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment
Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments...
Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments...
Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 2.343701952s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
2.430919468s
FAIL (2025-10-07 06:51:15 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
46 (329h16m46.051786948s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment...
Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment.
Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment
Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments...
Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments...
Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 2.430976876s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
2.601941245s
FAIL (2025-10-07 06:50:03 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
46 (329h16m46.051786948s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment...
Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment.
Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment
Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments...
Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments...
Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 2.601974535s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
2.306796414s
FAIL (2025-10-07 06:48:55 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
46 (329h16m46.051786948s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment...
Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment.
Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment
Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments...
Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments...
Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 2.306833066s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
2.451799054s
FAIL (2025-10-07 06:48:00 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
46 (329h16m46.051786948s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment...
Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment.
Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment
Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments...
Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments...
Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 2.451829401s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
1m0.101825645s
FAIL (2025-10-07 06:47:58 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
46 (329h16m46.051786948s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment...
Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment.
Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece...
Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment
Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments...
Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil>
Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments...
Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 1m0.101854574s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
410.238697ms
FAIL (2025-10-07 05:32:07 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 10)
======LOGS======
288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.525978676 (327h18m12.329443254s) [noise=true , address=89.149.201.147:4020 , piece_id=6X366FMMBQY7UMWRZWHWK24ODNSXNRK33EBT357XTN2QGNZ2XARQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.529349437 (327h18m12.332814029s) [noise=true , address=89.149.201.147:4007 , piece_id=GISIRIBAQZ4POYEFCQIFGEORZEB3DNANYK2FRE47HJD6KCHCWFIQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.531248958 (327h18m12.334713557s) [noise=true , address=81.171.14.70:4034 , piece_id=WXUMWYCPOYMDQRAC3GCH66ORLQEW4QMXCT6Z3IZGN75QPOHRJ44A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.544080578 (327h18m12.347545162s) [noise=true , address=89.149.201.147:4005 , piece_id=UN4KYWSCWFRCG6I44PP3HG3WQYJHHRYA73RE7XKYNWFMQZ4BPNHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.556041962 (327h18m12.359506556s) [noise=true , address=81.171.14.70:4030 , piece_id=W7VSTDTWQZKAKTQ4XXJEZEWWULVUS3VWLK7GXAIC5DC5JXWGDB6A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.612240410 (327h18m12.41570499s) [noise=true , address=81.171.14.70:4025 , piece_id=EFCX2CLEJGLGAS5HY66X7YJUNZEMYDXRDTRG4TZQ25IF4J6QZPUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:49:24.310483073 (327h19m12.113947666s) [noise=true , address=185.167.204.205:4011 , piece_id=2VDETVRCNNWJC7CRL6WRZJZMZFIHI76LWIJAFRIBLMJUECKZ3SKA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CiRxadZgQLgJ41bv86hSTEHLv2CzXC4jz9Q3r5dLEX6doYoZ3, address:185.167.204.205:4011): protocol: rpc: dial tcp 185.167.204.205:4011: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310527972 (327h19m12.113992556s) [noise=true , address=81.171.14.70:4028 , piece_id=HEFFOFVIYWTKT4RBKADCIREX354RWAO5USR2VTTIVVIERFV3AKDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310706256 (327h19m12.114170848s) [noise=true , address=89.149.201.147:4016 , piece_id=NR6WTUSOAMYODXHX2V62FGTKTBFQTK5JENXKR5J4HTBGIGKTTGMA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5, address:89.149.201.147:4016): protocol: rpc: dial tcp 89.149.201.147:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310894451 (327h19m12.114359047s) [noise=true , address=185.167.204.206:4058 , piece_id=WFFNR3LXSVF3HVHKHU4O6LBYVDDFGBLGCG4ZQZHOKJEECNNJIKXQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: rpc: dial tcp 185.167.204.206:4058: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310931505 (327h19m12.114396086s) [noise=true , address=185.167.204.206:4068 , piece_id=HBYTPAXUFS7ZM7OQRAKPDNITIVQOLSOH4RN362XBXOEZB4RFEGEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1sqDsiZKL7g4FbDyc9Ew2ojD1tTLzj4B19ma8hrCvWoTCy4j4D, address:185.167.204.206:4068): protocol: rpc: dial tcp 185.167.204.206:4068: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311073712 (327h19m12.114538311s) [noise=true , address=185.167.204.205:4018 , piece_id=KJA3E24NTU4J3FTWCJ2U4B5UICGUWDLS5VSTMB45SFGQNIGO3BSA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: rpc: dial tcp 185.167.204.205:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311100756 (327h19m12.114565337s) [noise=true , address=185.167.204.205:4013 , piece_id=6ZBB2XTBENBEQMD34O7LIISQHOCEOEZADNPXZOGD5XYJY4OLRHHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: write tcp 139.59.144.22:59630->185.167.204.205:4013: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311261060 (327h19m12.114725661s) [noise=true , address=81.171.14.70:4048 , piece_id=QUGF3RADQM3FHXACWUHOYCUAYYBI33B4XMXZ62JL5O7L2KIVUOUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: rpc: dial tcp 81.171.14.70:4048: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311315674 (327h19m12.114780271s) [noise=true , address=89.149.201.147:4012 , piece_id=MSSU4KAU2AR3YULH6W7JOXM6IG3NIAFU6UGX3FCFEKFLN72BSVYQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311573657 (327h19m12.115038254s) [noise=true , address=185.167.204.205:4025 , piece_id=XB2P432VMB2BJHSWNQCB4B33ZIDAR43QQD74ABRFF43MYRJZDAEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311641089 (327h19m12.11510567s) [noise=true , address=185.167.204.206:4060 , piece_id=FMVOGLOA67LN7G5JVY6NZZCNPR7PMZHBURUQGNKIRIUK7ZV4EGQA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311759265 (327h19m12.115223862s) [noise=true , address=185.167.204.205:4028 , piece_id=26AU5QE6RTN7FTB7XXOB2CGR5VJ7SIRURQUN2DB5G64EIIU6YV5Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1HZoGwBuBh9Kg7gkvH9pfrjtxQurwxdnF7pWfycGuLHrHTSq61, address:185.167.204.205:4028): protocol: rpc: dial tcp 185.167.204.205:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311824674 (327h19m12.115289254s) [noise=true , address=185.167.204.206:4062 , piece_id=WWXKTOXCQ3X5QURTWE6DJ7VC25IOK4U7LKPZ72FRCQ7UUMQMD45A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 139.59.144.22:48898->185.167.204.206:4062: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311937619 (327h19m12.115402215s) [noise=true , address=185.167.204.206:4055 , piece_id=3QNJ4SSDSMAZVSU4KKKB2XO4I24RAVORE3G375K4Z5ARSLIODWCQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CYRieezm1mjtGchWdPrB2mNaHVRvTS85umjB6cQyYKnvisXSB, address:185.167.204.206:4055): protocol: rpc: dial tcp 185.167.204.206:4055: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312048594 (327h19m12.115513172s) [noise=true , address=185.167.204.205:4022 , piece_id=UMO7EQ4TYZTTBYTOTCSD7PIWPBKDE6UQNRRWJ62UJXMAQEK7VE2Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: rpc: dial tcp 185.167.204.205:4022: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312109260 (327h19m12.115573856s) [noise=true , address=89.149.201.147:4014 , piece_id=DFU2C7O7SSA3SUZ5RFKRN5XU6BEDCJVNNOAUPRDLFXOO5WVTWPDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 139.59.144.22:45014->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312154108 (327h19m12.115618686s) [noise=true , address=185.167.204.206:4051 , piece_id=G3ZZYDQ3RA37A56XPHCN5SZP5OX32HOYS5QRK6R3IPVETEGZJYKQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: write tcp 139.59.144.22:55302->185.167.204.206:4051: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312192236 (327h19m12.115656819s) [noise=true , address=185.167.204.205:4021 , piece_id=Y3IZRKFF75CAXPCZS56QGCO65NTLFJBIQB2ZT6NND5J33FWGEENA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12FDxcFVTdQeudwXi3bRkqnBSQ7NhjChgqkCZLmEC9T4bxdGA4w, address:185.167.204.205:4021): protocol: rpc: dial tcp 185.167.204.205:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312223288 (327h19m12.115687882s) [seg_pos={0 0}, upload=288035]: Done waiting for segment. successful: 10 optimal: 26 errs: <nil>
Oct 7 04:49:24.327964660 (327h19m12.131429241s) [upload=288035]: Done uploading segments...
Oct 7 04:49:24.327992813 (327h19m12.131457395s) [upload=288035]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 10)
Duration: 410.289567ms
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
386.223729ms
FAIL (2025-10-07 05:31:41 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 10)
======LOGS======
288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.525978676 (327h18m12.329443254s) [noise=true , address=89.149.201.147:4020 , piece_id=6X366FMMBQY7UMWRZWHWK24ODNSXNRK33EBT357XTN2QGNZ2XARQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.529349437 (327h18m12.332814029s) [noise=true , address=89.149.201.147:4007 , piece_id=GISIRIBAQZ4POYEFCQIFGEORZEB3DNANYK2FRE47HJD6KCHCWFIQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.531248958 (327h18m12.334713557s) [noise=true , address=81.171.14.70:4034 , piece_id=WXUMWYCPOYMDQRAC3GCH66ORLQEW4QMXCT6Z3IZGN75QPOHRJ44A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.544080578 (327h18m12.347545162s) [noise=true , address=89.149.201.147:4005 , piece_id=UN4KYWSCWFRCG6I44PP3HG3WQYJHHRYA73RE7XKYNWFMQZ4BPNHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.556041962 (327h18m12.359506556s) [noise=true , address=81.171.14.70:4030 , piece_id=W7VSTDTWQZKAKTQ4XXJEZEWWULVUS3VWLK7GXAIC5DC5JXWGDB6A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.612240410 (327h18m12.41570499s) [noise=true , address=81.171.14.70:4025 , piece_id=EFCX2CLEJGLGAS5HY66X7YJUNZEMYDXRDTRG4TZQ25IF4J6QZPUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:49:24.310483073 (327h19m12.113947666s) [noise=true , address=185.167.204.205:4011 , piece_id=2VDETVRCNNWJC7CRL6WRZJZMZFIHI76LWIJAFRIBLMJUECKZ3SKA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CiRxadZgQLgJ41bv86hSTEHLv2CzXC4jz9Q3r5dLEX6doYoZ3, address:185.167.204.205:4011): protocol: rpc: dial tcp 185.167.204.205:4011: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310527972 (327h19m12.113992556s) [noise=true , address=81.171.14.70:4028 , piece_id=HEFFOFVIYWTKT4RBKADCIREX354RWAO5USR2VTTIVVIERFV3AKDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310706256 (327h19m12.114170848s) [noise=true , address=89.149.201.147:4016 , piece_id=NR6WTUSOAMYODXHX2V62FGTKTBFQTK5JENXKR5J4HTBGIGKTTGMA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5, address:89.149.201.147:4016): protocol: rpc: dial tcp 89.149.201.147:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310894451 (327h19m12.114359047s) [noise=true , address=185.167.204.206:4058 , piece_id=WFFNR3LXSVF3HVHKHU4O6LBYVDDFGBLGCG4ZQZHOKJEECNNJIKXQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: rpc: dial tcp 185.167.204.206:4058: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310931505 (327h19m12.114396086s) [noise=true , address=185.167.204.206:4068 , piece_id=HBYTPAXUFS7ZM7OQRAKPDNITIVQOLSOH4RN362XBXOEZB4RFEGEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1sqDsiZKL7g4FbDyc9Ew2ojD1tTLzj4B19ma8hrCvWoTCy4j4D, address:185.167.204.206:4068): protocol: rpc: dial tcp 185.167.204.206:4068: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311073712 (327h19m12.114538311s) [noise=true , address=185.167.204.205:4018 , piece_id=KJA3E24NTU4J3FTWCJ2U4B5UICGUWDLS5VSTMB45SFGQNIGO3BSA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: rpc: dial tcp 185.167.204.205:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311100756 (327h19m12.114565337s) [noise=true , address=185.167.204.205:4013 , piece_id=6ZBB2XTBENBEQMD34O7LIISQHOCEOEZADNPXZOGD5XYJY4OLRHHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: write tcp 139.59.144.22:59630->185.167.204.205:4013: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311261060 (327h19m12.114725661s) [noise=true , address=81.171.14.70:4048 , piece_id=QUGF3RADQM3FHXACWUHOYCUAYYBI33B4XMXZ62JL5O7L2KIVUOUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: rpc: dial tcp 81.171.14.70:4048: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311315674 (327h19m12.114780271s) [noise=true , address=89.149.201.147:4012 , piece_id=MSSU4KAU2AR3YULH6W7JOXM6IG3NIAFU6UGX3FCFEKFLN72BSVYQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311573657 (327h19m12.115038254s) [noise=true , address=185.167.204.205:4025 , piece_id=XB2P432VMB2BJHSWNQCB4B33ZIDAR43QQD74ABRFF43MYRJZDAEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311641089 (327h19m12.11510567s) [noise=true , address=185.167.204.206:4060 , piece_id=FMVOGLOA67LN7G5JVY6NZZCNPR7PMZHBURUQGNKIRIUK7ZV4EGQA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311759265 (327h19m12.115223862s) [noise=true , address=185.167.204.205:4028 , piece_id=26AU5QE6RTN7FTB7XXOB2CGR5VJ7SIRURQUN2DB5G64EIIU6YV5Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1HZoGwBuBh9Kg7gkvH9pfrjtxQurwxdnF7pWfycGuLHrHTSq61, address:185.167.204.205:4028): protocol: rpc: dial tcp 185.167.204.205:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311824674 (327h19m12.115289254s) [noise=true , address=185.167.204.206:4062 , piece_id=WWXKTOXCQ3X5QURTWE6DJ7VC25IOK4U7LKPZ72FRCQ7UUMQMD45A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 139.59.144.22:48898->185.167.204.206:4062: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311937619 (327h19m12.115402215s) [noise=true , address=185.167.204.206:4055 , piece_id=3QNJ4SSDSMAZVSU4KKKB2XO4I24RAVORE3G375K4Z5ARSLIODWCQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CYRieezm1mjtGchWdPrB2mNaHVRvTS85umjB6cQyYKnvisXSB, address:185.167.204.206:4055): protocol: rpc: dial tcp 185.167.204.206:4055: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312048594 (327h19m12.115513172s) [noise=true , address=185.167.204.205:4022 , piece_id=UMO7EQ4TYZTTBYTOTCSD7PIWPBKDE6UQNRRWJ62UJXMAQEK7VE2Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: rpc: dial tcp 185.167.204.205:4022: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312109260 (327h19m12.115573856s) [noise=true , address=89.149.201.147:4014 , piece_id=DFU2C7O7SSA3SUZ5RFKRN5XU6BEDCJVNNOAUPRDLFXOO5WVTWPDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 139.59.144.22:45014->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312154108 (327h19m12.115618686s) [noise=true , address=185.167.204.206:4051 , piece_id=G3ZZYDQ3RA37A56XPHCN5SZP5OX32HOYS5QRK6R3IPVETEGZJYKQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: write tcp 139.59.144.22:55302->185.167.204.206:4051: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312192236 (327h19m12.115656819s) [noise=true , address=185.167.204.205:4021 , piece_id=Y3IZRKFF75CAXPCZS56QGCO65NTLFJBIQB2ZT6NND5J33FWGEENA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12FDxcFVTdQeudwXi3bRkqnBSQ7NhjChgqkCZLmEC9T4bxdGA4w, address:185.167.204.205:4021): protocol: rpc: dial tcp 185.167.204.205:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312223288 (327h19m12.115687882s) [seg_pos={0 0}, upload=288035]: Done waiting for segment. successful: 10 optimal: 26 errs: <nil>
Oct 7 04:49:24.327964660 (327h19m12.131429241s) [upload=288035]: Done uploading segments...
Oct 7 04:49:24.327992813 (327h19m12.131457395s) [upload=288035]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 10)
Duration: 386.252493ms
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
470.363128ms
FAIL (2025-10-07 05:31:15 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 10)
======LOGS======
288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.525978676 (327h18m12.329443254s) [noise=true , address=89.149.201.147:4020 , piece_id=6X366FMMBQY7UMWRZWHWK24ODNSXNRK33EBT357XTN2QGNZ2XARQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.529349437 (327h18m12.332814029s) [noise=true , address=89.149.201.147:4007 , piece_id=GISIRIBAQZ4POYEFCQIFGEORZEB3DNANYK2FRE47HJD6KCHCWFIQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.531248958 (327h18m12.334713557s) [noise=true , address=81.171.14.70:4034 , piece_id=WXUMWYCPOYMDQRAC3GCH66ORLQEW4QMXCT6Z3IZGN75QPOHRJ44A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.544080578 (327h18m12.347545162s) [noise=true , address=89.149.201.147:4005 , piece_id=UN4KYWSCWFRCG6I44PP3HG3WQYJHHRYA73RE7XKYNWFMQZ4BPNHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.556041962 (327h18m12.359506556s) [noise=true , address=81.171.14.70:4030 , piece_id=W7VSTDTWQZKAKTQ4XXJEZEWWULVUS3VWLK7GXAIC5DC5JXWGDB6A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.612240410 (327h18m12.41570499s) [noise=true , address=81.171.14.70:4025 , piece_id=EFCX2CLEJGLGAS5HY66X7YJUNZEMYDXRDTRG4TZQ25IF4J6QZPUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:49:24.310483073 (327h19m12.113947666s) [noise=true , address=185.167.204.205:4011 , piece_id=2VDETVRCNNWJC7CRL6WRZJZMZFIHI76LWIJAFRIBLMJUECKZ3SKA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CiRxadZgQLgJ41bv86hSTEHLv2CzXC4jz9Q3r5dLEX6doYoZ3, address:185.167.204.205:4011): protocol: rpc: dial tcp 185.167.204.205:4011: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310527972 (327h19m12.113992556s) [noise=true , address=81.171.14.70:4028 , piece_id=HEFFOFVIYWTKT4RBKADCIREX354RWAO5USR2VTTIVVIERFV3AKDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310706256 (327h19m12.114170848s) [noise=true , address=89.149.201.147:4016 , piece_id=NR6WTUSOAMYODXHX2V62FGTKTBFQTK5JENXKR5J4HTBGIGKTTGMA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5, address:89.149.201.147:4016): protocol: rpc: dial tcp 89.149.201.147:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310894451 (327h19m12.114359047s) [noise=true , address=185.167.204.206:4058 , piece_id=WFFNR3LXSVF3HVHKHU4O6LBYVDDFGBLGCG4ZQZHOKJEECNNJIKXQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: rpc: dial tcp 185.167.204.206:4058: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310931505 (327h19m12.114396086s) [noise=true , address=185.167.204.206:4068 , piece_id=HBYTPAXUFS7ZM7OQRAKPDNITIVQOLSOH4RN362XBXOEZB4RFEGEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1sqDsiZKL7g4FbDyc9Ew2ojD1tTLzj4B19ma8hrCvWoTCy4j4D, address:185.167.204.206:4068): protocol: rpc: dial tcp 185.167.204.206:4068: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311073712 (327h19m12.114538311s) [noise=true , address=185.167.204.205:4018 , piece_id=KJA3E24NTU4J3FTWCJ2U4B5UICGUWDLS5VSTMB45SFGQNIGO3BSA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: rpc: dial tcp 185.167.204.205:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311100756 (327h19m12.114565337s) [noise=true , address=185.167.204.205:4013 , piece_id=6ZBB2XTBENBEQMD34O7LIISQHOCEOEZADNPXZOGD5XYJY4OLRHHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: write tcp 139.59.144.22:59630->185.167.204.205:4013: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311261060 (327h19m12.114725661s) [noise=true , address=81.171.14.70:4048 , piece_id=QUGF3RADQM3FHXACWUHOYCUAYYBI33B4XMXZ62JL5O7L2KIVUOUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: rpc: dial tcp 81.171.14.70:4048: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311315674 (327h19m12.114780271s) [noise=true , address=89.149.201.147:4012 , piece_id=MSSU4KAU2AR3YULH6W7JOXM6IG3NIAFU6UGX3FCFEKFLN72BSVYQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311573657 (327h19m12.115038254s) [noise=true , address=185.167.204.205:4025 , piece_id=XB2P432VMB2BJHSWNQCB4B33ZIDAR43QQD74ABRFF43MYRJZDAEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311641089 (327h19m12.11510567s) [noise=true , address=185.167.204.206:4060 , piece_id=FMVOGLOA67LN7G5JVY6NZZCNPR7PMZHBURUQGNKIRIUK7ZV4EGQA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311759265 (327h19m12.115223862s) [noise=true , address=185.167.204.205:4028 , piece_id=26AU5QE6RTN7FTB7XXOB2CGR5VJ7SIRURQUN2DB5G64EIIU6YV5Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1HZoGwBuBh9Kg7gkvH9pfrjtxQurwxdnF7pWfycGuLHrHTSq61, address:185.167.204.205:4028): protocol: rpc: dial tcp 185.167.204.205:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311824674 (327h19m12.115289254s) [noise=true , address=185.167.204.206:4062 , piece_id=WWXKTOXCQ3X5QURTWE6DJ7VC25IOK4U7LKPZ72FRCQ7UUMQMD45A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 139.59.144.22:48898->185.167.204.206:4062: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311937619 (327h19m12.115402215s) [noise=true , address=185.167.204.206:4055 , piece_id=3QNJ4SSDSMAZVSU4KKKB2XO4I24RAVORE3G375K4Z5ARSLIODWCQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CYRieezm1mjtGchWdPrB2mNaHVRvTS85umjB6cQyYKnvisXSB, address:185.167.204.206:4055): protocol: rpc: dial tcp 185.167.204.206:4055: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312048594 (327h19m12.115513172s) [noise=true , address=185.167.204.205:4022 , piece_id=UMO7EQ4TYZTTBYTOTCSD7PIWPBKDE6UQNRRWJ62UJXMAQEK7VE2Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: rpc: dial tcp 185.167.204.205:4022: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312109260 (327h19m12.115573856s) [noise=true , address=89.149.201.147:4014 , piece_id=DFU2C7O7SSA3SUZ5RFKRN5XU6BEDCJVNNOAUPRDLFXOO5WVTWPDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 139.59.144.22:45014->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312154108 (327h19m12.115618686s) [noise=true , address=185.167.204.206:4051 , piece_id=G3ZZYDQ3RA37A56XPHCN5SZP5OX32HOYS5QRK6R3IPVETEGZJYKQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: write tcp 139.59.144.22:55302->185.167.204.206:4051: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312192236 (327h19m12.115656819s) [noise=true , address=185.167.204.205:4021 , piece_id=Y3IZRKFF75CAXPCZS56QGCO65NTLFJBIQB2ZT6NND5J33FWGEENA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12FDxcFVTdQeudwXi3bRkqnBSQ7NhjChgqkCZLmEC9T4bxdGA4w, address:185.167.204.205:4021): protocol: rpc: dial tcp 185.167.204.205:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312223288 (327h19m12.115687882s) [seg_pos={0 0}, upload=288035]: Done waiting for segment. successful: 10 optimal: 26 errs: <nil>
Oct 7 04:49:24.327964660 (327h19m12.131429241s) [upload=288035]: Done uploading segments...
Oct 7 04:49:24.327992813 (327h19m12.131457395s) [upload=288035]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 10)
Duration: 470.410685ms
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
593.007414ms
FAIL (2025-10-07 05:30:30 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 10)
======LOGS======
288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.525978676 (327h18m12.329443254s) [noise=true , address=89.149.201.147:4020 , piece_id=6X366FMMBQY7UMWRZWHWK24ODNSXNRK33EBT357XTN2QGNZ2XARQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.529349437 (327h18m12.332814029s) [noise=true , address=89.149.201.147:4007 , piece_id=GISIRIBAQZ4POYEFCQIFGEORZEB3DNANYK2FRE47HJD6KCHCWFIQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.531248958 (327h18m12.334713557s) [noise=true , address=81.171.14.70:4034 , piece_id=WXUMWYCPOYMDQRAC3GCH66ORLQEW4QMXCT6Z3IZGN75QPOHRJ44A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.544080578 (327h18m12.347545162s) [noise=true , address=89.149.201.147:4005 , piece_id=UN4KYWSCWFRCG6I44PP3HG3WQYJHHRYA73RE7XKYNWFMQZ4BPNHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.556041962 (327h18m12.359506556s) [noise=true , address=81.171.14.70:4030 , piece_id=W7VSTDTWQZKAKTQ4XXJEZEWWULVUS3VWLK7GXAIC5DC5JXWGDB6A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.612240410 (327h18m12.41570499s) [noise=true , address=81.171.14.70:4025 , piece_id=EFCX2CLEJGLGAS5HY66X7YJUNZEMYDXRDTRG4TZQ25IF4J6QZPUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:49:24.310483073 (327h19m12.113947666s) [noise=true , address=185.167.204.205:4011 , piece_id=2VDETVRCNNWJC7CRL6WRZJZMZFIHI76LWIJAFRIBLMJUECKZ3SKA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CiRxadZgQLgJ41bv86hSTEHLv2CzXC4jz9Q3r5dLEX6doYoZ3, address:185.167.204.205:4011): protocol: rpc: dial tcp 185.167.204.205:4011: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310527972 (327h19m12.113992556s) [noise=true , address=81.171.14.70:4028 , piece_id=HEFFOFVIYWTKT4RBKADCIREX354RWAO5USR2VTTIVVIERFV3AKDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310706256 (327h19m12.114170848s) [noise=true , address=89.149.201.147:4016 , piece_id=NR6WTUSOAMYODXHX2V62FGTKTBFQTK5JENXKR5J4HTBGIGKTTGMA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5, address:89.149.201.147:4016): protocol: rpc: dial tcp 89.149.201.147:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310894451 (327h19m12.114359047s) [noise=true , address=185.167.204.206:4058 , piece_id=WFFNR3LXSVF3HVHKHU4O6LBYVDDFGBLGCG4ZQZHOKJEECNNJIKXQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: rpc: dial tcp 185.167.204.206:4058: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310931505 (327h19m12.114396086s) [noise=true , address=185.167.204.206:4068 , piece_id=HBYTPAXUFS7ZM7OQRAKPDNITIVQOLSOH4RN362XBXOEZB4RFEGEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1sqDsiZKL7g4FbDyc9Ew2ojD1tTLzj4B19ma8hrCvWoTCy4j4D, address:185.167.204.206:4068): protocol: rpc: dial tcp 185.167.204.206:4068: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311073712 (327h19m12.114538311s) [noise=true , address=185.167.204.205:4018 , piece_id=KJA3E24NTU4J3FTWCJ2U4B5UICGUWDLS5VSTMB45SFGQNIGO3BSA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: rpc: dial tcp 185.167.204.205:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311100756 (327h19m12.114565337s) [noise=true , address=185.167.204.205:4013 , piece_id=6ZBB2XTBENBEQMD34O7LIISQHOCEOEZADNPXZOGD5XYJY4OLRHHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: write tcp 139.59.144.22:59630->185.167.204.205:4013: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311261060 (327h19m12.114725661s) [noise=true , address=81.171.14.70:4048 , piece_id=QUGF3RADQM3FHXACWUHOYCUAYYBI33B4XMXZ62JL5O7L2KIVUOUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: rpc: dial tcp 81.171.14.70:4048: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311315674 (327h19m12.114780271s) [noise=true , address=89.149.201.147:4012 , piece_id=MSSU4KAU2AR3YULH6W7JOXM6IG3NIAFU6UGX3FCFEKFLN72BSVYQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311573657 (327h19m12.115038254s) [noise=true , address=185.167.204.205:4025 , piece_id=XB2P432VMB2BJHSWNQCB4B33ZIDAR43QQD74ABRFF43MYRJZDAEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311641089 (327h19m12.11510567s) [noise=true , address=185.167.204.206:4060 , piece_id=FMVOGLOA67LN7G5JVY6NZZCNPR7PMZHBURUQGNKIRIUK7ZV4EGQA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311759265 (327h19m12.115223862s) [noise=true , address=185.167.204.205:4028 , piece_id=26AU5QE6RTN7FTB7XXOB2CGR5VJ7SIRURQUN2DB5G64EIIU6YV5Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1HZoGwBuBh9Kg7gkvH9pfrjtxQurwxdnF7pWfycGuLHrHTSq61, address:185.167.204.205:4028): protocol: rpc: dial tcp 185.167.204.205:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311824674 (327h19m12.115289254s) [noise=true , address=185.167.204.206:4062 , piece_id=WWXKTOXCQ3X5QURTWE6DJ7VC25IOK4U7LKPZ72FRCQ7UUMQMD45A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 139.59.144.22:48898->185.167.204.206:4062: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311937619 (327h19m12.115402215s) [noise=true , address=185.167.204.206:4055 , piece_id=3QNJ4SSDSMAZVSU4KKKB2XO4I24RAVORE3G375K4Z5ARSLIODWCQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CYRieezm1mjtGchWdPrB2mNaHVRvTS85umjB6cQyYKnvisXSB, address:185.167.204.206:4055): protocol: rpc: dial tcp 185.167.204.206:4055: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312048594 (327h19m12.115513172s) [noise=true , address=185.167.204.205:4022 , piece_id=UMO7EQ4TYZTTBYTOTCSD7PIWPBKDE6UQNRRWJ62UJXMAQEK7VE2Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: rpc: dial tcp 185.167.204.205:4022: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312109260 (327h19m12.115573856s) [noise=true , address=89.149.201.147:4014 , piece_id=DFU2C7O7SSA3SUZ5RFKRN5XU6BEDCJVNNOAUPRDLFXOO5WVTWPDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 139.59.144.22:45014->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312154108 (327h19m12.115618686s) [noise=true , address=185.167.204.206:4051 , piece_id=G3ZZYDQ3RA37A56XPHCN5SZP5OX32HOYS5QRK6R3IPVETEGZJYKQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: write tcp 139.59.144.22:55302->185.167.204.206:4051: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312192236 (327h19m12.115656819s) [noise=true , address=185.167.204.205:4021 , piece_id=Y3IZRKFF75CAXPCZS56QGCO65NTLFJBIQB2ZT6NND5J33FWGEENA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12FDxcFVTdQeudwXi3bRkqnBSQ7NhjChgqkCZLmEC9T4bxdGA4w, address:185.167.204.205:4021): protocol: rpc: dial tcp 185.167.204.205:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312223288 (327h19m12.115687882s) [seg_pos={0 0}, upload=288035]: Done waiting for segment. successful: 10 optimal: 26 errs: <nil>
Oct 7 04:49:24.327964660 (327h19m12.131429241s) [upload=288035]: Done uploading segments...
Oct 7 04:49:24.327992813 (327h19m12.131457395s) [upload=288035]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 10)
Duration: 593.048388ms
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
401.476599ms
FAIL (2025-10-07 05:29:41 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 10)
======LOGS======
288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.525978676 (327h18m12.329443254s) [noise=true , address=89.149.201.147:4020 , piece_id=6X366FMMBQY7UMWRZWHWK24ODNSXNRK33EBT357XTN2QGNZ2XARQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.529349437 (327h18m12.332814029s) [noise=true , address=89.149.201.147:4007 , piece_id=GISIRIBAQZ4POYEFCQIFGEORZEB3DNANYK2FRE47HJD6KCHCWFIQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.531248958 (327h18m12.334713557s) [noise=true , address=81.171.14.70:4034 , piece_id=WXUMWYCPOYMDQRAC3GCH66ORLQEW4QMXCT6Z3IZGN75QPOHRJ44A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.544080578 (327h18m12.347545162s) [noise=true , address=89.149.201.147:4005 , piece_id=UN4KYWSCWFRCG6I44PP3HG3WQYJHHRYA73RE7XKYNWFMQZ4BPNHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.556041962 (327h18m12.359506556s) [noise=true , address=81.171.14.70:4030 , piece_id=W7VSTDTWQZKAKTQ4XXJEZEWWULVUS3VWLK7GXAIC5DC5JXWGDB6A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.612240410 (327h18m12.41570499s) [noise=true , address=81.171.14.70:4025 , piece_id=EFCX2CLEJGLGAS5HY66X7YJUNZEMYDXRDTRG4TZQ25IF4J6QZPUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:49:24.310483073 (327h19m12.113947666s) [noise=true , address=185.167.204.205:4011 , piece_id=2VDETVRCNNWJC7CRL6WRZJZMZFIHI76LWIJAFRIBLMJUECKZ3SKA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CiRxadZgQLgJ41bv86hSTEHLv2CzXC4jz9Q3r5dLEX6doYoZ3, address:185.167.204.205:4011): protocol: rpc: dial tcp 185.167.204.205:4011: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310527972 (327h19m12.113992556s) [noise=true , address=81.171.14.70:4028 , piece_id=HEFFOFVIYWTKT4RBKADCIREX354RWAO5USR2VTTIVVIERFV3AKDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310706256 (327h19m12.114170848s) [noise=true , address=89.149.201.147:4016 , piece_id=NR6WTUSOAMYODXHX2V62FGTKTBFQTK5JENXKR5J4HTBGIGKTTGMA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5, address:89.149.201.147:4016): protocol: rpc: dial tcp 89.149.201.147:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310894451 (327h19m12.114359047s) [noise=true , address=185.167.204.206:4058 , piece_id=WFFNR3LXSVF3HVHKHU4O6LBYVDDFGBLGCG4ZQZHOKJEECNNJIKXQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: rpc: dial tcp 185.167.204.206:4058: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310931505 (327h19m12.114396086s) [noise=true , address=185.167.204.206:4068 , piece_id=HBYTPAXUFS7ZM7OQRAKPDNITIVQOLSOH4RN362XBXOEZB4RFEGEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1sqDsiZKL7g4FbDyc9Ew2ojD1tTLzj4B19ma8hrCvWoTCy4j4D, address:185.167.204.206:4068): protocol: rpc: dial tcp 185.167.204.206:4068: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311073712 (327h19m12.114538311s) [noise=true , address=185.167.204.205:4018 , piece_id=KJA3E24NTU4J3FTWCJ2U4B5UICGUWDLS5VSTMB45SFGQNIGO3BSA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: rpc: dial tcp 185.167.204.205:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311100756 (327h19m12.114565337s) [noise=true , address=185.167.204.205:4013 , piece_id=6ZBB2XTBENBEQMD34O7LIISQHOCEOEZADNPXZOGD5XYJY4OLRHHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: write tcp 139.59.144.22:59630->185.167.204.205:4013: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311261060 (327h19m12.114725661s) [noise=true , address=81.171.14.70:4048 , piece_id=QUGF3RADQM3FHXACWUHOYCUAYYBI33B4XMXZ62JL5O7L2KIVUOUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: rpc: dial tcp 81.171.14.70:4048: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311315674 (327h19m12.114780271s) [noise=true , address=89.149.201.147:4012 , piece_id=MSSU4KAU2AR3YULH6W7JOXM6IG3NIAFU6UGX3FCFEKFLN72BSVYQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311573657 (327h19m12.115038254s) [noise=true , address=185.167.204.205:4025 , piece_id=XB2P432VMB2BJHSWNQCB4B33ZIDAR43QQD74ABRFF43MYRJZDAEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311641089 (327h19m12.11510567s) [noise=true , address=185.167.204.206:4060 , piece_id=FMVOGLOA67LN7G5JVY6NZZCNPR7PMZHBURUQGNKIRIUK7ZV4EGQA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311759265 (327h19m12.115223862s) [noise=true , address=185.167.204.205:4028 , piece_id=26AU5QE6RTN7FTB7XXOB2CGR5VJ7SIRURQUN2DB5G64EIIU6YV5Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1HZoGwBuBh9Kg7gkvH9pfrjtxQurwxdnF7pWfycGuLHrHTSq61, address:185.167.204.205:4028): protocol: rpc: dial tcp 185.167.204.205:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311824674 (327h19m12.115289254s) [noise=true , address=185.167.204.206:4062 , piece_id=WWXKTOXCQ3X5QURTWE6DJ7VC25IOK4U7LKPZ72FRCQ7UUMQMD45A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 139.59.144.22:48898->185.167.204.206:4062: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311937619 (327h19m12.115402215s) [noise=true , address=185.167.204.206:4055 , piece_id=3QNJ4SSDSMAZVSU4KKKB2XO4I24RAVORE3G375K4Z5ARSLIODWCQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CYRieezm1mjtGchWdPrB2mNaHVRvTS85umjB6cQyYKnvisXSB, address:185.167.204.206:4055): protocol: rpc: dial tcp 185.167.204.206:4055: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312048594 (327h19m12.115513172s) [noise=true , address=185.167.204.205:4022 , piece_id=UMO7EQ4TYZTTBYTOTCSD7PIWPBKDE6UQNRRWJ62UJXMAQEK7VE2Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: rpc: dial tcp 185.167.204.205:4022: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312109260 (327h19m12.115573856s) [noise=true , address=89.149.201.147:4014 , piece_id=DFU2C7O7SSA3SUZ5RFKRN5XU6BEDCJVNNOAUPRDLFXOO5WVTWPDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 139.59.144.22:45014->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312154108 (327h19m12.115618686s) [noise=true , address=185.167.204.206:4051 , piece_id=G3ZZYDQ3RA37A56XPHCN5SZP5OX32HOYS5QRK6R3IPVETEGZJYKQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: write tcp 139.59.144.22:55302->185.167.204.206:4051: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312192236 (327h19m12.115656819s) [noise=true , address=185.167.204.205:4021 , piece_id=Y3IZRKFF75CAXPCZS56QGCO65NTLFJBIQB2ZT6NND5J33FWGEENA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12FDxcFVTdQeudwXi3bRkqnBSQ7NhjChgqkCZLmEC9T4bxdGA4w, address:185.167.204.205:4021): protocol: rpc: dial tcp 185.167.204.205:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312223288 (327h19m12.115687882s) [seg_pos={0 0}, upload=288035]: Done waiting for segment. successful: 10 optimal: 26 errs: <nil>
Oct 7 04:49:24.327964660 (327h19m12.131429241s) [upload=288035]: Done uploading segments...
Oct 7 04:49:24.327992813 (327h19m12.131457395s) [upload=288035]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 10)
Duration: 401.510388ms
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
385.937954ms
FAIL (2025-10-07 05:28:51 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 10)
======LOGS======
288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.525978676 (327h18m12.329443254s) [noise=true , address=89.149.201.147:4020 , piece_id=6X366FMMBQY7UMWRZWHWK24ODNSXNRK33EBT357XTN2QGNZ2XARQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.529349437 (327h18m12.332814029s) [noise=true , address=89.149.201.147:4007 , piece_id=GISIRIBAQZ4POYEFCQIFGEORZEB3DNANYK2FRE47HJD6KCHCWFIQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.531248958 (327h18m12.334713557s) [noise=true , address=81.171.14.70:4034 , piece_id=WXUMWYCPOYMDQRAC3GCH66ORLQEW4QMXCT6Z3IZGN75QPOHRJ44A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.544080578 (327h18m12.347545162s) [noise=true , address=89.149.201.147:4005 , piece_id=UN4KYWSCWFRCG6I44PP3HG3WQYJHHRYA73RE7XKYNWFMQZ4BPNHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.556041962 (327h18m12.359506556s) [noise=true , address=81.171.14.70:4030 , piece_id=W7VSTDTWQZKAKTQ4XXJEZEWWULVUS3VWLK7GXAIC5DC5JXWGDB6A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.612240410 (327h18m12.41570499s) [noise=true , address=81.171.14.70:4025 , piece_id=EFCX2CLEJGLGAS5HY66X7YJUNZEMYDXRDTRG4TZQ25IF4J6QZPUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:49:24.310483073 (327h19m12.113947666s) [noise=true , address=185.167.204.205:4011 , piece_id=2VDETVRCNNWJC7CRL6WRZJZMZFIHI76LWIJAFRIBLMJUECKZ3SKA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CiRxadZgQLgJ41bv86hSTEHLv2CzXC4jz9Q3r5dLEX6doYoZ3, address:185.167.204.205:4011): protocol: rpc: dial tcp 185.167.204.205:4011: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310527972 (327h19m12.113992556s) [noise=true , address=81.171.14.70:4028 , piece_id=HEFFOFVIYWTKT4RBKADCIREX354RWAO5USR2VTTIVVIERFV3AKDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310706256 (327h19m12.114170848s) [noise=true , address=89.149.201.147:4016 , piece_id=NR6WTUSOAMYODXHX2V62FGTKTBFQTK5JENXKR5J4HTBGIGKTTGMA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5, address:89.149.201.147:4016): protocol: rpc: dial tcp 89.149.201.147:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310894451 (327h19m12.114359047s) [noise=true , address=185.167.204.206:4058 , piece_id=WFFNR3LXSVF3HVHKHU4O6LBYVDDFGBLGCG4ZQZHOKJEECNNJIKXQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: rpc: dial tcp 185.167.204.206:4058: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310931505 (327h19m12.114396086s) [noise=true , address=185.167.204.206:4068 , piece_id=HBYTPAXUFS7ZM7OQRAKPDNITIVQOLSOH4RN362XBXOEZB4RFEGEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1sqDsiZKL7g4FbDyc9Ew2ojD1tTLzj4B19ma8hrCvWoTCy4j4D, address:185.167.204.206:4068): protocol: rpc: dial tcp 185.167.204.206:4068: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311073712 (327h19m12.114538311s) [noise=true , address=185.167.204.205:4018 , piece_id=KJA3E24NTU4J3FTWCJ2U4B5UICGUWDLS5VSTMB45SFGQNIGO3BSA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: rpc: dial tcp 185.167.204.205:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311100756 (327h19m12.114565337s) [noise=true , address=185.167.204.205:4013 , piece_id=6ZBB2XTBENBEQMD34O7LIISQHOCEOEZADNPXZOGD5XYJY4OLRHHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: write tcp 139.59.144.22:59630->185.167.204.205:4013: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311261060 (327h19m12.114725661s) [noise=true , address=81.171.14.70:4048 , piece_id=QUGF3RADQM3FHXACWUHOYCUAYYBI33B4XMXZ62JL5O7L2KIVUOUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: rpc: dial tcp 81.171.14.70:4048: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311315674 (327h19m12.114780271s) [noise=true , address=89.149.201.147:4012 , piece_id=MSSU4KAU2AR3YULH6W7JOXM6IG3NIAFU6UGX3FCFEKFLN72BSVYQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311573657 (327h19m12.115038254s) [noise=true , address=185.167.204.205:4025 , piece_id=XB2P432VMB2BJHSWNQCB4B33ZIDAR43QQD74ABRFF43MYRJZDAEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311641089 (327h19m12.11510567s) [noise=true , address=185.167.204.206:4060 , piece_id=FMVOGLOA67LN7G5JVY6NZZCNPR7PMZHBURUQGNKIRIUK7ZV4EGQA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311759265 (327h19m12.115223862s) [noise=true , address=185.167.204.205:4028 , piece_id=26AU5QE6RTN7FTB7XXOB2CGR5VJ7SIRURQUN2DB5G64EIIU6YV5Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1HZoGwBuBh9Kg7gkvH9pfrjtxQurwxdnF7pWfycGuLHrHTSq61, address:185.167.204.205:4028): protocol: rpc: dial tcp 185.167.204.205:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311824674 (327h19m12.115289254s) [noise=true , address=185.167.204.206:4062 , piece_id=WWXKTOXCQ3X5QURTWE6DJ7VC25IOK4U7LKPZ72FRCQ7UUMQMD45A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 139.59.144.22:48898->185.167.204.206:4062: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311937619 (327h19m12.115402215s) [noise=true , address=185.167.204.206:4055 , piece_id=3QNJ4SSDSMAZVSU4KKKB2XO4I24RAVORE3G375K4Z5ARSLIODWCQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CYRieezm1mjtGchWdPrB2mNaHVRvTS85umjB6cQyYKnvisXSB, address:185.167.204.206:4055): protocol: rpc: dial tcp 185.167.204.206:4055: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312048594 (327h19m12.115513172s) [noise=true , address=185.167.204.205:4022 , piece_id=UMO7EQ4TYZTTBYTOTCSD7PIWPBKDE6UQNRRWJ62UJXMAQEK7VE2Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: rpc: dial tcp 185.167.204.205:4022: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312109260 (327h19m12.115573856s) [noise=true , address=89.149.201.147:4014 , piece_id=DFU2C7O7SSA3SUZ5RFKRN5XU6BEDCJVNNOAUPRDLFXOO5WVTWPDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 139.59.144.22:45014->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312154108 (327h19m12.115618686s) [noise=true , address=185.167.204.206:4051 , piece_id=G3ZZYDQ3RA37A56XPHCN5SZP5OX32HOYS5QRK6R3IPVETEGZJYKQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: write tcp 139.59.144.22:55302->185.167.204.206:4051: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312192236 (327h19m12.115656819s) [noise=true , address=185.167.204.205:4021 , piece_id=Y3IZRKFF75CAXPCZS56QGCO65NTLFJBIQB2ZT6NND5J33FWGEENA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12FDxcFVTdQeudwXi3bRkqnBSQ7NhjChgqkCZLmEC9T4bxdGA4w, address:185.167.204.205:4021): protocol: rpc: dial tcp 185.167.204.205:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312223288 (327h19m12.115687882s) [seg_pos={0 0}, upload=288035]: Done waiting for segment. successful: 10 optimal: 26 errs: <nil>
Oct 7 04:49:24.327964660 (327h19m12.131429241s) [upload=288035]: Done uploading segments...
Oct 7 04:49:24.327992813 (327h19m12.131457395s) [upload=288035]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 10)
Duration: 385.966036ms
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
482.698558ms
FAIL (2025-10-07 05:28:03 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 10)
======LOGS======
288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.525978676 (327h18m12.329443254s) [noise=true , address=89.149.201.147:4020 , piece_id=6X366FMMBQY7UMWRZWHWK24ODNSXNRK33EBT357XTN2QGNZ2XARQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.529349437 (327h18m12.332814029s) [noise=true , address=89.149.201.147:4007 , piece_id=GISIRIBAQZ4POYEFCQIFGEORZEB3DNANYK2FRE47HJD6KCHCWFIQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.531248958 (327h18m12.334713557s) [noise=true , address=81.171.14.70:4034 , piece_id=WXUMWYCPOYMDQRAC3GCH66ORLQEW4QMXCT6Z3IZGN75QPOHRJ44A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.544080578 (327h18m12.347545162s) [noise=true , address=89.149.201.147:4005 , piece_id=UN4KYWSCWFRCG6I44PP3HG3WQYJHHRYA73RE7XKYNWFMQZ4BPNHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.556041962 (327h18m12.359506556s) [noise=true , address=81.171.14.70:4030 , piece_id=W7VSTDTWQZKAKTQ4XXJEZEWWULVUS3VWLK7GXAIC5DC5JXWGDB6A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.612240410 (327h18m12.41570499s) [noise=true , address=81.171.14.70:4025 , piece_id=EFCX2CLEJGLGAS5HY66X7YJUNZEMYDXRDTRG4TZQ25IF4J6QZPUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:49:24.310483073 (327h19m12.113947666s) [noise=true , address=185.167.204.205:4011 , piece_id=2VDETVRCNNWJC7CRL6WRZJZMZFIHI76LWIJAFRIBLMJUECKZ3SKA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CiRxadZgQLgJ41bv86hSTEHLv2CzXC4jz9Q3r5dLEX6doYoZ3, address:185.167.204.205:4011): protocol: rpc: dial tcp 185.167.204.205:4011: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310527972 (327h19m12.113992556s) [noise=true , address=81.171.14.70:4028 , piece_id=HEFFOFVIYWTKT4RBKADCIREX354RWAO5USR2VTTIVVIERFV3AKDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310706256 (327h19m12.114170848s) [noise=true , address=89.149.201.147:4016 , piece_id=NR6WTUSOAMYODXHX2V62FGTKTBFQTK5JENXKR5J4HTBGIGKTTGMA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5, address:89.149.201.147:4016): protocol: rpc: dial tcp 89.149.201.147:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310894451 (327h19m12.114359047s) [noise=true , address=185.167.204.206:4058 , piece_id=WFFNR3LXSVF3HVHKHU4O6LBYVDDFGBLGCG4ZQZHOKJEECNNJIKXQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: rpc: dial tcp 185.167.204.206:4058: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310931505 (327h19m12.114396086s) [noise=true , address=185.167.204.206:4068 , piece_id=HBYTPAXUFS7ZM7OQRAKPDNITIVQOLSOH4RN362XBXOEZB4RFEGEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1sqDsiZKL7g4FbDyc9Ew2ojD1tTLzj4B19ma8hrCvWoTCy4j4D, address:185.167.204.206:4068): protocol: rpc: dial tcp 185.167.204.206:4068: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311073712 (327h19m12.114538311s) [noise=true , address=185.167.204.205:4018 , piece_id=KJA3E24NTU4J3FTWCJ2U4B5UICGUWDLS5VSTMB45SFGQNIGO3BSA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: rpc: dial tcp 185.167.204.205:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311100756 (327h19m12.114565337s) [noise=true , address=185.167.204.205:4013 , piece_id=6ZBB2XTBENBEQMD34O7LIISQHOCEOEZADNPXZOGD5XYJY4OLRHHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: write tcp 139.59.144.22:59630->185.167.204.205:4013: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311261060 (327h19m12.114725661s) [noise=true , address=81.171.14.70:4048 , piece_id=QUGF3RADQM3FHXACWUHOYCUAYYBI33B4XMXZ62JL5O7L2KIVUOUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: rpc: dial tcp 81.171.14.70:4048: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311315674 (327h19m12.114780271s) [noise=true , address=89.149.201.147:4012 , piece_id=MSSU4KAU2AR3YULH6W7JOXM6IG3NIAFU6UGX3FCFEKFLN72BSVYQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311573657 (327h19m12.115038254s) [noise=true , address=185.167.204.205:4025 , piece_id=XB2P432VMB2BJHSWNQCB4B33ZIDAR43QQD74ABRFF43MYRJZDAEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311641089 (327h19m12.11510567s) [noise=true , address=185.167.204.206:4060 , piece_id=FMVOGLOA67LN7G5JVY6NZZCNPR7PMZHBURUQGNKIRIUK7ZV4EGQA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311759265 (327h19m12.115223862s) [noise=true , address=185.167.204.205:4028 , piece_id=26AU5QE6RTN7FTB7XXOB2CGR5VJ7SIRURQUN2DB5G64EIIU6YV5Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1HZoGwBuBh9Kg7gkvH9pfrjtxQurwxdnF7pWfycGuLHrHTSq61, address:185.167.204.205:4028): protocol: rpc: dial tcp 185.167.204.205:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311824674 (327h19m12.115289254s) [noise=true , address=185.167.204.206:4062 , piece_id=WWXKTOXCQ3X5QURTWE6DJ7VC25IOK4U7LKPZ72FRCQ7UUMQMD45A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 139.59.144.22:48898->185.167.204.206:4062: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311937619 (327h19m12.115402215s) [noise=true , address=185.167.204.206:4055 , piece_id=3QNJ4SSDSMAZVSU4KKKB2XO4I24RAVORE3G375K4Z5ARSLIODWCQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CYRieezm1mjtGchWdPrB2mNaHVRvTS85umjB6cQyYKnvisXSB, address:185.167.204.206:4055): protocol: rpc: dial tcp 185.167.204.206:4055: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312048594 (327h19m12.115513172s) [noise=true , address=185.167.204.205:4022 , piece_id=UMO7EQ4TYZTTBYTOTCSD7PIWPBKDE6UQNRRWJ62UJXMAQEK7VE2Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: rpc: dial tcp 185.167.204.205:4022: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312109260 (327h19m12.115573856s) [noise=true , address=89.149.201.147:4014 , piece_id=DFU2C7O7SSA3SUZ5RFKRN5XU6BEDCJVNNOAUPRDLFXOO5WVTWPDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 139.59.144.22:45014->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312154108 (327h19m12.115618686s) [noise=true , address=185.167.204.206:4051 , piece_id=G3ZZYDQ3RA37A56XPHCN5SZP5OX32HOYS5QRK6R3IPVETEGZJYKQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: write tcp 139.59.144.22:55302->185.167.204.206:4051: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312192236 (327h19m12.115656819s) [noise=true , address=185.167.204.205:4021 , piece_id=Y3IZRKFF75CAXPCZS56QGCO65NTLFJBIQB2ZT6NND5J33FWGEENA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12FDxcFVTdQeudwXi3bRkqnBSQ7NhjChgqkCZLmEC9T4bxdGA4w, address:185.167.204.205:4021): protocol: rpc: dial tcp 185.167.204.205:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312223288 (327h19m12.115687882s) [seg_pos={0 0}, upload=288035]: Done waiting for segment. successful: 10 optimal: 26 errs: <nil>
Oct 7 04:49:24.327964660 (327h19m12.131429241s) [upload=288035]: Done uploading segments...
Oct 7 04:49:24.327992813 (327h19m12.131457395s) [upload=288035]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 10)
Duration: 482.755642ms
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
399.809809ms
FAIL (2025-10-07 05:26:51 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 10)
======LOGS======
288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.525978676 (327h18m12.329443254s) [noise=true , address=89.149.201.147:4020 , piece_id=6X366FMMBQY7UMWRZWHWK24ODNSXNRK33EBT357XTN2QGNZ2XARQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.529349437 (327h18m12.332814029s) [noise=true , address=89.149.201.147:4007 , piece_id=GISIRIBAQZ4POYEFCQIFGEORZEB3DNANYK2FRE47HJD6KCHCWFIQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.531248958 (327h18m12.334713557s) [noise=true , address=81.171.14.70:4034 , piece_id=WXUMWYCPOYMDQRAC3GCH66ORLQEW4QMXCT6Z3IZGN75QPOHRJ44A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.544080578 (327h18m12.347545162s) [noise=true , address=89.149.201.147:4005 , piece_id=UN4KYWSCWFRCG6I44PP3HG3WQYJHHRYA73RE7XKYNWFMQZ4BPNHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.556041962 (327h18m12.359506556s) [noise=true , address=81.171.14.70:4030 , piece_id=W7VSTDTWQZKAKTQ4XXJEZEWWULVUS3VWLK7GXAIC5DC5JXWGDB6A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.612240410 (327h18m12.41570499s) [noise=true , address=81.171.14.70:4025 , piece_id=EFCX2CLEJGLGAS5HY66X7YJUNZEMYDXRDTRG4TZQ25IF4J6QZPUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:49:24.310483073 (327h19m12.113947666s) [noise=true , address=185.167.204.205:4011 , piece_id=2VDETVRCNNWJC7CRL6WRZJZMZFIHI76LWIJAFRIBLMJUECKZ3SKA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CiRxadZgQLgJ41bv86hSTEHLv2CzXC4jz9Q3r5dLEX6doYoZ3, address:185.167.204.205:4011): protocol: rpc: dial tcp 185.167.204.205:4011: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310527972 (327h19m12.113992556s) [noise=true , address=81.171.14.70:4028 , piece_id=HEFFOFVIYWTKT4RBKADCIREX354RWAO5USR2VTTIVVIERFV3AKDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310706256 (327h19m12.114170848s) [noise=true , address=89.149.201.147:4016 , piece_id=NR6WTUSOAMYODXHX2V62FGTKTBFQTK5JENXKR5J4HTBGIGKTTGMA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5, address:89.149.201.147:4016): protocol: rpc: dial tcp 89.149.201.147:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310894451 (327h19m12.114359047s) [noise=true , address=185.167.204.206:4058 , piece_id=WFFNR3LXSVF3HVHKHU4O6LBYVDDFGBLGCG4ZQZHOKJEECNNJIKXQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: rpc: dial tcp 185.167.204.206:4058: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310931505 (327h19m12.114396086s) [noise=true , address=185.167.204.206:4068 , piece_id=HBYTPAXUFS7ZM7OQRAKPDNITIVQOLSOH4RN362XBXOEZB4RFEGEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1sqDsiZKL7g4FbDyc9Ew2ojD1tTLzj4B19ma8hrCvWoTCy4j4D, address:185.167.204.206:4068): protocol: rpc: dial tcp 185.167.204.206:4068: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311073712 (327h19m12.114538311s) [noise=true , address=185.167.204.205:4018 , piece_id=KJA3E24NTU4J3FTWCJ2U4B5UICGUWDLS5VSTMB45SFGQNIGO3BSA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: rpc: dial tcp 185.167.204.205:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311100756 (327h19m12.114565337s) [noise=true , address=185.167.204.205:4013 , piece_id=6ZBB2XTBENBEQMD34O7LIISQHOCEOEZADNPXZOGD5XYJY4OLRHHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: write tcp 139.59.144.22:59630->185.167.204.205:4013: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311261060 (327h19m12.114725661s) [noise=true , address=81.171.14.70:4048 , piece_id=QUGF3RADQM3FHXACWUHOYCUAYYBI33B4XMXZ62JL5O7L2KIVUOUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: rpc: dial tcp 81.171.14.70:4048: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311315674 (327h19m12.114780271s) [noise=true , address=89.149.201.147:4012 , piece_id=MSSU4KAU2AR3YULH6W7JOXM6IG3NIAFU6UGX3FCFEKFLN72BSVYQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311573657 (327h19m12.115038254s) [noise=true , address=185.167.204.205:4025 , piece_id=XB2P432VMB2BJHSWNQCB4B33ZIDAR43QQD74ABRFF43MYRJZDAEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311641089 (327h19m12.11510567s) [noise=true , address=185.167.204.206:4060 , piece_id=FMVOGLOA67LN7G5JVY6NZZCNPR7PMZHBURUQGNKIRIUK7ZV4EGQA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311759265 (327h19m12.115223862s) [noise=true , address=185.167.204.205:4028 , piece_id=26AU5QE6RTN7FTB7XXOB2CGR5VJ7SIRURQUN2DB5G64EIIU6YV5Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1HZoGwBuBh9Kg7gkvH9pfrjtxQurwxdnF7pWfycGuLHrHTSq61, address:185.167.204.205:4028): protocol: rpc: dial tcp 185.167.204.205:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311824674 (327h19m12.115289254s) [noise=true , address=185.167.204.206:4062 , piece_id=WWXKTOXCQ3X5QURTWE6DJ7VC25IOK4U7LKPZ72FRCQ7UUMQMD45A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 139.59.144.22:48898->185.167.204.206:4062: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311937619 (327h19m12.115402215s) [noise=true , address=185.167.204.206:4055 , piece_id=3QNJ4SSDSMAZVSU4KKKB2XO4I24RAVORE3G375K4Z5ARSLIODWCQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CYRieezm1mjtGchWdPrB2mNaHVRvTS85umjB6cQyYKnvisXSB, address:185.167.204.206:4055): protocol: rpc: dial tcp 185.167.204.206:4055: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312048594 (327h19m12.115513172s) [noise=true , address=185.167.204.205:4022 , piece_id=UMO7EQ4TYZTTBYTOTCSD7PIWPBKDE6UQNRRWJ62UJXMAQEK7VE2Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: rpc: dial tcp 185.167.204.205:4022: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312109260 (327h19m12.115573856s) [noise=true , address=89.149.201.147:4014 , piece_id=DFU2C7O7SSA3SUZ5RFKRN5XU6BEDCJVNNOAUPRDLFXOO5WVTWPDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 139.59.144.22:45014->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312154108 (327h19m12.115618686s) [noise=true , address=185.167.204.206:4051 , piece_id=G3ZZYDQ3RA37A56XPHCN5SZP5OX32HOYS5QRK6R3IPVETEGZJYKQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: write tcp 139.59.144.22:55302->185.167.204.206:4051: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312192236 (327h19m12.115656819s) [noise=true , address=185.167.204.205:4021 , piece_id=Y3IZRKFF75CAXPCZS56QGCO65NTLFJBIQB2ZT6NND5J33FWGEENA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12FDxcFVTdQeudwXi3bRkqnBSQ7NhjChgqkCZLmEC9T4bxdGA4w, address:185.167.204.205:4021): protocol: rpc: dial tcp 185.167.204.205:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312223288 (327h19m12.115687882s) [seg_pos={0 0}, upload=288035]: Done waiting for segment. successful: 10 optimal: 26 errs: <nil>
Oct 7 04:49:24.327964660 (327h19m12.131429241s) [upload=288035]: Done uploading segments...
Oct 7 04:49:24.327992813 (327h19m12.131457395s) [upload=288035]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 10)
Duration: 399.848061ms
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
471.698734ms
FAIL (2025-10-07 05:25:40 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 10)
======LOGS======
288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.525978676 (327h18m12.329443254s) [noise=true , address=89.149.201.147:4020 , piece_id=6X366FMMBQY7UMWRZWHWK24ODNSXNRK33EBT357XTN2QGNZ2XARQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.529349437 (327h18m12.332814029s) [noise=true , address=89.149.201.147:4007 , piece_id=GISIRIBAQZ4POYEFCQIFGEORZEB3DNANYK2FRE47HJD6KCHCWFIQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.531248958 (327h18m12.334713557s) [noise=true , address=81.171.14.70:4034 , piece_id=WXUMWYCPOYMDQRAC3GCH66ORLQEW4QMXCT6Z3IZGN75QPOHRJ44A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.544080578 (327h18m12.347545162s) [noise=true , address=89.149.201.147:4005 , piece_id=UN4KYWSCWFRCG6I44PP3HG3WQYJHHRYA73RE7XKYNWFMQZ4BPNHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.556041962 (327h18m12.359506556s) [noise=true , address=81.171.14.70:4030 , piece_id=W7VSTDTWQZKAKTQ4XXJEZEWWULVUS3VWLK7GXAIC5DC5JXWGDB6A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.612240410 (327h18m12.41570499s) [noise=true , address=81.171.14.70:4025 , piece_id=EFCX2CLEJGLGAS5HY66X7YJUNZEMYDXRDTRG4TZQ25IF4J6QZPUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:49:24.310483073 (327h19m12.113947666s) [noise=true , address=185.167.204.205:4011 , piece_id=2VDETVRCNNWJC7CRL6WRZJZMZFIHI76LWIJAFRIBLMJUECKZ3SKA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CiRxadZgQLgJ41bv86hSTEHLv2CzXC4jz9Q3r5dLEX6doYoZ3, address:185.167.204.205:4011): protocol: rpc: dial tcp 185.167.204.205:4011: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310527972 (327h19m12.113992556s) [noise=true , address=81.171.14.70:4028 , piece_id=HEFFOFVIYWTKT4RBKADCIREX354RWAO5USR2VTTIVVIERFV3AKDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310706256 (327h19m12.114170848s) [noise=true , address=89.149.201.147:4016 , piece_id=NR6WTUSOAMYODXHX2V62FGTKTBFQTK5JENXKR5J4HTBGIGKTTGMA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5, address:89.149.201.147:4016): protocol: rpc: dial tcp 89.149.201.147:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310894451 (327h19m12.114359047s) [noise=true , address=185.167.204.206:4058 , piece_id=WFFNR3LXSVF3HVHKHU4O6LBYVDDFGBLGCG4ZQZHOKJEECNNJIKXQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: rpc: dial tcp 185.167.204.206:4058: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310931505 (327h19m12.114396086s) [noise=true , address=185.167.204.206:4068 , piece_id=HBYTPAXUFS7ZM7OQRAKPDNITIVQOLSOH4RN362XBXOEZB4RFEGEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1sqDsiZKL7g4FbDyc9Ew2ojD1tTLzj4B19ma8hrCvWoTCy4j4D, address:185.167.204.206:4068): protocol: rpc: dial tcp 185.167.204.206:4068: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311073712 (327h19m12.114538311s) [noise=true , address=185.167.204.205:4018 , piece_id=KJA3E24NTU4J3FTWCJ2U4B5UICGUWDLS5VSTMB45SFGQNIGO3BSA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: rpc: dial tcp 185.167.204.205:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311100756 (327h19m12.114565337s) [noise=true , address=185.167.204.205:4013 , piece_id=6ZBB2XTBENBEQMD34O7LIISQHOCEOEZADNPXZOGD5XYJY4OLRHHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: write tcp 139.59.144.22:59630->185.167.204.205:4013: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311261060 (327h19m12.114725661s) [noise=true , address=81.171.14.70:4048 , piece_id=QUGF3RADQM3FHXACWUHOYCUAYYBI33B4XMXZ62JL5O7L2KIVUOUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: rpc: dial tcp 81.171.14.70:4048: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311315674 (327h19m12.114780271s) [noise=true , address=89.149.201.147:4012 , piece_id=MSSU4KAU2AR3YULH6W7JOXM6IG3NIAFU6UGX3FCFEKFLN72BSVYQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311573657 (327h19m12.115038254s) [noise=true , address=185.167.204.205:4025 , piece_id=XB2P432VMB2BJHSWNQCB4B33ZIDAR43QQD74ABRFF43MYRJZDAEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311641089 (327h19m12.11510567s) [noise=true , address=185.167.204.206:4060 , piece_id=FMVOGLOA67LN7G5JVY6NZZCNPR7PMZHBURUQGNKIRIUK7ZV4EGQA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311759265 (327h19m12.115223862s) [noise=true , address=185.167.204.205:4028 , piece_id=26AU5QE6RTN7FTB7XXOB2CGR5VJ7SIRURQUN2DB5G64EIIU6YV5Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1HZoGwBuBh9Kg7gkvH9pfrjtxQurwxdnF7pWfycGuLHrHTSq61, address:185.167.204.205:4028): protocol: rpc: dial tcp 185.167.204.205:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311824674 (327h19m12.115289254s) [noise=true , address=185.167.204.206:4062 , piece_id=WWXKTOXCQ3X5QURTWE6DJ7VC25IOK4U7LKPZ72FRCQ7UUMQMD45A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 139.59.144.22:48898->185.167.204.206:4062: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311937619 (327h19m12.115402215s) [noise=true , address=185.167.204.206:4055 , piece_id=3QNJ4SSDSMAZVSU4KKKB2XO4I24RAVORE3G375K4Z5ARSLIODWCQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CYRieezm1mjtGchWdPrB2mNaHVRvTS85umjB6cQyYKnvisXSB, address:185.167.204.206:4055): protocol: rpc: dial tcp 185.167.204.206:4055: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312048594 (327h19m12.115513172s) [noise=true , address=185.167.204.205:4022 , piece_id=UMO7EQ4TYZTTBYTOTCSD7PIWPBKDE6UQNRRWJ62UJXMAQEK7VE2Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: rpc: dial tcp 185.167.204.205:4022: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312109260 (327h19m12.115573856s) [noise=true , address=89.149.201.147:4014 , piece_id=DFU2C7O7SSA3SUZ5RFKRN5XU6BEDCJVNNOAUPRDLFXOO5WVTWPDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 139.59.144.22:45014->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312154108 (327h19m12.115618686s) [noise=true , address=185.167.204.206:4051 , piece_id=G3ZZYDQ3RA37A56XPHCN5SZP5OX32HOYS5QRK6R3IPVETEGZJYKQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: write tcp 139.59.144.22:55302->185.167.204.206:4051: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312192236 (327h19m12.115656819s) [noise=true , address=185.167.204.205:4021 , piece_id=Y3IZRKFF75CAXPCZS56QGCO65NTLFJBIQB2ZT6NND5J33FWGEENA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12FDxcFVTdQeudwXi3bRkqnBSQ7NhjChgqkCZLmEC9T4bxdGA4w, address:185.167.204.205:4021): protocol: rpc: dial tcp 185.167.204.205:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312223288 (327h19m12.115687882s) [seg_pos={0 0}, upload=288035]: Done waiting for segment. successful: 10 optimal: 26 errs: <nil>
Oct 7 04:49:24.327964660 (327h19m12.131429241s) [upload=288035]: Done uploading segments...
Oct 7 04:49:24.327992813 (327h19m12.131457395s) [upload=288035]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 10)
Duration: 471.733478ms
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
853.843107ms
FAIL (2025-10-07 05:24:53 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 10)
======LOGS======
288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.525978676 (327h18m12.329443254s) [noise=true , address=89.149.201.147:4020 , piece_id=6X366FMMBQY7UMWRZWHWK24ODNSXNRK33EBT357XTN2QGNZ2XARQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.529349437 (327h18m12.332814029s) [noise=true , address=89.149.201.147:4007 , piece_id=GISIRIBAQZ4POYEFCQIFGEORZEB3DNANYK2FRE47HJD6KCHCWFIQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.531248958 (327h18m12.334713557s) [noise=true , address=81.171.14.70:4034 , piece_id=WXUMWYCPOYMDQRAC3GCH66ORLQEW4QMXCT6Z3IZGN75QPOHRJ44A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.544080578 (327h18m12.347545162s) [noise=true , address=89.149.201.147:4005 , piece_id=UN4KYWSCWFRCG6I44PP3HG3WQYJHHRYA73RE7XKYNWFMQZ4BPNHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.556041962 (327h18m12.359506556s) [noise=true , address=81.171.14.70:4030 , piece_id=W7VSTDTWQZKAKTQ4XXJEZEWWULVUS3VWLK7GXAIC5DC5JXWGDB6A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.612240410 (327h18m12.41570499s) [noise=true , address=81.171.14.70:4025 , piece_id=EFCX2CLEJGLGAS5HY66X7YJUNZEMYDXRDTRG4TZQ25IF4J6QZPUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:49:24.310483073 (327h19m12.113947666s) [noise=true , address=185.167.204.205:4011 , piece_id=2VDETVRCNNWJC7CRL6WRZJZMZFIHI76LWIJAFRIBLMJUECKZ3SKA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CiRxadZgQLgJ41bv86hSTEHLv2CzXC4jz9Q3r5dLEX6doYoZ3, address:185.167.204.205:4011): protocol: rpc: dial tcp 185.167.204.205:4011: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310527972 (327h19m12.113992556s) [noise=true , address=81.171.14.70:4028 , piece_id=HEFFOFVIYWTKT4RBKADCIREX354RWAO5USR2VTTIVVIERFV3AKDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310706256 (327h19m12.114170848s) [noise=true , address=89.149.201.147:4016 , piece_id=NR6WTUSOAMYODXHX2V62FGTKTBFQTK5JENXKR5J4HTBGIGKTTGMA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5, address:89.149.201.147:4016): protocol: rpc: dial tcp 89.149.201.147:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310894451 (327h19m12.114359047s) [noise=true , address=185.167.204.206:4058 , piece_id=WFFNR3LXSVF3HVHKHU4O6LBYVDDFGBLGCG4ZQZHOKJEECNNJIKXQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: rpc: dial tcp 185.167.204.206:4058: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310931505 (327h19m12.114396086s) [noise=true , address=185.167.204.206:4068 , piece_id=HBYTPAXUFS7ZM7OQRAKPDNITIVQOLSOH4RN362XBXOEZB4RFEGEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1sqDsiZKL7g4FbDyc9Ew2ojD1tTLzj4B19ma8hrCvWoTCy4j4D, address:185.167.204.206:4068): protocol: rpc: dial tcp 185.167.204.206:4068: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311073712 (327h19m12.114538311s) [noise=true , address=185.167.204.205:4018 , piece_id=KJA3E24NTU4J3FTWCJ2U4B5UICGUWDLS5VSTMB45SFGQNIGO3BSA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: rpc: dial tcp 185.167.204.205:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311100756 (327h19m12.114565337s) [noise=true , address=185.167.204.205:4013 , piece_id=6ZBB2XTBENBEQMD34O7LIISQHOCEOEZADNPXZOGD5XYJY4OLRHHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: write tcp 139.59.144.22:59630->185.167.204.205:4013: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311261060 (327h19m12.114725661s) [noise=true , address=81.171.14.70:4048 , piece_id=QUGF3RADQM3FHXACWUHOYCUAYYBI33B4XMXZ62JL5O7L2KIVUOUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: rpc: dial tcp 81.171.14.70:4048: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311315674 (327h19m12.114780271s) [noise=true , address=89.149.201.147:4012 , piece_id=MSSU4KAU2AR3YULH6W7JOXM6IG3NIAFU6UGX3FCFEKFLN72BSVYQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311573657 (327h19m12.115038254s) [noise=true , address=185.167.204.205:4025 , piece_id=XB2P432VMB2BJHSWNQCB4B33ZIDAR43QQD74ABRFF43MYRJZDAEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311641089 (327h19m12.11510567s) [noise=true , address=185.167.204.206:4060 , piece_id=FMVOGLOA67LN7G5JVY6NZZCNPR7PMZHBURUQGNKIRIUK7ZV4EGQA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311759265 (327h19m12.115223862s) [noise=true , address=185.167.204.205:4028 , piece_id=26AU5QE6RTN7FTB7XXOB2CGR5VJ7SIRURQUN2DB5G64EIIU6YV5Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1HZoGwBuBh9Kg7gkvH9pfrjtxQurwxdnF7pWfycGuLHrHTSq61, address:185.167.204.205:4028): protocol: rpc: dial tcp 185.167.204.205:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311824674 (327h19m12.115289254s) [noise=true , address=185.167.204.206:4062 , piece_id=WWXKTOXCQ3X5QURTWE6DJ7VC25IOK4U7LKPZ72FRCQ7UUMQMD45A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 139.59.144.22:48898->185.167.204.206:4062: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311937619 (327h19m12.115402215s) [noise=true , address=185.167.204.206:4055 , piece_id=3QNJ4SSDSMAZVSU4KKKB2XO4I24RAVORE3G375K4Z5ARSLIODWCQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CYRieezm1mjtGchWdPrB2mNaHVRvTS85umjB6cQyYKnvisXSB, address:185.167.204.206:4055): protocol: rpc: dial tcp 185.167.204.206:4055: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312048594 (327h19m12.115513172s) [noise=true , address=185.167.204.205:4022 , piece_id=UMO7EQ4TYZTTBYTOTCSD7PIWPBKDE6UQNRRWJ62UJXMAQEK7VE2Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: rpc: dial tcp 185.167.204.205:4022: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312109260 (327h19m12.115573856s) [noise=true , address=89.149.201.147:4014 , piece_id=DFU2C7O7SSA3SUZ5RFKRN5XU6BEDCJVNNOAUPRDLFXOO5WVTWPDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 139.59.144.22:45014->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312154108 (327h19m12.115618686s) [noise=true , address=185.167.204.206:4051 , piece_id=G3ZZYDQ3RA37A56XPHCN5SZP5OX32HOYS5QRK6R3IPVETEGZJYKQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: write tcp 139.59.144.22:55302->185.167.204.206:4051: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312192236 (327h19m12.115656819s) [noise=true , address=185.167.204.205:4021 , piece_id=Y3IZRKFF75CAXPCZS56QGCO65NTLFJBIQB2ZT6NND5J33FWGEENA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12FDxcFVTdQeudwXi3bRkqnBSQ7NhjChgqkCZLmEC9T4bxdGA4w, address:185.167.204.205:4021): protocol: rpc: dial tcp 185.167.204.205:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312223288 (327h19m12.115687882s) [seg_pos={0 0}, upload=288035]: Done waiting for segment. successful: 10 optimal: 26 errs: <nil>
Oct 7 04:49:24.327964660 (327h19m12.131429241s) [upload=288035]: Done uploading segments...
Oct 7 04:49:24.327992813 (327h19m12.131457395s) [upload=288035]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 10)
Duration: 853.87503ms
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
539.111352ms
FAIL (2025-10-07 05:24:02 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 10)
======LOGS======
288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.525978676 (327h18m12.329443254s) [noise=true , address=89.149.201.147:4020 , piece_id=6X366FMMBQY7UMWRZWHWK24ODNSXNRK33EBT357XTN2QGNZ2XARQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.529349437 (327h18m12.332814029s) [noise=true , address=89.149.201.147:4007 , piece_id=GISIRIBAQZ4POYEFCQIFGEORZEB3DNANYK2FRE47HJD6KCHCWFIQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.531248958 (327h18m12.334713557s) [noise=true , address=81.171.14.70:4034 , piece_id=WXUMWYCPOYMDQRAC3GCH66ORLQEW4QMXCT6Z3IZGN75QPOHRJ44A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.544080578 (327h18m12.347545162s) [noise=true , address=89.149.201.147:4005 , piece_id=UN4KYWSCWFRCG6I44PP3HG3WQYJHHRYA73RE7XKYNWFMQZ4BPNHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.556041962 (327h18m12.359506556s) [noise=true , address=81.171.14.70:4030 , piece_id=W7VSTDTWQZKAKTQ4XXJEZEWWULVUS3VWLK7GXAIC5DC5JXWGDB6A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.612240410 (327h18m12.41570499s) [noise=true , address=81.171.14.70:4025 , piece_id=EFCX2CLEJGLGAS5HY66X7YJUNZEMYDXRDTRG4TZQ25IF4J6QZPUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:49:24.310483073 (327h19m12.113947666s) [noise=true , address=185.167.204.205:4011 , piece_id=2VDETVRCNNWJC7CRL6WRZJZMZFIHI76LWIJAFRIBLMJUECKZ3SKA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CiRxadZgQLgJ41bv86hSTEHLv2CzXC4jz9Q3r5dLEX6doYoZ3, address:185.167.204.205:4011): protocol: rpc: dial tcp 185.167.204.205:4011: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310527972 (327h19m12.113992556s) [noise=true , address=81.171.14.70:4028 , piece_id=HEFFOFVIYWTKT4RBKADCIREX354RWAO5USR2VTTIVVIERFV3AKDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310706256 (327h19m12.114170848s) [noise=true , address=89.149.201.147:4016 , piece_id=NR6WTUSOAMYODXHX2V62FGTKTBFQTK5JENXKR5J4HTBGIGKTTGMA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5, address:89.149.201.147:4016): protocol: rpc: dial tcp 89.149.201.147:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310894451 (327h19m12.114359047s) [noise=true , address=185.167.204.206:4058 , piece_id=WFFNR3LXSVF3HVHKHU4O6LBYVDDFGBLGCG4ZQZHOKJEECNNJIKXQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: rpc: dial tcp 185.167.204.206:4058: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310931505 (327h19m12.114396086s) [noise=true , address=185.167.204.206:4068 , piece_id=HBYTPAXUFS7ZM7OQRAKPDNITIVQOLSOH4RN362XBXOEZB4RFEGEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1sqDsiZKL7g4FbDyc9Ew2ojD1tTLzj4B19ma8hrCvWoTCy4j4D, address:185.167.204.206:4068): protocol: rpc: dial tcp 185.167.204.206:4068: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311073712 (327h19m12.114538311s) [noise=true , address=185.167.204.205:4018 , piece_id=KJA3E24NTU4J3FTWCJ2U4B5UICGUWDLS5VSTMB45SFGQNIGO3BSA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: rpc: dial tcp 185.167.204.205:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311100756 (327h19m12.114565337s) [noise=true , address=185.167.204.205:4013 , piece_id=6ZBB2XTBENBEQMD34O7LIISQHOCEOEZADNPXZOGD5XYJY4OLRHHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: write tcp 139.59.144.22:59630->185.167.204.205:4013: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311261060 (327h19m12.114725661s) [noise=true , address=81.171.14.70:4048 , piece_id=QUGF3RADQM3FHXACWUHOYCUAYYBI33B4XMXZ62JL5O7L2KIVUOUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: rpc: dial tcp 81.171.14.70:4048: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311315674 (327h19m12.114780271s) [noise=true , address=89.149.201.147:4012 , piece_id=MSSU4KAU2AR3YULH6W7JOXM6IG3NIAFU6UGX3FCFEKFLN72BSVYQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311573657 (327h19m12.115038254s) [noise=true , address=185.167.204.205:4025 , piece_id=XB2P432VMB2BJHSWNQCB4B33ZIDAR43QQD74ABRFF43MYRJZDAEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311641089 (327h19m12.11510567s) [noise=true , address=185.167.204.206:4060 , piece_id=FMVOGLOA67LN7G5JVY6NZZCNPR7PMZHBURUQGNKIRIUK7ZV4EGQA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311759265 (327h19m12.115223862s) [noise=true , address=185.167.204.205:4028 , piece_id=26AU5QE6RTN7FTB7XXOB2CGR5VJ7SIRURQUN2DB5G64EIIU6YV5Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1HZoGwBuBh9Kg7gkvH9pfrjtxQurwxdnF7pWfycGuLHrHTSq61, address:185.167.204.205:4028): protocol: rpc: dial tcp 185.167.204.205:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311824674 (327h19m12.115289254s) [noise=true , address=185.167.204.206:4062 , piece_id=WWXKTOXCQ3X5QURTWE6DJ7VC25IOK4U7LKPZ72FRCQ7UUMQMD45A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 139.59.144.22:48898->185.167.204.206:4062: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311937619 (327h19m12.115402215s) [noise=true , address=185.167.204.206:4055 , piece_id=3QNJ4SSDSMAZVSU4KKKB2XO4I24RAVORE3G375K4Z5ARSLIODWCQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CYRieezm1mjtGchWdPrB2mNaHVRvTS85umjB6cQyYKnvisXSB, address:185.167.204.206:4055): protocol: rpc: dial tcp 185.167.204.206:4055: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312048594 (327h19m12.115513172s) [noise=true , address=185.167.204.205:4022 , piece_id=UMO7EQ4TYZTTBYTOTCSD7PIWPBKDE6UQNRRWJ62UJXMAQEK7VE2Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: rpc: dial tcp 185.167.204.205:4022: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312109260 (327h19m12.115573856s) [noise=true , address=89.149.201.147:4014 , piece_id=DFU2C7O7SSA3SUZ5RFKRN5XU6BEDCJVNNOAUPRDLFXOO5WVTWPDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 139.59.144.22:45014->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312154108 (327h19m12.115618686s) [noise=true , address=185.167.204.206:4051 , piece_id=G3ZZYDQ3RA37A56XPHCN5SZP5OX32HOYS5QRK6R3IPVETEGZJYKQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: write tcp 139.59.144.22:55302->185.167.204.206:4051: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312192236 (327h19m12.115656819s) [noise=true , address=185.167.204.205:4021 , piece_id=Y3IZRKFF75CAXPCZS56QGCO65NTLFJBIQB2ZT6NND5J33FWGEENA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12FDxcFVTdQeudwXi3bRkqnBSQ7NhjChgqkCZLmEC9T4bxdGA4w, address:185.167.204.205:4021): protocol: rpc: dial tcp 185.167.204.205:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312223288 (327h19m12.115687882s) [seg_pos={0 0}, upload=288035]: Done waiting for segment. successful: 10 optimal: 26 errs: <nil>
Oct 7 04:49:24.327964660 (327h19m12.131429241s) [upload=288035]: Done uploading segments...
Oct 7 04:49:24.327992813 (327h19m12.131457395s) [upload=288035]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 10)
Duration: 539.166587ms
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
717.013827ms
FAIL (2025-10-07 05:23:00 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 10)
======LOGS======
288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.525978676 (327h18m12.329443254s) [noise=true , address=89.149.201.147:4020 , piece_id=6X366FMMBQY7UMWRZWHWK24ODNSXNRK33EBT357XTN2QGNZ2XARQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.529349437 (327h18m12.332814029s) [noise=true , address=89.149.201.147:4007 , piece_id=GISIRIBAQZ4POYEFCQIFGEORZEB3DNANYK2FRE47HJD6KCHCWFIQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.531248958 (327h18m12.334713557s) [noise=true , address=81.171.14.70:4034 , piece_id=WXUMWYCPOYMDQRAC3GCH66ORLQEW4QMXCT6Z3IZGN75QPOHRJ44A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.544080578 (327h18m12.347545162s) [noise=true , address=89.149.201.147:4005 , piece_id=UN4KYWSCWFRCG6I44PP3HG3WQYJHHRYA73RE7XKYNWFMQZ4BPNHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.556041962 (327h18m12.359506556s) [noise=true , address=81.171.14.70:4030 , piece_id=W7VSTDTWQZKAKTQ4XXJEZEWWULVUS3VWLK7GXAIC5DC5JXWGDB6A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.612240410 (327h18m12.41570499s) [noise=true , address=81.171.14.70:4025 , piece_id=EFCX2CLEJGLGAS5HY66X7YJUNZEMYDXRDTRG4TZQ25IF4J6QZPUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:49:24.310483073 (327h19m12.113947666s) [noise=true , address=185.167.204.205:4011 , piece_id=2VDETVRCNNWJC7CRL6WRZJZMZFIHI76LWIJAFRIBLMJUECKZ3SKA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CiRxadZgQLgJ41bv86hSTEHLv2CzXC4jz9Q3r5dLEX6doYoZ3, address:185.167.204.205:4011): protocol: rpc: dial tcp 185.167.204.205:4011: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310527972 (327h19m12.113992556s) [noise=true , address=81.171.14.70:4028 , piece_id=HEFFOFVIYWTKT4RBKADCIREX354RWAO5USR2VTTIVVIERFV3AKDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310706256 (327h19m12.114170848s) [noise=true , address=89.149.201.147:4016 , piece_id=NR6WTUSOAMYODXHX2V62FGTKTBFQTK5JENXKR5J4HTBGIGKTTGMA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5, address:89.149.201.147:4016): protocol: rpc: dial tcp 89.149.201.147:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310894451 (327h19m12.114359047s) [noise=true , address=185.167.204.206:4058 , piece_id=WFFNR3LXSVF3HVHKHU4O6LBYVDDFGBLGCG4ZQZHOKJEECNNJIKXQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: rpc: dial tcp 185.167.204.206:4058: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310931505 (327h19m12.114396086s) [noise=true , address=185.167.204.206:4068 , piece_id=HBYTPAXUFS7ZM7OQRAKPDNITIVQOLSOH4RN362XBXOEZB4RFEGEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1sqDsiZKL7g4FbDyc9Ew2ojD1tTLzj4B19ma8hrCvWoTCy4j4D, address:185.167.204.206:4068): protocol: rpc: dial tcp 185.167.204.206:4068: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311073712 (327h19m12.114538311s) [noise=true , address=185.167.204.205:4018 , piece_id=KJA3E24NTU4J3FTWCJ2U4B5UICGUWDLS5VSTMB45SFGQNIGO3BSA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: rpc: dial tcp 185.167.204.205:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311100756 (327h19m12.114565337s) [noise=true , address=185.167.204.205:4013 , piece_id=6ZBB2XTBENBEQMD34O7LIISQHOCEOEZADNPXZOGD5XYJY4OLRHHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: write tcp 139.59.144.22:59630->185.167.204.205:4013: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311261060 (327h19m12.114725661s) [noise=true , address=81.171.14.70:4048 , piece_id=QUGF3RADQM3FHXACWUHOYCUAYYBI33B4XMXZ62JL5O7L2KIVUOUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: rpc: dial tcp 81.171.14.70:4048: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311315674 (327h19m12.114780271s) [noise=true , address=89.149.201.147:4012 , piece_id=MSSU4KAU2AR3YULH6W7JOXM6IG3NIAFU6UGX3FCFEKFLN72BSVYQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311573657 (327h19m12.115038254s) [noise=true , address=185.167.204.205:4025 , piece_id=XB2P432VMB2BJHSWNQCB4B33ZIDAR43QQD74ABRFF43MYRJZDAEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311641089 (327h19m12.11510567s) [noise=true , address=185.167.204.206:4060 , piece_id=FMVOGLOA67LN7G5JVY6NZZCNPR7PMZHBURUQGNKIRIUK7ZV4EGQA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311759265 (327h19m12.115223862s) [noise=true , address=185.167.204.205:4028 , piece_id=26AU5QE6RTN7FTB7XXOB2CGR5VJ7SIRURQUN2DB5G64EIIU6YV5Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1HZoGwBuBh9Kg7gkvH9pfrjtxQurwxdnF7pWfycGuLHrHTSq61, address:185.167.204.205:4028): protocol: rpc: dial tcp 185.167.204.205:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311824674 (327h19m12.115289254s) [noise=true , address=185.167.204.206:4062 , piece_id=WWXKTOXCQ3X5QURTWE6DJ7VC25IOK4U7LKPZ72FRCQ7UUMQMD45A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 139.59.144.22:48898->185.167.204.206:4062: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311937619 (327h19m12.115402215s) [noise=true , address=185.167.204.206:4055 , piece_id=3QNJ4SSDSMAZVSU4KKKB2XO4I24RAVORE3G375K4Z5ARSLIODWCQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CYRieezm1mjtGchWdPrB2mNaHVRvTS85umjB6cQyYKnvisXSB, address:185.167.204.206:4055): protocol: rpc: dial tcp 185.167.204.206:4055: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312048594 (327h19m12.115513172s) [noise=true , address=185.167.204.205:4022 , piece_id=UMO7EQ4TYZTTBYTOTCSD7PIWPBKDE6UQNRRWJ62UJXMAQEK7VE2Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: rpc: dial tcp 185.167.204.205:4022: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312109260 (327h19m12.115573856s) [noise=true , address=89.149.201.147:4014 , piece_id=DFU2C7O7SSA3SUZ5RFKRN5XU6BEDCJVNNOAUPRDLFXOO5WVTWPDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 139.59.144.22:45014->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312154108 (327h19m12.115618686s) [noise=true , address=185.167.204.206:4051 , piece_id=G3ZZYDQ3RA37A56XPHCN5SZP5OX32HOYS5QRK6R3IPVETEGZJYKQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: write tcp 139.59.144.22:55302->185.167.204.206:4051: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312192236 (327h19m12.115656819s) [noise=true , address=185.167.204.205:4021 , piece_id=Y3IZRKFF75CAXPCZS56QGCO65NTLFJBIQB2ZT6NND5J33FWGEENA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12FDxcFVTdQeudwXi3bRkqnBSQ7NhjChgqkCZLmEC9T4bxdGA4w, address:185.167.204.205:4021): protocol: rpc: dial tcp 185.167.204.205:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312223288 (327h19m12.115687882s) [seg_pos={0 0}, upload=288035]: Done waiting for segment. successful: 10 optimal: 26 errs: <nil>
Oct 7 04:49:24.327964660 (327h19m12.131429241s) [upload=288035]: Done uploading segments...
Oct 7 04:49:24.327992813 (327h19m12.131457395s) [upload=288035]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 10)
Duration: 717.053574ms
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
454.025743ms
FAIL (2025-10-07 05:21:51 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 10)
======LOGS======
288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.525978676 (327h18m12.329443254s) [noise=true , address=89.149.201.147:4020 , piece_id=6X366FMMBQY7UMWRZWHWK24ODNSXNRK33EBT357XTN2QGNZ2XARQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.529349437 (327h18m12.332814029s) [noise=true , address=89.149.201.147:4007 , piece_id=GISIRIBAQZ4POYEFCQIFGEORZEB3DNANYK2FRE47HJD6KCHCWFIQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.531248958 (327h18m12.334713557s) [noise=true , address=81.171.14.70:4034 , piece_id=WXUMWYCPOYMDQRAC3GCH66ORLQEW4QMXCT6Z3IZGN75QPOHRJ44A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.544080578 (327h18m12.347545162s) [noise=true , address=89.149.201.147:4005 , piece_id=UN4KYWSCWFRCG6I44PP3HG3WQYJHHRYA73RE7XKYNWFMQZ4BPNHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.556041962 (327h18m12.359506556s) [noise=true , address=81.171.14.70:4030 , piece_id=W7VSTDTWQZKAKTQ4XXJEZEWWULVUS3VWLK7GXAIC5DC5JXWGDB6A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.612240410 (327h18m12.41570499s) [noise=true , address=81.171.14.70:4025 , piece_id=EFCX2CLEJGLGAS5HY66X7YJUNZEMYDXRDTRG4TZQ25IF4J6QZPUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:49:24.310483073 (327h19m12.113947666s) [noise=true , address=185.167.204.205:4011 , piece_id=2VDETVRCNNWJC7CRL6WRZJZMZFIHI76LWIJAFRIBLMJUECKZ3SKA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CiRxadZgQLgJ41bv86hSTEHLv2CzXC4jz9Q3r5dLEX6doYoZ3, address:185.167.204.205:4011): protocol: rpc: dial tcp 185.167.204.205:4011: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310527972 (327h19m12.113992556s) [noise=true , address=81.171.14.70:4028 , piece_id=HEFFOFVIYWTKT4RBKADCIREX354RWAO5USR2VTTIVVIERFV3AKDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310706256 (327h19m12.114170848s) [noise=true , address=89.149.201.147:4016 , piece_id=NR6WTUSOAMYODXHX2V62FGTKTBFQTK5JENXKR5J4HTBGIGKTTGMA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5, address:89.149.201.147:4016): protocol: rpc: dial tcp 89.149.201.147:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310894451 (327h19m12.114359047s) [noise=true , address=185.167.204.206:4058 , piece_id=WFFNR3LXSVF3HVHKHU4O6LBYVDDFGBLGCG4ZQZHOKJEECNNJIKXQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: rpc: dial tcp 185.167.204.206:4058: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310931505 (327h19m12.114396086s) [noise=true , address=185.167.204.206:4068 , piece_id=HBYTPAXUFS7ZM7OQRAKPDNITIVQOLSOH4RN362XBXOEZB4RFEGEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1sqDsiZKL7g4FbDyc9Ew2ojD1tTLzj4B19ma8hrCvWoTCy4j4D, address:185.167.204.206:4068): protocol: rpc: dial tcp 185.167.204.206:4068: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311073712 (327h19m12.114538311s) [noise=true , address=185.167.204.205:4018 , piece_id=KJA3E24NTU4J3FTWCJ2U4B5UICGUWDLS5VSTMB45SFGQNIGO3BSA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: rpc: dial tcp 185.167.204.205:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311100756 (327h19m12.114565337s) [noise=true , address=185.167.204.205:4013 , piece_id=6ZBB2XTBENBEQMD34O7LIISQHOCEOEZADNPXZOGD5XYJY4OLRHHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: write tcp 139.59.144.22:59630->185.167.204.205:4013: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311261060 (327h19m12.114725661s) [noise=true , address=81.171.14.70:4048 , piece_id=QUGF3RADQM3FHXACWUHOYCUAYYBI33B4XMXZ62JL5O7L2KIVUOUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: rpc: dial tcp 81.171.14.70:4048: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311315674 (327h19m12.114780271s) [noise=true , address=89.149.201.147:4012 , piece_id=MSSU4KAU2AR3YULH6W7JOXM6IG3NIAFU6UGX3FCFEKFLN72BSVYQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311573657 (327h19m12.115038254s) [noise=true , address=185.167.204.205:4025 , piece_id=XB2P432VMB2BJHSWNQCB4B33ZIDAR43QQD74ABRFF43MYRJZDAEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311641089 (327h19m12.11510567s) [noise=true , address=185.167.204.206:4060 , piece_id=FMVOGLOA67LN7G5JVY6NZZCNPR7PMZHBURUQGNKIRIUK7ZV4EGQA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311759265 (327h19m12.115223862s) [noise=true , address=185.167.204.205:4028 , piece_id=26AU5QE6RTN7FTB7XXOB2CGR5VJ7SIRURQUN2DB5G64EIIU6YV5Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1HZoGwBuBh9Kg7gkvH9pfrjtxQurwxdnF7pWfycGuLHrHTSq61, address:185.167.204.205:4028): protocol: rpc: dial tcp 185.167.204.205:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311824674 (327h19m12.115289254s) [noise=true , address=185.167.204.206:4062 , piece_id=WWXKTOXCQ3X5QURTWE6DJ7VC25IOK4U7LKPZ72FRCQ7UUMQMD45A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 139.59.144.22:48898->185.167.204.206:4062: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311937619 (327h19m12.115402215s) [noise=true , address=185.167.204.206:4055 , piece_id=3QNJ4SSDSMAZVSU4KKKB2XO4I24RAVORE3G375K4Z5ARSLIODWCQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CYRieezm1mjtGchWdPrB2mNaHVRvTS85umjB6cQyYKnvisXSB, address:185.167.204.206:4055): protocol: rpc: dial tcp 185.167.204.206:4055: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312048594 (327h19m12.115513172s) [noise=true , address=185.167.204.205:4022 , piece_id=UMO7EQ4TYZTTBYTOTCSD7PIWPBKDE6UQNRRWJ62UJXMAQEK7VE2Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: rpc: dial tcp 185.167.204.205:4022: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312109260 (327h19m12.115573856s) [noise=true , address=89.149.201.147:4014 , piece_id=DFU2C7O7SSA3SUZ5RFKRN5XU6BEDCJVNNOAUPRDLFXOO5WVTWPDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 139.59.144.22:45014->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312154108 (327h19m12.115618686s) [noise=true , address=185.167.204.206:4051 , piece_id=G3ZZYDQ3RA37A56XPHCN5SZP5OX32HOYS5QRK6R3IPVETEGZJYKQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: write tcp 139.59.144.22:55302->185.167.204.206:4051: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312192236 (327h19m12.115656819s) [noise=true , address=185.167.204.205:4021 , piece_id=Y3IZRKFF75CAXPCZS56QGCO65NTLFJBIQB2ZT6NND5J33FWGEENA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12FDxcFVTdQeudwXi3bRkqnBSQ7NhjChgqkCZLmEC9T4bxdGA4w, address:185.167.204.205:4021): protocol: rpc: dial tcp 185.167.204.205:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312223288 (327h19m12.115687882s) [seg_pos={0 0}, upload=288035]: Done waiting for segment. successful: 10 optimal: 26 errs: <nil>
Oct 7 04:49:24.327964660 (327h19m12.131429241s) [upload=288035]: Done uploading segments...
Oct 7 04:49:24.327992813 (327h19m12.131457395s) [upload=288035]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 10)
Duration: 454.059097ms
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
721.010799ms
FAIL (2025-10-07 05:21:00 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 10)
======LOGS======
288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.525978676 (327h18m12.329443254s) [noise=true , address=89.149.201.147:4020 , piece_id=6X366FMMBQY7UMWRZWHWK24ODNSXNRK33EBT357XTN2QGNZ2XARQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.529349437 (327h18m12.332814029s) [noise=true , address=89.149.201.147:4007 , piece_id=GISIRIBAQZ4POYEFCQIFGEORZEB3DNANYK2FRE47HJD6KCHCWFIQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.531248958 (327h18m12.334713557s) [noise=true , address=81.171.14.70:4034 , piece_id=WXUMWYCPOYMDQRAC3GCH66ORLQEW4QMXCT6Z3IZGN75QPOHRJ44A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.544080578 (327h18m12.347545162s) [noise=true , address=89.149.201.147:4005 , piece_id=UN4KYWSCWFRCG6I44PP3HG3WQYJHHRYA73RE7XKYNWFMQZ4BPNHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.556041962 (327h18m12.359506556s) [noise=true , address=81.171.14.70:4030 , piece_id=W7VSTDTWQZKAKTQ4XXJEZEWWULVUS3VWLK7GXAIC5DC5JXWGDB6A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.612240410 (327h18m12.41570499s) [noise=true , address=81.171.14.70:4025 , piece_id=EFCX2CLEJGLGAS5HY66X7YJUNZEMYDXRDTRG4TZQ25IF4J6QZPUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:49:24.310483073 (327h19m12.113947666s) [noise=true , address=185.167.204.205:4011 , piece_id=2VDETVRCNNWJC7CRL6WRZJZMZFIHI76LWIJAFRIBLMJUECKZ3SKA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CiRxadZgQLgJ41bv86hSTEHLv2CzXC4jz9Q3r5dLEX6doYoZ3, address:185.167.204.205:4011): protocol: rpc: dial tcp 185.167.204.205:4011: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310527972 (327h19m12.113992556s) [noise=true , address=81.171.14.70:4028 , piece_id=HEFFOFVIYWTKT4RBKADCIREX354RWAO5USR2VTTIVVIERFV3AKDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310706256 (327h19m12.114170848s) [noise=true , address=89.149.201.147:4016 , piece_id=NR6WTUSOAMYODXHX2V62FGTKTBFQTK5JENXKR5J4HTBGIGKTTGMA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5, address:89.149.201.147:4016): protocol: rpc: dial tcp 89.149.201.147:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310894451 (327h19m12.114359047s) [noise=true , address=185.167.204.206:4058 , piece_id=WFFNR3LXSVF3HVHKHU4O6LBYVDDFGBLGCG4ZQZHOKJEECNNJIKXQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: rpc: dial tcp 185.167.204.206:4058: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310931505 (327h19m12.114396086s) [noise=true , address=185.167.204.206:4068 , piece_id=HBYTPAXUFS7ZM7OQRAKPDNITIVQOLSOH4RN362XBXOEZB4RFEGEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1sqDsiZKL7g4FbDyc9Ew2ojD1tTLzj4B19ma8hrCvWoTCy4j4D, address:185.167.204.206:4068): protocol: rpc: dial tcp 185.167.204.206:4068: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311073712 (327h19m12.114538311s) [noise=true , address=185.167.204.205:4018 , piece_id=KJA3E24NTU4J3FTWCJ2U4B5UICGUWDLS5VSTMB45SFGQNIGO3BSA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: rpc: dial tcp 185.167.204.205:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311100756 (327h19m12.114565337s) [noise=true , address=185.167.204.205:4013 , piece_id=6ZBB2XTBENBEQMD34O7LIISQHOCEOEZADNPXZOGD5XYJY4OLRHHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: write tcp 139.59.144.22:59630->185.167.204.205:4013: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311261060 (327h19m12.114725661s) [noise=true , address=81.171.14.70:4048 , piece_id=QUGF3RADQM3FHXACWUHOYCUAYYBI33B4XMXZ62JL5O7L2KIVUOUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: rpc: dial tcp 81.171.14.70:4048: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311315674 (327h19m12.114780271s) [noise=true , address=89.149.201.147:4012 , piece_id=MSSU4KAU2AR3YULH6W7JOXM6IG3NIAFU6UGX3FCFEKFLN72BSVYQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311573657 (327h19m12.115038254s) [noise=true , address=185.167.204.205:4025 , piece_id=XB2P432VMB2BJHSWNQCB4B33ZIDAR43QQD74ABRFF43MYRJZDAEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311641089 (327h19m12.11510567s) [noise=true , address=185.167.204.206:4060 , piece_id=FMVOGLOA67LN7G5JVY6NZZCNPR7PMZHBURUQGNKIRIUK7ZV4EGQA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311759265 (327h19m12.115223862s) [noise=true , address=185.167.204.205:4028 , piece_id=26AU5QE6RTN7FTB7XXOB2CGR5VJ7SIRURQUN2DB5G64EIIU6YV5Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1HZoGwBuBh9Kg7gkvH9pfrjtxQurwxdnF7pWfycGuLHrHTSq61, address:185.167.204.205:4028): protocol: rpc: dial tcp 185.167.204.205:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311824674 (327h19m12.115289254s) [noise=true , address=185.167.204.206:4062 , piece_id=WWXKTOXCQ3X5QURTWE6DJ7VC25IOK4U7LKPZ72FRCQ7UUMQMD45A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 139.59.144.22:48898->185.167.204.206:4062: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311937619 (327h19m12.115402215s) [noise=true , address=185.167.204.206:4055 , piece_id=3QNJ4SSDSMAZVSU4KKKB2XO4I24RAVORE3G375K4Z5ARSLIODWCQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CYRieezm1mjtGchWdPrB2mNaHVRvTS85umjB6cQyYKnvisXSB, address:185.167.204.206:4055): protocol: rpc: dial tcp 185.167.204.206:4055: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312048594 (327h19m12.115513172s) [noise=true , address=185.167.204.205:4022 , piece_id=UMO7EQ4TYZTTBYTOTCSD7PIWPBKDE6UQNRRWJ62UJXMAQEK7VE2Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: rpc: dial tcp 185.167.204.205:4022: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312109260 (327h19m12.115573856s) [noise=true , address=89.149.201.147:4014 , piece_id=DFU2C7O7SSA3SUZ5RFKRN5XU6BEDCJVNNOAUPRDLFXOO5WVTWPDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 139.59.144.22:45014->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312154108 (327h19m12.115618686s) [noise=true , address=185.167.204.206:4051 , piece_id=G3ZZYDQ3RA37A56XPHCN5SZP5OX32HOYS5QRK6R3IPVETEGZJYKQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: write tcp 139.59.144.22:55302->185.167.204.206:4051: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312192236 (327h19m12.115656819s) [noise=true , address=185.167.204.205:4021 , piece_id=Y3IZRKFF75CAXPCZS56QGCO65NTLFJBIQB2ZT6NND5J33FWGEENA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12FDxcFVTdQeudwXi3bRkqnBSQ7NhjChgqkCZLmEC9T4bxdGA4w, address:185.167.204.205:4021): protocol: rpc: dial tcp 185.167.204.205:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312223288 (327h19m12.115687882s) [seg_pos={0 0}, upload=288035]: Done waiting for segment. successful: 10 optimal: 26 errs: <nil>
Oct 7 04:49:24.327964660 (327h19m12.131429241s) [upload=288035]: Done uploading segments...
Oct 7 04:49:24.327992813 (327h19m12.131457395s) [upload=288035]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 10)
Duration: 721.048833ms
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
548.652201ms
FAIL (2025-10-07 05:20:41 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 10)
======LOGS======
288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.525978676 (327h18m12.329443254s) [noise=true , address=89.149.201.147:4020 , piece_id=6X366FMMBQY7UMWRZWHWK24ODNSXNRK33EBT357XTN2QGNZ2XARQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.529349437 (327h18m12.332814029s) [noise=true , address=89.149.201.147:4007 , piece_id=GISIRIBAQZ4POYEFCQIFGEORZEB3DNANYK2FRE47HJD6KCHCWFIQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.531248958 (327h18m12.334713557s) [noise=true , address=81.171.14.70:4034 , piece_id=WXUMWYCPOYMDQRAC3GCH66ORLQEW4QMXCT6Z3IZGN75QPOHRJ44A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.544080578 (327h18m12.347545162s) [noise=true , address=89.149.201.147:4005 , piece_id=UN4KYWSCWFRCG6I44PP3HG3WQYJHHRYA73RE7XKYNWFMQZ4BPNHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.556041962 (327h18m12.359506556s) [noise=true , address=81.171.14.70:4030 , piece_id=W7VSTDTWQZKAKTQ4XXJEZEWWULVUS3VWLK7GXAIC5DC5JXWGDB6A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.612240410 (327h18m12.41570499s) [noise=true , address=81.171.14.70:4025 , piece_id=EFCX2CLEJGLGAS5HY66X7YJUNZEMYDXRDTRG4TZQ25IF4J6QZPUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:49:24.310483073 (327h19m12.113947666s) [noise=true , address=185.167.204.205:4011 , piece_id=2VDETVRCNNWJC7CRL6WRZJZMZFIHI76LWIJAFRIBLMJUECKZ3SKA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CiRxadZgQLgJ41bv86hSTEHLv2CzXC4jz9Q3r5dLEX6doYoZ3, address:185.167.204.205:4011): protocol: rpc: dial tcp 185.167.204.205:4011: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310527972 (327h19m12.113992556s) [noise=true , address=81.171.14.70:4028 , piece_id=HEFFOFVIYWTKT4RBKADCIREX354RWAO5USR2VTTIVVIERFV3AKDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310706256 (327h19m12.114170848s) [noise=true , address=89.149.201.147:4016 , piece_id=NR6WTUSOAMYODXHX2V62FGTKTBFQTK5JENXKR5J4HTBGIGKTTGMA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5, address:89.149.201.147:4016): protocol: rpc: dial tcp 89.149.201.147:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310894451 (327h19m12.114359047s) [noise=true , address=185.167.204.206:4058 , piece_id=WFFNR3LXSVF3HVHKHU4O6LBYVDDFGBLGCG4ZQZHOKJEECNNJIKXQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: rpc: dial tcp 185.167.204.206:4058: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310931505 (327h19m12.114396086s) [noise=true , address=185.167.204.206:4068 , piece_id=HBYTPAXUFS7ZM7OQRAKPDNITIVQOLSOH4RN362XBXOEZB4RFEGEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1sqDsiZKL7g4FbDyc9Ew2ojD1tTLzj4B19ma8hrCvWoTCy4j4D, address:185.167.204.206:4068): protocol: rpc: dial tcp 185.167.204.206:4068: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311073712 (327h19m12.114538311s) [noise=true , address=185.167.204.205:4018 , piece_id=KJA3E24NTU4J3FTWCJ2U4B5UICGUWDLS5VSTMB45SFGQNIGO3BSA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: rpc: dial tcp 185.167.204.205:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311100756 (327h19m12.114565337s) [noise=true , address=185.167.204.205:4013 , piece_id=6ZBB2XTBENBEQMD34O7LIISQHOCEOEZADNPXZOGD5XYJY4OLRHHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: write tcp 139.59.144.22:59630->185.167.204.205:4013: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311261060 (327h19m12.114725661s) [noise=true , address=81.171.14.70:4048 , piece_id=QUGF3RADQM3FHXACWUHOYCUAYYBI33B4XMXZ62JL5O7L2KIVUOUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: rpc: dial tcp 81.171.14.70:4048: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311315674 (327h19m12.114780271s) [noise=true , address=89.149.201.147:4012 , piece_id=MSSU4KAU2AR3YULH6W7JOXM6IG3NIAFU6UGX3FCFEKFLN72BSVYQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311573657 (327h19m12.115038254s) [noise=true , address=185.167.204.205:4025 , piece_id=XB2P432VMB2BJHSWNQCB4B33ZIDAR43QQD74ABRFF43MYRJZDAEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311641089 (327h19m12.11510567s) [noise=true , address=185.167.204.206:4060 , piece_id=FMVOGLOA67LN7G5JVY6NZZCNPR7PMZHBURUQGNKIRIUK7ZV4EGQA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311759265 (327h19m12.115223862s) [noise=true , address=185.167.204.205:4028 , piece_id=26AU5QE6RTN7FTB7XXOB2CGR5VJ7SIRURQUN2DB5G64EIIU6YV5Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1HZoGwBuBh9Kg7gkvH9pfrjtxQurwxdnF7pWfycGuLHrHTSq61, address:185.167.204.205:4028): protocol: rpc: dial tcp 185.167.204.205:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311824674 (327h19m12.115289254s) [noise=true , address=185.167.204.206:4062 , piece_id=WWXKTOXCQ3X5QURTWE6DJ7VC25IOK4U7LKPZ72FRCQ7UUMQMD45A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 139.59.144.22:48898->185.167.204.206:4062: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311937619 (327h19m12.115402215s) [noise=true , address=185.167.204.206:4055 , piece_id=3QNJ4SSDSMAZVSU4KKKB2XO4I24RAVORE3G375K4Z5ARSLIODWCQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CYRieezm1mjtGchWdPrB2mNaHVRvTS85umjB6cQyYKnvisXSB, address:185.167.204.206:4055): protocol: rpc: dial tcp 185.167.204.206:4055: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312048594 (327h19m12.115513172s) [noise=true , address=185.167.204.205:4022 , piece_id=UMO7EQ4TYZTTBYTOTCSD7PIWPBKDE6UQNRRWJ62UJXMAQEK7VE2Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: rpc: dial tcp 185.167.204.205:4022: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312109260 (327h19m12.115573856s) [noise=true , address=89.149.201.147:4014 , piece_id=DFU2C7O7SSA3SUZ5RFKRN5XU6BEDCJVNNOAUPRDLFXOO5WVTWPDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 139.59.144.22:45014->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312154108 (327h19m12.115618686s) [noise=true , address=185.167.204.206:4051 , piece_id=G3ZZYDQ3RA37A56XPHCN5SZP5OX32HOYS5QRK6R3IPVETEGZJYKQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: write tcp 139.59.144.22:55302->185.167.204.206:4051: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312192236 (327h19m12.115656819s) [noise=true , address=185.167.204.205:4021 , piece_id=Y3IZRKFF75CAXPCZS56QGCO65NTLFJBIQB2ZT6NND5J33FWGEENA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12FDxcFVTdQeudwXi3bRkqnBSQ7NhjChgqkCZLmEC9T4bxdGA4w, address:185.167.204.205:4021): protocol: rpc: dial tcp 185.167.204.205:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312223288 (327h19m12.115687882s) [seg_pos={0 0}, upload=288035]: Done waiting for segment. successful: 10 optimal: 26 errs: <nil>
Oct 7 04:49:24.327964660 (327h19m12.131429241s) [upload=288035]: Done uploading segments...
Oct 7 04:49:24.327992813 (327h19m12.131457395s) [upload=288035]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 10)
Duration: 548.678247ms
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
624.074286ms
FAIL (2025-10-07 05:20:20 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 10)
======LOGS======
288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.525978676 (327h18m12.329443254s) [noise=true , address=89.149.201.147:4020 , piece_id=6X366FMMBQY7UMWRZWHWK24ODNSXNRK33EBT357XTN2QGNZ2XARQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.529349437 (327h18m12.332814029s) [noise=true , address=89.149.201.147:4007 , piece_id=GISIRIBAQZ4POYEFCQIFGEORZEB3DNANYK2FRE47HJD6KCHCWFIQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.531248958 (327h18m12.334713557s) [noise=true , address=81.171.14.70:4034 , piece_id=WXUMWYCPOYMDQRAC3GCH66ORLQEW4QMXCT6Z3IZGN75QPOHRJ44A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.544080578 (327h18m12.347545162s) [noise=true , address=89.149.201.147:4005 , piece_id=UN4KYWSCWFRCG6I44PP3HG3WQYJHHRYA73RE7XKYNWFMQZ4BPNHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.556041962 (327h18m12.359506556s) [noise=true , address=81.171.14.70:4030 , piece_id=W7VSTDTWQZKAKTQ4XXJEZEWWULVUS3VWLK7GXAIC5DC5JXWGDB6A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.612240410 (327h18m12.41570499s) [noise=true , address=81.171.14.70:4025 , piece_id=EFCX2CLEJGLGAS5HY66X7YJUNZEMYDXRDTRG4TZQ25IF4J6QZPUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:49:24.310483073 (327h19m12.113947666s) [noise=true , address=185.167.204.205:4011 , piece_id=2VDETVRCNNWJC7CRL6WRZJZMZFIHI76LWIJAFRIBLMJUECKZ3SKA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CiRxadZgQLgJ41bv86hSTEHLv2CzXC4jz9Q3r5dLEX6doYoZ3, address:185.167.204.205:4011): protocol: rpc: dial tcp 185.167.204.205:4011: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310527972 (327h19m12.113992556s) [noise=true , address=81.171.14.70:4028 , piece_id=HEFFOFVIYWTKT4RBKADCIREX354RWAO5USR2VTTIVVIERFV3AKDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310706256 (327h19m12.114170848s) [noise=true , address=89.149.201.147:4016 , piece_id=NR6WTUSOAMYODXHX2V62FGTKTBFQTK5JENXKR5J4HTBGIGKTTGMA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5, address:89.149.201.147:4016): protocol: rpc: dial tcp 89.149.201.147:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310894451 (327h19m12.114359047s) [noise=true , address=185.167.204.206:4058 , piece_id=WFFNR3LXSVF3HVHKHU4O6LBYVDDFGBLGCG4ZQZHOKJEECNNJIKXQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: rpc: dial tcp 185.167.204.206:4058: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310931505 (327h19m12.114396086s) [noise=true , address=185.167.204.206:4068 , piece_id=HBYTPAXUFS7ZM7OQRAKPDNITIVQOLSOH4RN362XBXOEZB4RFEGEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1sqDsiZKL7g4FbDyc9Ew2ojD1tTLzj4B19ma8hrCvWoTCy4j4D, address:185.167.204.206:4068): protocol: rpc: dial tcp 185.167.204.206:4068: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311073712 (327h19m12.114538311s) [noise=true , address=185.167.204.205:4018 , piece_id=KJA3E24NTU4J3FTWCJ2U4B5UICGUWDLS5VSTMB45SFGQNIGO3BSA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: rpc: dial tcp 185.167.204.205:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311100756 (327h19m12.114565337s) [noise=true , address=185.167.204.205:4013 , piece_id=6ZBB2XTBENBEQMD34O7LIISQHOCEOEZADNPXZOGD5XYJY4OLRHHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: write tcp 139.59.144.22:59630->185.167.204.205:4013: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311261060 (327h19m12.114725661s) [noise=true , address=81.171.14.70:4048 , piece_id=QUGF3RADQM3FHXACWUHOYCUAYYBI33B4XMXZ62JL5O7L2KIVUOUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: rpc: dial tcp 81.171.14.70:4048: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311315674 (327h19m12.114780271s) [noise=true , address=89.149.201.147:4012 , piece_id=MSSU4KAU2AR3YULH6W7JOXM6IG3NIAFU6UGX3FCFEKFLN72BSVYQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311573657 (327h19m12.115038254s) [noise=true , address=185.167.204.205:4025 , piece_id=XB2P432VMB2BJHSWNQCB4B33ZIDAR43QQD74ABRFF43MYRJZDAEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311641089 (327h19m12.11510567s) [noise=true , address=185.167.204.206:4060 , piece_id=FMVOGLOA67LN7G5JVY6NZZCNPR7PMZHBURUQGNKIRIUK7ZV4EGQA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311759265 (327h19m12.115223862s) [noise=true , address=185.167.204.205:4028 , piece_id=26AU5QE6RTN7FTB7XXOB2CGR5VJ7SIRURQUN2DB5G64EIIU6YV5Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1HZoGwBuBh9Kg7gkvH9pfrjtxQurwxdnF7pWfycGuLHrHTSq61, address:185.167.204.205:4028): protocol: rpc: dial tcp 185.167.204.205:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311824674 (327h19m12.115289254s) [noise=true , address=185.167.204.206:4062 , piece_id=WWXKTOXCQ3X5QURTWE6DJ7VC25IOK4U7LKPZ72FRCQ7UUMQMD45A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 139.59.144.22:48898->185.167.204.206:4062: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311937619 (327h19m12.115402215s) [noise=true , address=185.167.204.206:4055 , piece_id=3QNJ4SSDSMAZVSU4KKKB2XO4I24RAVORE3G375K4Z5ARSLIODWCQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CYRieezm1mjtGchWdPrB2mNaHVRvTS85umjB6cQyYKnvisXSB, address:185.167.204.206:4055): protocol: rpc: dial tcp 185.167.204.206:4055: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312048594 (327h19m12.115513172s) [noise=true , address=185.167.204.205:4022 , piece_id=UMO7EQ4TYZTTBYTOTCSD7PIWPBKDE6UQNRRWJ62UJXMAQEK7VE2Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: rpc: dial tcp 185.167.204.205:4022: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312109260 (327h19m12.115573856s) [noise=true , address=89.149.201.147:4014 , piece_id=DFU2C7O7SSA3SUZ5RFKRN5XU6BEDCJVNNOAUPRDLFXOO5WVTWPDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 139.59.144.22:45014->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312154108 (327h19m12.115618686s) [noise=true , address=185.167.204.206:4051 , piece_id=G3ZZYDQ3RA37A56XPHCN5SZP5OX32HOYS5QRK6R3IPVETEGZJYKQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: write tcp 139.59.144.22:55302->185.167.204.206:4051: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312192236 (327h19m12.115656819s) [noise=true , address=185.167.204.205:4021 , piece_id=Y3IZRKFF75CAXPCZS56QGCO65NTLFJBIQB2ZT6NND5J33FWGEENA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12FDxcFVTdQeudwXi3bRkqnBSQ7NhjChgqkCZLmEC9T4bxdGA4w, address:185.167.204.205:4021): protocol: rpc: dial tcp 185.167.204.205:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312223288 (327h19m12.115687882s) [seg_pos={0 0}, upload=288035]: Done waiting for segment. successful: 10 optimal: 26 errs: <nil>
Oct 7 04:49:24.327964660 (327h19m12.131429241s) [upload=288035]: Done uploading segments...
Oct 7 04:49:24.327992813 (327h19m12.131457395s) [upload=288035]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 10)
Duration: 624.105961ms
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
439.903574ms
FAIL (2025-10-07 05:19:22 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 10)
======LOGS======
288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.525978676 (327h18m12.329443254s) [noise=true , address=89.149.201.147:4020 , piece_id=6X366FMMBQY7UMWRZWHWK24ODNSXNRK33EBT357XTN2QGNZ2XARQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.529349437 (327h18m12.332814029s) [noise=true , address=89.149.201.147:4007 , piece_id=GISIRIBAQZ4POYEFCQIFGEORZEB3DNANYK2FRE47HJD6KCHCWFIQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.531248958 (327h18m12.334713557s) [noise=true , address=81.171.14.70:4034 , piece_id=WXUMWYCPOYMDQRAC3GCH66ORLQEW4QMXCT6Z3IZGN75QPOHRJ44A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.544080578 (327h18m12.347545162s) [noise=true , address=89.149.201.147:4005 , piece_id=UN4KYWSCWFRCG6I44PP3HG3WQYJHHRYA73RE7XKYNWFMQZ4BPNHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.556041962 (327h18m12.359506556s) [noise=true , address=81.171.14.70:4030 , piece_id=W7VSTDTWQZKAKTQ4XXJEZEWWULVUS3VWLK7GXAIC5DC5JXWGDB6A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.612240410 (327h18m12.41570499s) [noise=true , address=81.171.14.70:4025 , piece_id=EFCX2CLEJGLGAS5HY66X7YJUNZEMYDXRDTRG4TZQ25IF4J6QZPUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:49:24.310483073 (327h19m12.113947666s) [noise=true , address=185.167.204.205:4011 , piece_id=2VDETVRCNNWJC7CRL6WRZJZMZFIHI76LWIJAFRIBLMJUECKZ3SKA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CiRxadZgQLgJ41bv86hSTEHLv2CzXC4jz9Q3r5dLEX6doYoZ3, address:185.167.204.205:4011): protocol: rpc: dial tcp 185.167.204.205:4011: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310527972 (327h19m12.113992556s) [noise=true , address=81.171.14.70:4028 , piece_id=HEFFOFVIYWTKT4RBKADCIREX354RWAO5USR2VTTIVVIERFV3AKDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310706256 (327h19m12.114170848s) [noise=true , address=89.149.201.147:4016 , piece_id=NR6WTUSOAMYODXHX2V62FGTKTBFQTK5JENXKR5J4HTBGIGKTTGMA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5, address:89.149.201.147:4016): protocol: rpc: dial tcp 89.149.201.147:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310894451 (327h19m12.114359047s) [noise=true , address=185.167.204.206:4058 , piece_id=WFFNR3LXSVF3HVHKHU4O6LBYVDDFGBLGCG4ZQZHOKJEECNNJIKXQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: rpc: dial tcp 185.167.204.206:4058: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310931505 (327h19m12.114396086s) [noise=true , address=185.167.204.206:4068 , piece_id=HBYTPAXUFS7ZM7OQRAKPDNITIVQOLSOH4RN362XBXOEZB4RFEGEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1sqDsiZKL7g4FbDyc9Ew2ojD1tTLzj4B19ma8hrCvWoTCy4j4D, address:185.167.204.206:4068): protocol: rpc: dial tcp 185.167.204.206:4068: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311073712 (327h19m12.114538311s) [noise=true , address=185.167.204.205:4018 , piece_id=KJA3E24NTU4J3FTWCJ2U4B5UICGUWDLS5VSTMB45SFGQNIGO3BSA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: rpc: dial tcp 185.167.204.205:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311100756 (327h19m12.114565337s) [noise=true , address=185.167.204.205:4013 , piece_id=6ZBB2XTBENBEQMD34O7LIISQHOCEOEZADNPXZOGD5XYJY4OLRHHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: write tcp 139.59.144.22:59630->185.167.204.205:4013: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311261060 (327h19m12.114725661s) [noise=true , address=81.171.14.70:4048 , piece_id=QUGF3RADQM3FHXACWUHOYCUAYYBI33B4XMXZ62JL5O7L2KIVUOUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: rpc: dial tcp 81.171.14.70:4048: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311315674 (327h19m12.114780271s) [noise=true , address=89.149.201.147:4012 , piece_id=MSSU4KAU2AR3YULH6W7JOXM6IG3NIAFU6UGX3FCFEKFLN72BSVYQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311573657 (327h19m12.115038254s) [noise=true , address=185.167.204.205:4025 , piece_id=XB2P432VMB2BJHSWNQCB4B33ZIDAR43QQD74ABRFF43MYRJZDAEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311641089 (327h19m12.11510567s) [noise=true , address=185.167.204.206:4060 , piece_id=FMVOGLOA67LN7G5JVY6NZZCNPR7PMZHBURUQGNKIRIUK7ZV4EGQA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311759265 (327h19m12.115223862s) [noise=true , address=185.167.204.205:4028 , piece_id=26AU5QE6RTN7FTB7XXOB2CGR5VJ7SIRURQUN2DB5G64EIIU6YV5Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1HZoGwBuBh9Kg7gkvH9pfrjtxQurwxdnF7pWfycGuLHrHTSq61, address:185.167.204.205:4028): protocol: rpc: dial tcp 185.167.204.205:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311824674 (327h19m12.115289254s) [noise=true , address=185.167.204.206:4062 , piece_id=WWXKTOXCQ3X5QURTWE6DJ7VC25IOK4U7LKPZ72FRCQ7UUMQMD45A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 139.59.144.22:48898->185.167.204.206:4062: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311937619 (327h19m12.115402215s) [noise=true , address=185.167.204.206:4055 , piece_id=3QNJ4SSDSMAZVSU4KKKB2XO4I24RAVORE3G375K4Z5ARSLIODWCQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CYRieezm1mjtGchWdPrB2mNaHVRvTS85umjB6cQyYKnvisXSB, address:185.167.204.206:4055): protocol: rpc: dial tcp 185.167.204.206:4055: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312048594 (327h19m12.115513172s) [noise=true , address=185.167.204.205:4022 , piece_id=UMO7EQ4TYZTTBYTOTCSD7PIWPBKDE6UQNRRWJ62UJXMAQEK7VE2Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: rpc: dial tcp 185.167.204.205:4022: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312109260 (327h19m12.115573856s) [noise=true , address=89.149.201.147:4014 , piece_id=DFU2C7O7SSA3SUZ5RFKRN5XU6BEDCJVNNOAUPRDLFXOO5WVTWPDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 139.59.144.22:45014->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312154108 (327h19m12.115618686s) [noise=true , address=185.167.204.206:4051 , piece_id=G3ZZYDQ3RA37A56XPHCN5SZP5OX32HOYS5QRK6R3IPVETEGZJYKQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: write tcp 139.59.144.22:55302->185.167.204.206:4051: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312192236 (327h19m12.115656819s) [noise=true , address=185.167.204.205:4021 , piece_id=Y3IZRKFF75CAXPCZS56QGCO65NTLFJBIQB2ZT6NND5J33FWGEENA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12FDxcFVTdQeudwXi3bRkqnBSQ7NhjChgqkCZLmEC9T4bxdGA4w, address:185.167.204.205:4021): protocol: rpc: dial tcp 185.167.204.205:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312223288 (327h19m12.115687882s) [seg_pos={0 0}, upload=288035]: Done waiting for segment. successful: 10 optimal: 26 errs: <nil>
Oct 7 04:49:24.327964660 (327h19m12.131429241s) [upload=288035]: Done uploading segments...
Oct 7 04:49:24.327992813 (327h19m12.131457395s) [upload=288035]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 10)
Duration: 439.93105ms
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
450.45569ms
FAIL (2025-10-07 05:18:37 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 10)
======LOGS======
288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.525978676 (327h18m12.329443254s) [noise=true , address=89.149.201.147:4020 , piece_id=6X366FMMBQY7UMWRZWHWK24ODNSXNRK33EBT357XTN2QGNZ2XARQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.529349437 (327h18m12.332814029s) [noise=true , address=89.149.201.147:4007 , piece_id=GISIRIBAQZ4POYEFCQIFGEORZEB3DNANYK2FRE47HJD6KCHCWFIQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.531248958 (327h18m12.334713557s) [noise=true , address=81.171.14.70:4034 , piece_id=WXUMWYCPOYMDQRAC3GCH66ORLQEW4QMXCT6Z3IZGN75QPOHRJ44A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.544080578 (327h18m12.347545162s) [noise=true , address=89.149.201.147:4005 , piece_id=UN4KYWSCWFRCG6I44PP3HG3WQYJHHRYA73RE7XKYNWFMQZ4BPNHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.556041962 (327h18m12.359506556s) [noise=true , address=81.171.14.70:4030 , piece_id=W7VSTDTWQZKAKTQ4XXJEZEWWULVUS3VWLK7GXAIC5DC5JXWGDB6A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.612240410 (327h18m12.41570499s) [noise=true , address=81.171.14.70:4025 , piece_id=EFCX2CLEJGLGAS5HY66X7YJUNZEMYDXRDTRG4TZQ25IF4J6QZPUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:49:24.310483073 (327h19m12.113947666s) [noise=true , address=185.167.204.205:4011 , piece_id=2VDETVRCNNWJC7CRL6WRZJZMZFIHI76LWIJAFRIBLMJUECKZ3SKA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CiRxadZgQLgJ41bv86hSTEHLv2CzXC4jz9Q3r5dLEX6doYoZ3, address:185.167.204.205:4011): protocol: rpc: dial tcp 185.167.204.205:4011: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310527972 (327h19m12.113992556s) [noise=true , address=81.171.14.70:4028 , piece_id=HEFFOFVIYWTKT4RBKADCIREX354RWAO5USR2VTTIVVIERFV3AKDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310706256 (327h19m12.114170848s) [noise=true , address=89.149.201.147:4016 , piece_id=NR6WTUSOAMYODXHX2V62FGTKTBFQTK5JENXKR5J4HTBGIGKTTGMA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5, address:89.149.201.147:4016): protocol: rpc: dial tcp 89.149.201.147:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310894451 (327h19m12.114359047s) [noise=true , address=185.167.204.206:4058 , piece_id=WFFNR3LXSVF3HVHKHU4O6LBYVDDFGBLGCG4ZQZHOKJEECNNJIKXQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: rpc: dial tcp 185.167.204.206:4058: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310931505 (327h19m12.114396086s) [noise=true , address=185.167.204.206:4068 , piece_id=HBYTPAXUFS7ZM7OQRAKPDNITIVQOLSOH4RN362XBXOEZB4RFEGEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1sqDsiZKL7g4FbDyc9Ew2ojD1tTLzj4B19ma8hrCvWoTCy4j4D, address:185.167.204.206:4068): protocol: rpc: dial tcp 185.167.204.206:4068: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311073712 (327h19m12.114538311s) [noise=true , address=185.167.204.205:4018 , piece_id=KJA3E24NTU4J3FTWCJ2U4B5UICGUWDLS5VSTMB45SFGQNIGO3BSA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: rpc: dial tcp 185.167.204.205:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311100756 (327h19m12.114565337s) [noise=true , address=185.167.204.205:4013 , piece_id=6ZBB2XTBENBEQMD34O7LIISQHOCEOEZADNPXZOGD5XYJY4OLRHHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: write tcp 139.59.144.22:59630->185.167.204.205:4013: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311261060 (327h19m12.114725661s) [noise=true , address=81.171.14.70:4048 , piece_id=QUGF3RADQM3FHXACWUHOYCUAYYBI33B4XMXZ62JL5O7L2KIVUOUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: rpc: dial tcp 81.171.14.70:4048: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311315674 (327h19m12.114780271s) [noise=true , address=89.149.201.147:4012 , piece_id=MSSU4KAU2AR3YULH6W7JOXM6IG3NIAFU6UGX3FCFEKFLN72BSVYQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311573657 (327h19m12.115038254s) [noise=true , address=185.167.204.205:4025 , piece_id=XB2P432VMB2BJHSWNQCB4B33ZIDAR43QQD74ABRFF43MYRJZDAEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311641089 (327h19m12.11510567s) [noise=true , address=185.167.204.206:4060 , piece_id=FMVOGLOA67LN7G5JVY6NZZCNPR7PMZHBURUQGNKIRIUK7ZV4EGQA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311759265 (327h19m12.115223862s) [noise=true , address=185.167.204.205:4028 , piece_id=26AU5QE6RTN7FTB7XXOB2CGR5VJ7SIRURQUN2DB5G64EIIU6YV5Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1HZoGwBuBh9Kg7gkvH9pfrjtxQurwxdnF7pWfycGuLHrHTSq61, address:185.167.204.205:4028): protocol: rpc: dial tcp 185.167.204.205:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311824674 (327h19m12.115289254s) [noise=true , address=185.167.204.206:4062 , piece_id=WWXKTOXCQ3X5QURTWE6DJ7VC25IOK4U7LKPZ72FRCQ7UUMQMD45A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 139.59.144.22:48898->185.167.204.206:4062: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311937619 (327h19m12.115402215s) [noise=true , address=185.167.204.206:4055 , piece_id=3QNJ4SSDSMAZVSU4KKKB2XO4I24RAVORE3G375K4Z5ARSLIODWCQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CYRieezm1mjtGchWdPrB2mNaHVRvTS85umjB6cQyYKnvisXSB, address:185.167.204.206:4055): protocol: rpc: dial tcp 185.167.204.206:4055: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312048594 (327h19m12.115513172s) [noise=true , address=185.167.204.205:4022 , piece_id=UMO7EQ4TYZTTBYTOTCSD7PIWPBKDE6UQNRRWJ62UJXMAQEK7VE2Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: rpc: dial tcp 185.167.204.205:4022: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312109260 (327h19m12.115573856s) [noise=true , address=89.149.201.147:4014 , piece_id=DFU2C7O7SSA3SUZ5RFKRN5XU6BEDCJVNNOAUPRDLFXOO5WVTWPDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 139.59.144.22:45014->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312154108 (327h19m12.115618686s) [noise=true , address=185.167.204.206:4051 , piece_id=G3ZZYDQ3RA37A56XPHCN5SZP5OX32HOYS5QRK6R3IPVETEGZJYKQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: write tcp 139.59.144.22:55302->185.167.204.206:4051: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312192236 (327h19m12.115656819s) [noise=true , address=185.167.204.205:4021 , piece_id=Y3IZRKFF75CAXPCZS56QGCO65NTLFJBIQB2ZT6NND5J33FWGEENA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12FDxcFVTdQeudwXi3bRkqnBSQ7NhjChgqkCZLmEC9T4bxdGA4w, address:185.167.204.205:4021): protocol: rpc: dial tcp 185.167.204.205:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312223288 (327h19m12.115687882s) [seg_pos={0 0}, upload=288035]: Done waiting for segment. successful: 10 optimal: 26 errs: <nil>
Oct 7 04:49:24.327964660 (327h19m12.131429241s) [upload=288035]: Done uploading segments...
Oct 7 04:49:24.327992813 (327h19m12.131457395s) [upload=288035]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 10)
Duration: 450.488707ms
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
481.483962ms
FAIL (2025-10-07 05:18:15 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 10)
======LOGS======
288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.525978676 (327h18m12.329443254s) [noise=true , address=89.149.201.147:4020 , piece_id=6X366FMMBQY7UMWRZWHWK24ODNSXNRK33EBT357XTN2QGNZ2XARQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.529349437 (327h18m12.332814029s) [noise=true , address=89.149.201.147:4007 , piece_id=GISIRIBAQZ4POYEFCQIFGEORZEB3DNANYK2FRE47HJD6KCHCWFIQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.531248958 (327h18m12.334713557s) [noise=true , address=81.171.14.70:4034 , piece_id=WXUMWYCPOYMDQRAC3GCH66ORLQEW4QMXCT6Z3IZGN75QPOHRJ44A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.544080578 (327h18m12.347545162s) [noise=true , address=89.149.201.147:4005 , piece_id=UN4KYWSCWFRCG6I44PP3HG3WQYJHHRYA73RE7XKYNWFMQZ4BPNHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.556041962 (327h18m12.359506556s) [noise=true , address=81.171.14.70:4030 , piece_id=W7VSTDTWQZKAKTQ4XXJEZEWWULVUS3VWLK7GXAIC5DC5JXWGDB6A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.612240410 (327h18m12.41570499s) [noise=true , address=81.171.14.70:4025 , piece_id=EFCX2CLEJGLGAS5HY66X7YJUNZEMYDXRDTRG4TZQ25IF4J6QZPUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:49:24.310483073 (327h19m12.113947666s) [noise=true , address=185.167.204.205:4011 , piece_id=2VDETVRCNNWJC7CRL6WRZJZMZFIHI76LWIJAFRIBLMJUECKZ3SKA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CiRxadZgQLgJ41bv86hSTEHLv2CzXC4jz9Q3r5dLEX6doYoZ3, address:185.167.204.205:4011): protocol: rpc: dial tcp 185.167.204.205:4011: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310527972 (327h19m12.113992556s) [noise=true , address=81.171.14.70:4028 , piece_id=HEFFOFVIYWTKT4RBKADCIREX354RWAO5USR2VTTIVVIERFV3AKDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310706256 (327h19m12.114170848s) [noise=true , address=89.149.201.147:4016 , piece_id=NR6WTUSOAMYODXHX2V62FGTKTBFQTK5JENXKR5J4HTBGIGKTTGMA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5, address:89.149.201.147:4016): protocol: rpc: dial tcp 89.149.201.147:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310894451 (327h19m12.114359047s) [noise=true , address=185.167.204.206:4058 , piece_id=WFFNR3LXSVF3HVHKHU4O6LBYVDDFGBLGCG4ZQZHOKJEECNNJIKXQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: rpc: dial tcp 185.167.204.206:4058: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310931505 (327h19m12.114396086s) [noise=true , address=185.167.204.206:4068 , piece_id=HBYTPAXUFS7ZM7OQRAKPDNITIVQOLSOH4RN362XBXOEZB4RFEGEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1sqDsiZKL7g4FbDyc9Ew2ojD1tTLzj4B19ma8hrCvWoTCy4j4D, address:185.167.204.206:4068): protocol: rpc: dial tcp 185.167.204.206:4068: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311073712 (327h19m12.114538311s) [noise=true , address=185.167.204.205:4018 , piece_id=KJA3E24NTU4J3FTWCJ2U4B5UICGUWDLS5VSTMB45SFGQNIGO3BSA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: rpc: dial tcp 185.167.204.205:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311100756 (327h19m12.114565337s) [noise=true , address=185.167.204.205:4013 , piece_id=6ZBB2XTBENBEQMD34O7LIISQHOCEOEZADNPXZOGD5XYJY4OLRHHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: write tcp 139.59.144.22:59630->185.167.204.205:4013: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311261060 (327h19m12.114725661s) [noise=true , address=81.171.14.70:4048 , piece_id=QUGF3RADQM3FHXACWUHOYCUAYYBI33B4XMXZ62JL5O7L2KIVUOUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: rpc: dial tcp 81.171.14.70:4048: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311315674 (327h19m12.114780271s) [noise=true , address=89.149.201.147:4012 , piece_id=MSSU4KAU2AR3YULH6W7JOXM6IG3NIAFU6UGX3FCFEKFLN72BSVYQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311573657 (327h19m12.115038254s) [noise=true , address=185.167.204.205:4025 , piece_id=XB2P432VMB2BJHSWNQCB4B33ZIDAR43QQD74ABRFF43MYRJZDAEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311641089 (327h19m12.11510567s) [noise=true , address=185.167.204.206:4060 , piece_id=FMVOGLOA67LN7G5JVY6NZZCNPR7PMZHBURUQGNKIRIUK7ZV4EGQA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311759265 (327h19m12.115223862s) [noise=true , address=185.167.204.205:4028 , piece_id=26AU5QE6RTN7FTB7XXOB2CGR5VJ7SIRURQUN2DB5G64EIIU6YV5Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1HZoGwBuBh9Kg7gkvH9pfrjtxQurwxdnF7pWfycGuLHrHTSq61, address:185.167.204.205:4028): protocol: rpc: dial tcp 185.167.204.205:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311824674 (327h19m12.115289254s) [noise=true , address=185.167.204.206:4062 , piece_id=WWXKTOXCQ3X5QURTWE6DJ7VC25IOK4U7LKPZ72FRCQ7UUMQMD45A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 139.59.144.22:48898->185.167.204.206:4062: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311937619 (327h19m12.115402215s) [noise=true , address=185.167.204.206:4055 , piece_id=3QNJ4SSDSMAZVSU4KKKB2XO4I24RAVORE3G375K4Z5ARSLIODWCQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CYRieezm1mjtGchWdPrB2mNaHVRvTS85umjB6cQyYKnvisXSB, address:185.167.204.206:4055): protocol: rpc: dial tcp 185.167.204.206:4055: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312048594 (327h19m12.115513172s) [noise=true , address=185.167.204.205:4022 , piece_id=UMO7EQ4TYZTTBYTOTCSD7PIWPBKDE6UQNRRWJ62UJXMAQEK7VE2Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: rpc: dial tcp 185.167.204.205:4022: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312109260 (327h19m12.115573856s) [noise=true , address=89.149.201.147:4014 , piece_id=DFU2C7O7SSA3SUZ5RFKRN5XU6BEDCJVNNOAUPRDLFXOO5WVTWPDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 139.59.144.22:45014->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312154108 (327h19m12.115618686s) [noise=true , address=185.167.204.206:4051 , piece_id=G3ZZYDQ3RA37A56XPHCN5SZP5OX32HOYS5QRK6R3IPVETEGZJYKQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: write tcp 139.59.144.22:55302->185.167.204.206:4051: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312192236 (327h19m12.115656819s) [noise=true , address=185.167.204.205:4021 , piece_id=Y3IZRKFF75CAXPCZS56QGCO65NTLFJBIQB2ZT6NND5J33FWGEENA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12FDxcFVTdQeudwXi3bRkqnBSQ7NhjChgqkCZLmEC9T4bxdGA4w, address:185.167.204.205:4021): protocol: rpc: dial tcp 185.167.204.205:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312223288 (327h19m12.115687882s) [seg_pos={0 0}, upload=288035]: Done waiting for segment. successful: 10 optimal: 26 errs: <nil>
Oct 7 04:49:24.327964660 (327h19m12.131429241s) [upload=288035]: Done uploading segments...
Oct 7 04:49:24.327992813 (327h19m12.131457395s) [upload=288035]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 10)
Duration: 481.51012ms
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
620.649219ms
FAIL (2025-10-07 05:17:02 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 10)
======LOGS======
288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.525978676 (327h18m12.329443254s) [noise=true , address=89.149.201.147:4020 , piece_id=6X366FMMBQY7UMWRZWHWK24ODNSXNRK33EBT357XTN2QGNZ2XARQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.529349437 (327h18m12.332814029s) [noise=true , address=89.149.201.147:4007 , piece_id=GISIRIBAQZ4POYEFCQIFGEORZEB3DNANYK2FRE47HJD6KCHCWFIQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.531248958 (327h18m12.334713557s) [noise=true , address=81.171.14.70:4034 , piece_id=WXUMWYCPOYMDQRAC3GCH66ORLQEW4QMXCT6Z3IZGN75QPOHRJ44A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.544080578 (327h18m12.347545162s) [noise=true , address=89.149.201.147:4005 , piece_id=UN4KYWSCWFRCG6I44PP3HG3WQYJHHRYA73RE7XKYNWFMQZ4BPNHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.556041962 (327h18m12.359506556s) [noise=true , address=81.171.14.70:4030 , piece_id=W7VSTDTWQZKAKTQ4XXJEZEWWULVUS3VWLK7GXAIC5DC5JXWGDB6A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.612240410 (327h18m12.41570499s) [noise=true , address=81.171.14.70:4025 , piece_id=EFCX2CLEJGLGAS5HY66X7YJUNZEMYDXRDTRG4TZQ25IF4J6QZPUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:49:24.310483073 (327h19m12.113947666s) [noise=true , address=185.167.204.205:4011 , piece_id=2VDETVRCNNWJC7CRL6WRZJZMZFIHI76LWIJAFRIBLMJUECKZ3SKA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CiRxadZgQLgJ41bv86hSTEHLv2CzXC4jz9Q3r5dLEX6doYoZ3, address:185.167.204.205:4011): protocol: rpc: dial tcp 185.167.204.205:4011: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310527972 (327h19m12.113992556s) [noise=true , address=81.171.14.70:4028 , piece_id=HEFFOFVIYWTKT4RBKADCIREX354RWAO5USR2VTTIVVIERFV3AKDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310706256 (327h19m12.114170848s) [noise=true , address=89.149.201.147:4016 , piece_id=NR6WTUSOAMYODXHX2V62FGTKTBFQTK5JENXKR5J4HTBGIGKTTGMA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5, address:89.149.201.147:4016): protocol: rpc: dial tcp 89.149.201.147:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310894451 (327h19m12.114359047s) [noise=true , address=185.167.204.206:4058 , piece_id=WFFNR3LXSVF3HVHKHU4O6LBYVDDFGBLGCG4ZQZHOKJEECNNJIKXQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: rpc: dial tcp 185.167.204.206:4058: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310931505 (327h19m12.114396086s) [noise=true , address=185.167.204.206:4068 , piece_id=HBYTPAXUFS7ZM7OQRAKPDNITIVQOLSOH4RN362XBXOEZB4RFEGEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1sqDsiZKL7g4FbDyc9Ew2ojD1tTLzj4B19ma8hrCvWoTCy4j4D, address:185.167.204.206:4068): protocol: rpc: dial tcp 185.167.204.206:4068: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311073712 (327h19m12.114538311s) [noise=true , address=185.167.204.205:4018 , piece_id=KJA3E24NTU4J3FTWCJ2U4B5UICGUWDLS5VSTMB45SFGQNIGO3BSA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: rpc: dial tcp 185.167.204.205:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311100756 (327h19m12.114565337s) [noise=true , address=185.167.204.205:4013 , piece_id=6ZBB2XTBENBEQMD34O7LIISQHOCEOEZADNPXZOGD5XYJY4OLRHHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: write tcp 139.59.144.22:59630->185.167.204.205:4013: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311261060 (327h19m12.114725661s) [noise=true , address=81.171.14.70:4048 , piece_id=QUGF3RADQM3FHXACWUHOYCUAYYBI33B4XMXZ62JL5O7L2KIVUOUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: rpc: dial tcp 81.171.14.70:4048: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311315674 (327h19m12.114780271s) [noise=true , address=89.149.201.147:4012 , piece_id=MSSU4KAU2AR3YULH6W7JOXM6IG3NIAFU6UGX3FCFEKFLN72BSVYQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311573657 (327h19m12.115038254s) [noise=true , address=185.167.204.205:4025 , piece_id=XB2P432VMB2BJHSWNQCB4B33ZIDAR43QQD74ABRFF43MYRJZDAEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311641089 (327h19m12.11510567s) [noise=true , address=185.167.204.206:4060 , piece_id=FMVOGLOA67LN7G5JVY6NZZCNPR7PMZHBURUQGNKIRIUK7ZV4EGQA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311759265 (327h19m12.115223862s) [noise=true , address=185.167.204.205:4028 , piece_id=26AU5QE6RTN7FTB7XXOB2CGR5VJ7SIRURQUN2DB5G64EIIU6YV5Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1HZoGwBuBh9Kg7gkvH9pfrjtxQurwxdnF7pWfycGuLHrHTSq61, address:185.167.204.205:4028): protocol: rpc: dial tcp 185.167.204.205:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311824674 (327h19m12.115289254s) [noise=true , address=185.167.204.206:4062 , piece_id=WWXKTOXCQ3X5QURTWE6DJ7VC25IOK4U7LKPZ72FRCQ7UUMQMD45A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 139.59.144.22:48898->185.167.204.206:4062: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311937619 (327h19m12.115402215s) [noise=true , address=185.167.204.206:4055 , piece_id=3QNJ4SSDSMAZVSU4KKKB2XO4I24RAVORE3G375K4Z5ARSLIODWCQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CYRieezm1mjtGchWdPrB2mNaHVRvTS85umjB6cQyYKnvisXSB, address:185.167.204.206:4055): protocol: rpc: dial tcp 185.167.204.206:4055: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312048594 (327h19m12.115513172s) [noise=true , address=185.167.204.205:4022 , piece_id=UMO7EQ4TYZTTBYTOTCSD7PIWPBKDE6UQNRRWJ62UJXMAQEK7VE2Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: rpc: dial tcp 185.167.204.205:4022: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312109260 (327h19m12.115573856s) [noise=true , address=89.149.201.147:4014 , piece_id=DFU2C7O7SSA3SUZ5RFKRN5XU6BEDCJVNNOAUPRDLFXOO5WVTWPDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 139.59.144.22:45014->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312154108 (327h19m12.115618686s) [noise=true , address=185.167.204.206:4051 , piece_id=G3ZZYDQ3RA37A56XPHCN5SZP5OX32HOYS5QRK6R3IPVETEGZJYKQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: write tcp 139.59.144.22:55302->185.167.204.206:4051: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312192236 (327h19m12.115656819s) [noise=true , address=185.167.204.205:4021 , piece_id=Y3IZRKFF75CAXPCZS56QGCO65NTLFJBIQB2ZT6NND5J33FWGEENA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12FDxcFVTdQeudwXi3bRkqnBSQ7NhjChgqkCZLmEC9T4bxdGA4w, address:185.167.204.205:4021): protocol: rpc: dial tcp 185.167.204.205:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312223288 (327h19m12.115687882s) [seg_pos={0 0}, upload=288035]: Done waiting for segment. successful: 10 optimal: 26 errs: <nil>
Oct 7 04:49:24.327964660 (327h19m12.131429241s) [upload=288035]: Done uploading segments...
Oct 7 04:49:24.327992813 (327h19m12.131457395s) [upload=288035]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 10)
Duration: 620.750343ms
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
506.984448ms
FAIL (2025-10-07 05:16:03 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 10)
======LOGS======
288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.525978676 (327h18m12.329443254s) [noise=true , address=89.149.201.147:4020 , piece_id=6X366FMMBQY7UMWRZWHWK24ODNSXNRK33EBT357XTN2QGNZ2XARQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.529349437 (327h18m12.332814029s) [noise=true , address=89.149.201.147:4007 , piece_id=GISIRIBAQZ4POYEFCQIFGEORZEB3DNANYK2FRE47HJD6KCHCWFIQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.531248958 (327h18m12.334713557s) [noise=true , address=81.171.14.70:4034 , piece_id=WXUMWYCPOYMDQRAC3GCH66ORLQEW4QMXCT6Z3IZGN75QPOHRJ44A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.544080578 (327h18m12.347545162s) [noise=true , address=89.149.201.147:4005 , piece_id=UN4KYWSCWFRCG6I44PP3HG3WQYJHHRYA73RE7XKYNWFMQZ4BPNHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.556041962 (327h18m12.359506556s) [noise=true , address=81.171.14.70:4030 , piece_id=W7VSTDTWQZKAKTQ4XXJEZEWWULVUS3VWLK7GXAIC5DC5JXWGDB6A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.612240410 (327h18m12.41570499s) [noise=true , address=81.171.14.70:4025 , piece_id=EFCX2CLEJGLGAS5HY66X7YJUNZEMYDXRDTRG4TZQ25IF4J6QZPUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:49:24.310483073 (327h19m12.113947666s) [noise=true , address=185.167.204.205:4011 , piece_id=2VDETVRCNNWJC7CRL6WRZJZMZFIHI76LWIJAFRIBLMJUECKZ3SKA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CiRxadZgQLgJ41bv86hSTEHLv2CzXC4jz9Q3r5dLEX6doYoZ3, address:185.167.204.205:4011): protocol: rpc: dial tcp 185.167.204.205:4011: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310527972 (327h19m12.113992556s) [noise=true , address=81.171.14.70:4028 , piece_id=HEFFOFVIYWTKT4RBKADCIREX354RWAO5USR2VTTIVVIERFV3AKDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310706256 (327h19m12.114170848s) [noise=true , address=89.149.201.147:4016 , piece_id=NR6WTUSOAMYODXHX2V62FGTKTBFQTK5JENXKR5J4HTBGIGKTTGMA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5, address:89.149.201.147:4016): protocol: rpc: dial tcp 89.149.201.147:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310894451 (327h19m12.114359047s) [noise=true , address=185.167.204.206:4058 , piece_id=WFFNR3LXSVF3HVHKHU4O6LBYVDDFGBLGCG4ZQZHOKJEECNNJIKXQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: rpc: dial tcp 185.167.204.206:4058: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310931505 (327h19m12.114396086s) [noise=true , address=185.167.204.206:4068 , piece_id=HBYTPAXUFS7ZM7OQRAKPDNITIVQOLSOH4RN362XBXOEZB4RFEGEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1sqDsiZKL7g4FbDyc9Ew2ojD1tTLzj4B19ma8hrCvWoTCy4j4D, address:185.167.204.206:4068): protocol: rpc: dial tcp 185.167.204.206:4068: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311073712 (327h19m12.114538311s) [noise=true , address=185.167.204.205:4018 , piece_id=KJA3E24NTU4J3FTWCJ2U4B5UICGUWDLS5VSTMB45SFGQNIGO3BSA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: rpc: dial tcp 185.167.204.205:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311100756 (327h19m12.114565337s) [noise=true , address=185.167.204.205:4013 , piece_id=6ZBB2XTBENBEQMD34O7LIISQHOCEOEZADNPXZOGD5XYJY4OLRHHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: write tcp 139.59.144.22:59630->185.167.204.205:4013: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311261060 (327h19m12.114725661s) [noise=true , address=81.171.14.70:4048 , piece_id=QUGF3RADQM3FHXACWUHOYCUAYYBI33B4XMXZ62JL5O7L2KIVUOUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: rpc: dial tcp 81.171.14.70:4048: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311315674 (327h19m12.114780271s) [noise=true , address=89.149.201.147:4012 , piece_id=MSSU4KAU2AR3YULH6W7JOXM6IG3NIAFU6UGX3FCFEKFLN72BSVYQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311573657 (327h19m12.115038254s) [noise=true , address=185.167.204.205:4025 , piece_id=XB2P432VMB2BJHSWNQCB4B33ZIDAR43QQD74ABRFF43MYRJZDAEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311641089 (327h19m12.11510567s) [noise=true , address=185.167.204.206:4060 , piece_id=FMVOGLOA67LN7G5JVY6NZZCNPR7PMZHBURUQGNKIRIUK7ZV4EGQA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311759265 (327h19m12.115223862s) [noise=true , address=185.167.204.205:4028 , piece_id=26AU5QE6RTN7FTB7XXOB2CGR5VJ7SIRURQUN2DB5G64EIIU6YV5Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1HZoGwBuBh9Kg7gkvH9pfrjtxQurwxdnF7pWfycGuLHrHTSq61, address:185.167.204.205:4028): protocol: rpc: dial tcp 185.167.204.205:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311824674 (327h19m12.115289254s) [noise=true , address=185.167.204.206:4062 , piece_id=WWXKTOXCQ3X5QURTWE6DJ7VC25IOK4U7LKPZ72FRCQ7UUMQMD45A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 139.59.144.22:48898->185.167.204.206:4062: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311937619 (327h19m12.115402215s) [noise=true , address=185.167.204.206:4055 , piece_id=3QNJ4SSDSMAZVSU4KKKB2XO4I24RAVORE3G375K4Z5ARSLIODWCQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CYRieezm1mjtGchWdPrB2mNaHVRvTS85umjB6cQyYKnvisXSB, address:185.167.204.206:4055): protocol: rpc: dial tcp 185.167.204.206:4055: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312048594 (327h19m12.115513172s) [noise=true , address=185.167.204.205:4022 , piece_id=UMO7EQ4TYZTTBYTOTCSD7PIWPBKDE6UQNRRWJ62UJXMAQEK7VE2Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: rpc: dial tcp 185.167.204.205:4022: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312109260 (327h19m12.115573856s) [noise=true , address=89.149.201.147:4014 , piece_id=DFU2C7O7SSA3SUZ5RFKRN5XU6BEDCJVNNOAUPRDLFXOO5WVTWPDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 139.59.144.22:45014->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312154108 (327h19m12.115618686s) [noise=true , address=185.167.204.206:4051 , piece_id=G3ZZYDQ3RA37A56XPHCN5SZP5OX32HOYS5QRK6R3IPVETEGZJYKQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: write tcp 139.59.144.22:55302->185.167.204.206:4051: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312192236 (327h19m12.115656819s) [noise=true , address=185.167.204.205:4021 , piece_id=Y3IZRKFF75CAXPCZS56QGCO65NTLFJBIQB2ZT6NND5J33FWGEENA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12FDxcFVTdQeudwXi3bRkqnBSQ7NhjChgqkCZLmEC9T4bxdGA4w, address:185.167.204.205:4021): protocol: rpc: dial tcp 185.167.204.205:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312223288 (327h19m12.115687882s) [seg_pos={0 0}, upload=288035]: Done waiting for segment. successful: 10 optimal: 26 errs: <nil>
Oct 7 04:49:24.327964660 (327h19m12.131429241s) [upload=288035]: Done uploading segments...
Oct 7 04:49:24.327992813 (327h19m12.131457395s) [upload=288035]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 10)
Duration: 507.099135ms
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
636.46086ms
FAIL (2025-10-07 05:15:45 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 10)
======LOGS======
288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.525978676 (327h18m12.329443254s) [noise=true , address=89.149.201.147:4020 , piece_id=6X366FMMBQY7UMWRZWHWK24ODNSXNRK33EBT357XTN2QGNZ2XARQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.529349437 (327h18m12.332814029s) [noise=true , address=89.149.201.147:4007 , piece_id=GISIRIBAQZ4POYEFCQIFGEORZEB3DNANYK2FRE47HJD6KCHCWFIQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.531248958 (327h18m12.334713557s) [noise=true , address=81.171.14.70:4034 , piece_id=WXUMWYCPOYMDQRAC3GCH66ORLQEW4QMXCT6Z3IZGN75QPOHRJ44A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.544080578 (327h18m12.347545162s) [noise=true , address=89.149.201.147:4005 , piece_id=UN4KYWSCWFRCG6I44PP3HG3WQYJHHRYA73RE7XKYNWFMQZ4BPNHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.556041962 (327h18m12.359506556s) [noise=true , address=81.171.14.70:4030 , piece_id=W7VSTDTWQZKAKTQ4XXJEZEWWULVUS3VWLK7GXAIC5DC5JXWGDB6A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.612240410 (327h18m12.41570499s) [noise=true , address=81.171.14.70:4025 , piece_id=EFCX2CLEJGLGAS5HY66X7YJUNZEMYDXRDTRG4TZQ25IF4J6QZPUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:49:24.310483073 (327h19m12.113947666s) [noise=true , address=185.167.204.205:4011 , piece_id=2VDETVRCNNWJC7CRL6WRZJZMZFIHI76LWIJAFRIBLMJUECKZ3SKA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CiRxadZgQLgJ41bv86hSTEHLv2CzXC4jz9Q3r5dLEX6doYoZ3, address:185.167.204.205:4011): protocol: rpc: dial tcp 185.167.204.205:4011: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310527972 (327h19m12.113992556s) [noise=true , address=81.171.14.70:4028 , piece_id=HEFFOFVIYWTKT4RBKADCIREX354RWAO5USR2VTTIVVIERFV3AKDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310706256 (327h19m12.114170848s) [noise=true , address=89.149.201.147:4016 , piece_id=NR6WTUSOAMYODXHX2V62FGTKTBFQTK5JENXKR5J4HTBGIGKTTGMA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5, address:89.149.201.147:4016): protocol: rpc: dial tcp 89.149.201.147:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310894451 (327h19m12.114359047s) [noise=true , address=185.167.204.206:4058 , piece_id=WFFNR3LXSVF3HVHKHU4O6LBYVDDFGBLGCG4ZQZHOKJEECNNJIKXQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: rpc: dial tcp 185.167.204.206:4058: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310931505 (327h19m12.114396086s) [noise=true , address=185.167.204.206:4068 , piece_id=HBYTPAXUFS7ZM7OQRAKPDNITIVQOLSOH4RN362XBXOEZB4RFEGEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1sqDsiZKL7g4FbDyc9Ew2ojD1tTLzj4B19ma8hrCvWoTCy4j4D, address:185.167.204.206:4068): protocol: rpc: dial tcp 185.167.204.206:4068: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311073712 (327h19m12.114538311s) [noise=true , address=185.167.204.205:4018 , piece_id=KJA3E24NTU4J3FTWCJ2U4B5UICGUWDLS5VSTMB45SFGQNIGO3BSA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: rpc: dial tcp 185.167.204.205:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311100756 (327h19m12.114565337s) [noise=true , address=185.167.204.205:4013 , piece_id=6ZBB2XTBENBEQMD34O7LIISQHOCEOEZADNPXZOGD5XYJY4OLRHHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: write tcp 139.59.144.22:59630->185.167.204.205:4013: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311261060 (327h19m12.114725661s) [noise=true , address=81.171.14.70:4048 , piece_id=QUGF3RADQM3FHXACWUHOYCUAYYBI33B4XMXZ62JL5O7L2KIVUOUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: rpc: dial tcp 81.171.14.70:4048: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311315674 (327h19m12.114780271s) [noise=true , address=89.149.201.147:4012 , piece_id=MSSU4KAU2AR3YULH6W7JOXM6IG3NIAFU6UGX3FCFEKFLN72BSVYQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311573657 (327h19m12.115038254s) [noise=true , address=185.167.204.205:4025 , piece_id=XB2P432VMB2BJHSWNQCB4B33ZIDAR43QQD74ABRFF43MYRJZDAEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311641089 (327h19m12.11510567s) [noise=true , address=185.167.204.206:4060 , piece_id=FMVOGLOA67LN7G5JVY6NZZCNPR7PMZHBURUQGNKIRIUK7ZV4EGQA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311759265 (327h19m12.115223862s) [noise=true , address=185.167.204.205:4028 , piece_id=26AU5QE6RTN7FTB7XXOB2CGR5VJ7SIRURQUN2DB5G64EIIU6YV5Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1HZoGwBuBh9Kg7gkvH9pfrjtxQurwxdnF7pWfycGuLHrHTSq61, address:185.167.204.205:4028): protocol: rpc: dial tcp 185.167.204.205:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311824674 (327h19m12.115289254s) [noise=true , address=185.167.204.206:4062 , piece_id=WWXKTOXCQ3X5QURTWE6DJ7VC25IOK4U7LKPZ72FRCQ7UUMQMD45A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 139.59.144.22:48898->185.167.204.206:4062: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311937619 (327h19m12.115402215s) [noise=true , address=185.167.204.206:4055 , piece_id=3QNJ4SSDSMAZVSU4KKKB2XO4I24RAVORE3G375K4Z5ARSLIODWCQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CYRieezm1mjtGchWdPrB2mNaHVRvTS85umjB6cQyYKnvisXSB, address:185.167.204.206:4055): protocol: rpc: dial tcp 185.167.204.206:4055: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312048594 (327h19m12.115513172s) [noise=true , address=185.167.204.205:4022 , piece_id=UMO7EQ4TYZTTBYTOTCSD7PIWPBKDE6UQNRRWJ62UJXMAQEK7VE2Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: rpc: dial tcp 185.167.204.205:4022: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312109260 (327h19m12.115573856s) [noise=true , address=89.149.201.147:4014 , piece_id=DFU2C7O7SSA3SUZ5RFKRN5XU6BEDCJVNNOAUPRDLFXOO5WVTWPDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 139.59.144.22:45014->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312154108 (327h19m12.115618686s) [noise=true , address=185.167.204.206:4051 , piece_id=G3ZZYDQ3RA37A56XPHCN5SZP5OX32HOYS5QRK6R3IPVETEGZJYKQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: write tcp 139.59.144.22:55302->185.167.204.206:4051: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312192236 (327h19m12.115656819s) [noise=true , address=185.167.204.205:4021 , piece_id=Y3IZRKFF75CAXPCZS56QGCO65NTLFJBIQB2ZT6NND5J33FWGEENA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12FDxcFVTdQeudwXi3bRkqnBSQ7NhjChgqkCZLmEC9T4bxdGA4w, address:185.167.204.205:4021): protocol: rpc: dial tcp 185.167.204.205:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312223288 (327h19m12.115687882s) [seg_pos={0 0}, upload=288035]: Done waiting for segment. successful: 10 optimal: 26 errs: <nil>
Oct 7 04:49:24.327964660 (327h19m12.131429241s) [upload=288035]: Done uploading segments...
Oct 7 04:49:24.327992813 (327h19m12.131457395s) [upload=288035]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 10)
Duration: 636.494005ms
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
409.622685ms
FAIL (2025-10-07 05:15:08 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 10)
======LOGS======
288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.525978676 (327h18m12.329443254s) [noise=true , address=89.149.201.147:4020 , piece_id=6X366FMMBQY7UMWRZWHWK24ODNSXNRK33EBT357XTN2QGNZ2XARQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.529349437 (327h18m12.332814029s) [noise=true , address=89.149.201.147:4007 , piece_id=GISIRIBAQZ4POYEFCQIFGEORZEB3DNANYK2FRE47HJD6KCHCWFIQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.531248958 (327h18m12.334713557s) [noise=true , address=81.171.14.70:4034 , piece_id=WXUMWYCPOYMDQRAC3GCH66ORLQEW4QMXCT6Z3IZGN75QPOHRJ44A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.544080578 (327h18m12.347545162s) [noise=true , address=89.149.201.147:4005 , piece_id=UN4KYWSCWFRCG6I44PP3HG3WQYJHHRYA73RE7XKYNWFMQZ4BPNHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.556041962 (327h18m12.359506556s) [noise=true , address=81.171.14.70:4030 , piece_id=W7VSTDTWQZKAKTQ4XXJEZEWWULVUS3VWLK7GXAIC5DC5JXWGDB6A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.612240410 (327h18m12.41570499s) [noise=true , address=81.171.14.70:4025 , piece_id=EFCX2CLEJGLGAS5HY66X7YJUNZEMYDXRDTRG4TZQ25IF4J6QZPUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:49:24.310483073 (327h19m12.113947666s) [noise=true , address=185.167.204.205:4011 , piece_id=2VDETVRCNNWJC7CRL6WRZJZMZFIHI76LWIJAFRIBLMJUECKZ3SKA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CiRxadZgQLgJ41bv86hSTEHLv2CzXC4jz9Q3r5dLEX6doYoZ3, address:185.167.204.205:4011): protocol: rpc: dial tcp 185.167.204.205:4011: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310527972 (327h19m12.113992556s) [noise=true , address=81.171.14.70:4028 , piece_id=HEFFOFVIYWTKT4RBKADCIREX354RWAO5USR2VTTIVVIERFV3AKDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310706256 (327h19m12.114170848s) [noise=true , address=89.149.201.147:4016 , piece_id=NR6WTUSOAMYODXHX2V62FGTKTBFQTK5JENXKR5J4HTBGIGKTTGMA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5, address:89.149.201.147:4016): protocol: rpc: dial tcp 89.149.201.147:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310894451 (327h19m12.114359047s) [noise=true , address=185.167.204.206:4058 , piece_id=WFFNR3LXSVF3HVHKHU4O6LBYVDDFGBLGCG4ZQZHOKJEECNNJIKXQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: rpc: dial tcp 185.167.204.206:4058: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310931505 (327h19m12.114396086s) [noise=true , address=185.167.204.206:4068 , piece_id=HBYTPAXUFS7ZM7OQRAKPDNITIVQOLSOH4RN362XBXOEZB4RFEGEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1sqDsiZKL7g4FbDyc9Ew2ojD1tTLzj4B19ma8hrCvWoTCy4j4D, address:185.167.204.206:4068): protocol: rpc: dial tcp 185.167.204.206:4068: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311073712 (327h19m12.114538311s) [noise=true , address=185.167.204.205:4018 , piece_id=KJA3E24NTU4J3FTWCJ2U4B5UICGUWDLS5VSTMB45SFGQNIGO3BSA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: rpc: dial tcp 185.167.204.205:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311100756 (327h19m12.114565337s) [noise=true , address=185.167.204.205:4013 , piece_id=6ZBB2XTBENBEQMD34O7LIISQHOCEOEZADNPXZOGD5XYJY4OLRHHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: write tcp 139.59.144.22:59630->185.167.204.205:4013: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311261060 (327h19m12.114725661s) [noise=true , address=81.171.14.70:4048 , piece_id=QUGF3RADQM3FHXACWUHOYCUAYYBI33B4XMXZ62JL5O7L2KIVUOUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: rpc: dial tcp 81.171.14.70:4048: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311315674 (327h19m12.114780271s) [noise=true , address=89.149.201.147:4012 , piece_id=MSSU4KAU2AR3YULH6W7JOXM6IG3NIAFU6UGX3FCFEKFLN72BSVYQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311573657 (327h19m12.115038254s) [noise=true , address=185.167.204.205:4025 , piece_id=XB2P432VMB2BJHSWNQCB4B33ZIDAR43QQD74ABRFF43MYRJZDAEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311641089 (327h19m12.11510567s) [noise=true , address=185.167.204.206:4060 , piece_id=FMVOGLOA67LN7G5JVY6NZZCNPR7PMZHBURUQGNKIRIUK7ZV4EGQA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311759265 (327h19m12.115223862s) [noise=true , address=185.167.204.205:4028 , piece_id=26AU5QE6RTN7FTB7XXOB2CGR5VJ7SIRURQUN2DB5G64EIIU6YV5Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1HZoGwBuBh9Kg7gkvH9pfrjtxQurwxdnF7pWfycGuLHrHTSq61, address:185.167.204.205:4028): protocol: rpc: dial tcp 185.167.204.205:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311824674 (327h19m12.115289254s) [noise=true , address=185.167.204.206:4062 , piece_id=WWXKTOXCQ3X5QURTWE6DJ7VC25IOK4U7LKPZ72FRCQ7UUMQMD45A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 139.59.144.22:48898->185.167.204.206:4062: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311937619 (327h19m12.115402215s) [noise=true , address=185.167.204.206:4055 , piece_id=3QNJ4SSDSMAZVSU4KKKB2XO4I24RAVORE3G375K4Z5ARSLIODWCQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CYRieezm1mjtGchWdPrB2mNaHVRvTS85umjB6cQyYKnvisXSB, address:185.167.204.206:4055): protocol: rpc: dial tcp 185.167.204.206:4055: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312048594 (327h19m12.115513172s) [noise=true , address=185.167.204.205:4022 , piece_id=UMO7EQ4TYZTTBYTOTCSD7PIWPBKDE6UQNRRWJ62UJXMAQEK7VE2Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: rpc: dial tcp 185.167.204.205:4022: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312109260 (327h19m12.115573856s) [noise=true , address=89.149.201.147:4014 , piece_id=DFU2C7O7SSA3SUZ5RFKRN5XU6BEDCJVNNOAUPRDLFXOO5WVTWPDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 139.59.144.22:45014->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312154108 (327h19m12.115618686s) [noise=true , address=185.167.204.206:4051 , piece_id=G3ZZYDQ3RA37A56XPHCN5SZP5OX32HOYS5QRK6R3IPVETEGZJYKQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: write tcp 139.59.144.22:55302->185.167.204.206:4051: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312192236 (327h19m12.115656819s) [noise=true , address=185.167.204.205:4021 , piece_id=Y3IZRKFF75CAXPCZS56QGCO65NTLFJBIQB2ZT6NND5J33FWGEENA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12FDxcFVTdQeudwXi3bRkqnBSQ7NhjChgqkCZLmEC9T4bxdGA4w, address:185.167.204.205:4021): protocol: rpc: dial tcp 185.167.204.205:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312223288 (327h19m12.115687882s) [seg_pos={0 0}, upload=288035]: Done waiting for segment. successful: 10 optimal: 26 errs: <nil>
Oct 7 04:49:24.327964660 (327h19m12.131429241s) [upload=288035]: Done uploading segments...
Oct 7 04:49:24.327992813 (327h19m12.131457395s) [upload=288035]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 10)
Duration: 409.655373ms
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
420.984913ms
FAIL (2025-10-07 05:14:24 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 10)
======LOGS======
288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.525978676 (327h18m12.329443254s) [noise=true , address=89.149.201.147:4020 , piece_id=6X366FMMBQY7UMWRZWHWK24ODNSXNRK33EBT357XTN2QGNZ2XARQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.529349437 (327h18m12.332814029s) [noise=true , address=89.149.201.147:4007 , piece_id=GISIRIBAQZ4POYEFCQIFGEORZEB3DNANYK2FRE47HJD6KCHCWFIQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.531248958 (327h18m12.334713557s) [noise=true , address=81.171.14.70:4034 , piece_id=WXUMWYCPOYMDQRAC3GCH66ORLQEW4QMXCT6Z3IZGN75QPOHRJ44A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.544080578 (327h18m12.347545162s) [noise=true , address=89.149.201.147:4005 , piece_id=UN4KYWSCWFRCG6I44PP3HG3WQYJHHRYA73RE7XKYNWFMQZ4BPNHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.556041962 (327h18m12.359506556s) [noise=true , address=81.171.14.70:4030 , piece_id=W7VSTDTWQZKAKTQ4XXJEZEWWULVUS3VWLK7GXAIC5DC5JXWGDB6A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.612240410 (327h18m12.41570499s) [noise=true , address=81.171.14.70:4025 , piece_id=EFCX2CLEJGLGAS5HY66X7YJUNZEMYDXRDTRG4TZQ25IF4J6QZPUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:49:24.310483073 (327h19m12.113947666s) [noise=true , address=185.167.204.205:4011 , piece_id=2VDETVRCNNWJC7CRL6WRZJZMZFIHI76LWIJAFRIBLMJUECKZ3SKA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CiRxadZgQLgJ41bv86hSTEHLv2CzXC4jz9Q3r5dLEX6doYoZ3, address:185.167.204.205:4011): protocol: rpc: dial tcp 185.167.204.205:4011: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310527972 (327h19m12.113992556s) [noise=true , address=81.171.14.70:4028 , piece_id=HEFFOFVIYWTKT4RBKADCIREX354RWAO5USR2VTTIVVIERFV3AKDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310706256 (327h19m12.114170848s) [noise=true , address=89.149.201.147:4016 , piece_id=NR6WTUSOAMYODXHX2V62FGTKTBFQTK5JENXKR5J4HTBGIGKTTGMA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5, address:89.149.201.147:4016): protocol: rpc: dial tcp 89.149.201.147:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310894451 (327h19m12.114359047s) [noise=true , address=185.167.204.206:4058 , piece_id=WFFNR3LXSVF3HVHKHU4O6LBYVDDFGBLGCG4ZQZHOKJEECNNJIKXQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: rpc: dial tcp 185.167.204.206:4058: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310931505 (327h19m12.114396086s) [noise=true , address=185.167.204.206:4068 , piece_id=HBYTPAXUFS7ZM7OQRAKPDNITIVQOLSOH4RN362XBXOEZB4RFEGEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1sqDsiZKL7g4FbDyc9Ew2ojD1tTLzj4B19ma8hrCvWoTCy4j4D, address:185.167.204.206:4068): protocol: rpc: dial tcp 185.167.204.206:4068: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311073712 (327h19m12.114538311s) [noise=true , address=185.167.204.205:4018 , piece_id=KJA3E24NTU4J3FTWCJ2U4B5UICGUWDLS5VSTMB45SFGQNIGO3BSA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: rpc: dial tcp 185.167.204.205:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311100756 (327h19m12.114565337s) [noise=true , address=185.167.204.205:4013 , piece_id=6ZBB2XTBENBEQMD34O7LIISQHOCEOEZADNPXZOGD5XYJY4OLRHHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: write tcp 139.59.144.22:59630->185.167.204.205:4013: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311261060 (327h19m12.114725661s) [noise=true , address=81.171.14.70:4048 , piece_id=QUGF3RADQM3FHXACWUHOYCUAYYBI33B4XMXZ62JL5O7L2KIVUOUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: rpc: dial tcp 81.171.14.70:4048: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311315674 (327h19m12.114780271s) [noise=true , address=89.149.201.147:4012 , piece_id=MSSU4KAU2AR3YULH6W7JOXM6IG3NIAFU6UGX3FCFEKFLN72BSVYQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311573657 (327h19m12.115038254s) [noise=true , address=185.167.204.205:4025 , piece_id=XB2P432VMB2BJHSWNQCB4B33ZIDAR43QQD74ABRFF43MYRJZDAEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311641089 (327h19m12.11510567s) [noise=true , address=185.167.204.206:4060 , piece_id=FMVOGLOA67LN7G5JVY6NZZCNPR7PMZHBURUQGNKIRIUK7ZV4EGQA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311759265 (327h19m12.115223862s) [noise=true , address=185.167.204.205:4028 , piece_id=26AU5QE6RTN7FTB7XXOB2CGR5VJ7SIRURQUN2DB5G64EIIU6YV5Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1HZoGwBuBh9Kg7gkvH9pfrjtxQurwxdnF7pWfycGuLHrHTSq61, address:185.167.204.205:4028): protocol: rpc: dial tcp 185.167.204.205:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311824674 (327h19m12.115289254s) [noise=true , address=185.167.204.206:4062 , piece_id=WWXKTOXCQ3X5QURTWE6DJ7VC25IOK4U7LKPZ72FRCQ7UUMQMD45A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 139.59.144.22:48898->185.167.204.206:4062: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311937619 (327h19m12.115402215s) [noise=true , address=185.167.204.206:4055 , piece_id=3QNJ4SSDSMAZVSU4KKKB2XO4I24RAVORE3G375K4Z5ARSLIODWCQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CYRieezm1mjtGchWdPrB2mNaHVRvTS85umjB6cQyYKnvisXSB, address:185.167.204.206:4055): protocol: rpc: dial tcp 185.167.204.206:4055: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312048594 (327h19m12.115513172s) [noise=true , address=185.167.204.205:4022 , piece_id=UMO7EQ4TYZTTBYTOTCSD7PIWPBKDE6UQNRRWJ62UJXMAQEK7VE2Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: rpc: dial tcp 185.167.204.205:4022: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312109260 (327h19m12.115573856s) [noise=true , address=89.149.201.147:4014 , piece_id=DFU2C7O7SSA3SUZ5RFKRN5XU6BEDCJVNNOAUPRDLFXOO5WVTWPDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 139.59.144.22:45014->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312154108 (327h19m12.115618686s) [noise=true , address=185.167.204.206:4051 , piece_id=G3ZZYDQ3RA37A56XPHCN5SZP5OX32HOYS5QRK6R3IPVETEGZJYKQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: write tcp 139.59.144.22:55302->185.167.204.206:4051: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312192236 (327h19m12.115656819s) [noise=true , address=185.167.204.205:4021 , piece_id=Y3IZRKFF75CAXPCZS56QGCO65NTLFJBIQB2ZT6NND5J33FWGEENA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12FDxcFVTdQeudwXi3bRkqnBSQ7NhjChgqkCZLmEC9T4bxdGA4w, address:185.167.204.205:4021): protocol: rpc: dial tcp 185.167.204.205:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312223288 (327h19m12.115687882s) [seg_pos={0 0}, upload=288035]: Done waiting for segment. successful: 10 optimal: 26 errs: <nil>
Oct 7 04:49:24.327964660 (327h19m12.131429241s) [upload=288035]: Done uploading segments...
Oct 7 04:49:24.327992813 (327h19m12.131457395s) [upload=288035]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 10)
Duration: 421.020153ms
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
437.540187ms
FAIL (2025-10-07 05:14:01 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 10)
======LOGS======
288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.525978676 (327h18m12.329443254s) [noise=true , address=89.149.201.147:4020 , piece_id=6X366FMMBQY7UMWRZWHWK24ODNSXNRK33EBT357XTN2QGNZ2XARQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.529349437 (327h18m12.332814029s) [noise=true , address=89.149.201.147:4007 , piece_id=GISIRIBAQZ4POYEFCQIFGEORZEB3DNANYK2FRE47HJD6KCHCWFIQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.531248958 (327h18m12.334713557s) [noise=true , address=81.171.14.70:4034 , piece_id=WXUMWYCPOYMDQRAC3GCH66ORLQEW4QMXCT6Z3IZGN75QPOHRJ44A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.544080578 (327h18m12.347545162s) [noise=true , address=89.149.201.147:4005 , piece_id=UN4KYWSCWFRCG6I44PP3HG3WQYJHHRYA73RE7XKYNWFMQZ4BPNHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.556041962 (327h18m12.359506556s) [noise=true , address=81.171.14.70:4030 , piece_id=W7VSTDTWQZKAKTQ4XXJEZEWWULVUS3VWLK7GXAIC5DC5JXWGDB6A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:48:24.612240410 (327h18m12.41570499s) [noise=true , address=81.171.14.70:4025 , piece_id=EFCX2CLEJGLGAS5HY66X7YJUNZEMYDXRDTRG4TZQ25IF4J6QZPUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: <nil>
Oct 7 04:49:24.310483073 (327h19m12.113947666s) [noise=true , address=185.167.204.205:4011 , piece_id=2VDETVRCNNWJC7CRL6WRZJZMZFIHI76LWIJAFRIBLMJUECKZ3SKA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CiRxadZgQLgJ41bv86hSTEHLv2CzXC4jz9Q3r5dLEX6doYoZ3, address:185.167.204.205:4011): protocol: rpc: dial tcp 185.167.204.205:4011: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310527972 (327h19m12.113992556s) [noise=true , address=81.171.14.70:4028 , piece_id=HEFFOFVIYWTKT4RBKADCIREX354RWAO5USR2VTTIVVIERFV3AKDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310706256 (327h19m12.114170848s) [noise=true , address=89.149.201.147:4016 , piece_id=NR6WTUSOAMYODXHX2V62FGTKTBFQTK5JENXKR5J4HTBGIGKTTGMA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5, address:89.149.201.147:4016): protocol: rpc: dial tcp 89.149.201.147:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310894451 (327h19m12.114359047s) [noise=true , address=185.167.204.206:4058 , piece_id=WFFNR3LXSVF3HVHKHU4O6LBYVDDFGBLGCG4ZQZHOKJEECNNJIKXQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: rpc: dial tcp 185.167.204.206:4058: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.310931505 (327h19m12.114396086s) [noise=true , address=185.167.204.206:4068 , piece_id=HBYTPAXUFS7ZM7OQRAKPDNITIVQOLSOH4RN362XBXOEZB4RFEGEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1sqDsiZKL7g4FbDyc9Ew2ojD1tTLzj4B19ma8hrCvWoTCy4j4D, address:185.167.204.206:4068): protocol: rpc: dial tcp 185.167.204.206:4068: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311073712 (327h19m12.114538311s) [noise=true , address=185.167.204.205:4018 , piece_id=KJA3E24NTU4J3FTWCJ2U4B5UICGUWDLS5VSTMB45SFGQNIGO3BSA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: rpc: dial tcp 185.167.204.205:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311100756 (327h19m12.114565337s) [noise=true , address=185.167.204.205:4013 , piece_id=6ZBB2XTBENBEQMD34O7LIISQHOCEOEZADNPXZOGD5XYJY4OLRHHA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: write tcp 139.59.144.22:59630->185.167.204.205:4013: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311261060 (327h19m12.114725661s) [noise=true , address=81.171.14.70:4048 , piece_id=QUGF3RADQM3FHXACWUHOYCUAYYBI33B4XMXZ62JL5O7L2KIVUOUA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: rpc: dial tcp 81.171.14.70:4048: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311315674 (327h19m12.114780271s) [noise=true , address=89.149.201.147:4012 , piece_id=MSSU4KAU2AR3YULH6W7JOXM6IG3NIAFU6UGX3FCFEKFLN72BSVYQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311573657 (327h19m12.115038254s) [noise=true , address=185.167.204.205:4025 , piece_id=XB2P432VMB2BJHSWNQCB4B33ZIDAR43QQD74ABRFF43MYRJZDAEA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311641089 (327h19m12.11510567s) [noise=true , address=185.167.204.206:4060 , piece_id=FMVOGLOA67LN7G5JVY6NZZCNPR7PMZHBURUQGNKIRIUK7ZV4EGQA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311759265 (327h19m12.115223862s) [noise=true , address=185.167.204.205:4028 , piece_id=26AU5QE6RTN7FTB7XXOB2CGR5VJ7SIRURQUN2DB5G64EIIU6YV5Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1HZoGwBuBh9Kg7gkvH9pfrjtxQurwxdnF7pWfycGuLHrHTSq61, address:185.167.204.205:4028): protocol: rpc: dial tcp 185.167.204.205:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311824674 (327h19m12.115289254s) [noise=true , address=185.167.204.206:4062 , piece_id=WWXKTOXCQ3X5QURTWE6DJ7VC25IOK4U7LKPZ72FRCQ7UUMQMD45A, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 139.59.144.22:48898->185.167.204.206:4062: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.311937619 (327h19m12.115402215s) [noise=true , address=185.167.204.206:4055 , piece_id=3QNJ4SSDSMAZVSU4KKKB2XO4I24RAVORE3G375K4Z5ARSLIODWCQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12CYRieezm1mjtGchWdPrB2mNaHVRvTS85umjB6cQyYKnvisXSB, address:185.167.204.206:4055): protocol: rpc: dial tcp 185.167.204.206:4055: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312048594 (327h19m12.115513172s) [noise=true , address=185.167.204.205:4022 , piece_id=UMO7EQ4TYZTTBYTOTCSD7PIWPBKDE6UQNRRWJ62UJXMAQEK7VE2Q, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: rpc: dial tcp 185.167.204.205:4022: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312109260 (327h19m12.115573856s) [noise=true , address=89.149.201.147:4014 , piece_id=DFU2C7O7SSA3SUZ5RFKRN5XU6BEDCJVNNOAUPRDLFXOO5WVTWPDA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: write tcp 139.59.144.22:45014->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312154108 (327h19m12.115618686s) [noise=true , address=185.167.204.206:4051 , piece_id=G3ZZYDQ3RA37A56XPHCN5SZP5OX32HOYS5QRK6R3IPVETEGZJYKQ, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, address:185.167.204.206:4051): protocol: write tcp 139.59.144.22:55302->185.167.204.206:4051: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312192236 (327h19m12.115656819s) [noise=true , address=185.167.204.205:4021 , piece_id=Y3IZRKFF75CAXPCZS56QGCO65NTLFJBIQB2ZT6NND5J33FWGEENA, seg_pos={0 0}, upload=288035]: Done uploading piece. err: ecclient: upload failed (node:12FDxcFVTdQeudwXi3bRkqnBSQ7NhjChgqkCZLmEC9T4bxdGA4w, address:185.167.204.205:4021): protocol: rpc: dial tcp 185.167.204.205:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 7 04:49:24.312223288 (327h19m12.115687882s) [seg_pos={0 0}, upload=288035]: Done waiting for segment. successful: 10 optimal: 26 errs: <nil>
Oct 7 04:49:24.327964660 (327h19m12.131429241s) [upload=288035]: Done uploading segments...
Oct 7 04:49:24.327992813 (327h19m12.131457395s) [upload=288035]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 10)
Duration: 437.639893ms
core
uplink
eu-select-1
eu1-placement-15
|