Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 2.309011827s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:52:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.309053984s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 2.675433957s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:51:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.675463611s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 2.272229341s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:51:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.272256513s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 2.399268351s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:50:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.39929239s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 2.477699057s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:50:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.477722667s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 2.240687033s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:49:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.240712791s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 2.821697569s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:49:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.821733254s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 2.268220294s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:49:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.268246913s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 2.38837331s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:48:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.38840126s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 2.391473888s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:48:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.39149695s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 2.533718412s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:48:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.533837622s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 2.571796497s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:47:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.571823337s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 2.3180238s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:47:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.318049693s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 2.736018992s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:47:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.736045123s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 2.667898298s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:46:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.667932936s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 2.441646037s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:46:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.441712732s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 2.829114962s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:45:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.829141481s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 2.25550156s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:45:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.25553114s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 2.348020756s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:44:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.348051714s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 2.44104348s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:44:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.441070618s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 2.287670459s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:44:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.287696042s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 2.540952377s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:44:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.541016836s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 2.441590995s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:43:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.441619377s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 2.552101005s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:43:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.552161339s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 2.319537214s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:43:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.319572567s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 2.40887666s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:42:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.40892361s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 2.548545694s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:42:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.54858597s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 2.254337513s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:41:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.25437214s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 2.806298277s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:41:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.806352212s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 2.381240209s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:40:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.381268457s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 2.53615468s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:40:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.536184274s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 2.533103741s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:40:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.533135516s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 2.723508694s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:39:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.723537078s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 2.697530221s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:39:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.697559234s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 2.452430503s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:39:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.452455345s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 2.419710585s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:38:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.419736095s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 2.750354269s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:38:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.750416614s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 2.426416109s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:37:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.426448551s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 2.606278366s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:37:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.606302721s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 2.414289507s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:37:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.414330386s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 2.521098107s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:36:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.521131354s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 2.5448415s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:36:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.544867785s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 2.504859058s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:36:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.504883452s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 2.478705317s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:35:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.478727224s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 2.551756622s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:35:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.551779561s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 2.355721686s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:35:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.355750226s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 2.724212984s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:34:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.724246917s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 2.623265597s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:34:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.623290921s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 2.619092742s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:33:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.619118457s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 2.665374418s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:33:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.665394125s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)