Orbiter History

Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure

Orbiter Check History

Showing 50 of 197932 reports (Page 169 of 3959)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 2.459151073s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:33:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.459181381s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 2.370130063s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:32:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.370160205s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 2.571321415s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:32:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.571345076s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 2.441126296s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:31:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.441201215s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 2.730076346s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:31:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.730111264s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 2.710862007s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:31:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.710884278s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 2.767684951s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:30:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.767707777s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 2.271348169s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:30:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.271373201s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 2.366904827s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:30:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.366945079s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 2.746435208s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:29:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.746464517s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 2.480606341s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:29:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.480640148s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 2.571517552s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:29:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.571587227s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 2.413769971s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:28:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.413799862s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 2.284216214s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:27:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.284245743s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 2.391659864s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:27:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.391684628s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 2.386678955s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:27:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.386718809s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 2.302014041s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:26:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.30203014s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 2.449350248s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:25:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.449376224s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 2.369121972s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:25:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.369152386s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 2.65931123s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:25:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.659336548s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 2.37469942s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:24:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.374731253s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 2.982885202s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:24:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.982912854s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 2.376664746s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:23:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.376692067s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 2.46288611s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:23:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.462911137s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 2.342630398s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:22:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.342658077s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 2.408855737s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:22:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.408888856s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 2.388933556s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:22:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.388960745s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 2.746042075s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:22:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.74609235s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 2.360543153s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:21:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.360600301s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 2.769287742s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:20:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.76930947s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 2.341535015s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:20:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.34155652s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 2.318529419s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:20:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.318616726s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 2.82330899s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:19:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.823403296s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 2.352998677s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:19:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.353025402s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 2.452652882s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:19:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.452688594s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 2.733815676s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:18:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.733895783s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 2.271501353s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:18:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.271530592s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 2.690062658s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:17:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.690091024s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 2.315712727s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:16:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.315755955s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 2.544505647s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:16:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.544555735s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 2.293287781s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:16:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.293316315s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 460.691778ms trace: https://jaeger.dev.storj.io/trace/7b371892eef4dd63 FAIL (2026-01-07 04:15:45 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== , upload=686441]: Uploading piece... Jan 7 03:26:43.387872805 (779h58m23.368225447s) [noise=true , address=81.171.14.70:4026 , piece_id=D6V7S54H3EHP7OGPZ5QXLL7BTFINC2WXI62SO7JOOQA3DY4AMDLA, seg_pos={0 0}, upload=686441]: Uploading piece... Jan 7 03:26:43.389178235 (779h58m23.369530882s) [noise=true , address=89.149.201.147:4002 , piece_id=SSYLTLUQ4FQF6PGLTSTKPJ5IZRQJ4E54VHGD7GXJK62HEBRTO76Q, seg_pos={0 0}, upload=686441]: Uploading piece... Jan 7 03:26:43.389739604 (779h58m23.370092243s) [noise=true , address=81.171.14.70:4035 , piece_id=WTUPVS7JQZMTPBY6BZDFOFHPX26Y7MZO6BCGYOIQDZJHWWMSYLXQ, seg_pos={0 0}, upload=686441]: Uploading piece... Jan 7 03:26:43.390234914 (779h58m23.370587552s) [noise=true , address=185.167.204.205:4010 , piece_id=JIUNU2MQONHWOCZHMHYP6U7II55LAN3QZOGHGFV6C5UGNNN7WN3Q, seg_pos={0 0}, upload=686441]: Uploading piece... Jan 7 03:26:43.390479507 (779h58m23.370832149s) [noise=true , address=89.149.201.147:4004 , piece_id=ZFRSJDEEINM7XYDXLVEA44IMJPY5L3CCZ7DNDY5YYXVOP3QIXECQ, seg_pos={0 0}, upload=686441]: Uploading piece... Jan 7 03:26:43.390725806 (779h58m23.371078448s) [noise=true , address=89.149.201.147:4009 , piece_id=OIA5BFYG3T3Q562VX7DSN325RRQVO2LCTKOWMSMU7CNYSLSYKIHQ, seg_pos={0 0}, upload=686441]: Uploading piece... Jan 7 03:26:43.391155258 (779h58m23.371507903s) [noise=true , address=185.167.204.206:4038 , piece_id=OIMOMB7DTTLRNEUN2CNIH3EDBRSKTBZBBW6Y2YVJ2MSUIXQUOBWA, seg_pos={0 0}, upload=686441]: Uploading piece... Jan 7 03:26:43.391639750 (779h58m23.371992392s) [noise=true , address=185.167.204.205:4006 , piece_id=MHY77DOTEOQ3GJAKT4XAPI4T26UPVA4V7GOHP4VXE7XELXWZCBJQ, seg_pos={0 0}, upload=686441]: Uploading piece... Jan 7 03:26:43.392116840 (779h58m23.372469479s) [noise=true , address=81.171.14.70:4025 , piece_id=FBE46UULDT5THRXYPUNWLIEAGC4WMXOZP65J3ERVKVJPLTVAMKCQ, seg_pos={0 0}, upload=686441]: Uploading piece... Jan 7 03:26:43.392750335 (779h58m23.373102981s) [noise=true , address=89.149.201.147:4018 , piece_id=NQSTC75YLUXJMXORNC5Y3HD344Q65BAEEYLKLMG4H5AFLQSVB7YQ, seg_pos={0 0}, upload=686441]: Uploading piece... Jan 7 03:26:43.500488982 (779h58m23.480841619s) [upload=686441]: Next returned nil segment Jan 7 03:26:43.500502704 (779h58m23.480855344s) [upload=686441]: Waiting for error group managing segments... Jan 7 03:26:43.519244889 (779h58m23.499597525s) [noise=true , address=89.149.201.147:4009 , piece_id=OIA5BFYG3T3Q562VX7DSN325RRQVO2LCTKOWMSMU7CNYSLSYKIHQ, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.521603031 (779h58m23.501955672s) [noise=true , address=89.149.201.147:4015 , piece_id=7XWP6ZCNJ4Y76PFLX3X4HCJ4FLQUKNPIFBUIFL5AZZHF5TNGXJSA, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.522270327 (779h58m23.502622973s) [noise=true , address=81.171.14.70:4026 , piece_id=D6V7S54H3EHP7OGPZ5QXLL7BTFINC2WXI62SO7JOOQA3DY4AMDLA, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.522813706 (779h58m23.503166363s) [noise=true , address=81.171.14.70:4035 , piece_id=WTUPVS7JQZMTPBY6BZDFOFHPX26Y7MZO6BCGYOIQDZJHWWMSYLXQ, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.523516032 (779h58m23.503868675s) [noise=true , address=89.149.201.147:4004 , piece_id=ZFRSJDEEINM7XYDXLVEA44IMJPY5L3CCZ7DNDY5YYXVOP3QIXECQ, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.523730457 (779h58m23.5040831s) [noise=true , address=81.171.14.70:4043 , piece_id=OOIX4QERWXDD3NLKOKT6JKNTQJ2QDT7Z7C25TME277PUSVINLUWA, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.529666198 (779h58m23.510018838s) [noise=true , address=81.171.14.70:4039 , piece_id=WJYTHK6LCJXQ5OHFVUAHJD5A4YG3ZRABHKLICBAFBQ4ELNDN24KQ, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.531785009 (779h58m23.512137648s) [noise=true , address=185.167.204.205:4010 , piece_id=JIUNU2MQONHWOCZHMHYP6U7II55LAN3QZOGHGFV6C5UGNNN7WN3Q, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.531964414 (779h58m23.512317053s) [noise=true , address=185.167.204.205:4023 , piece_id=ZHIVHZ57WMZVXMB6WR3DE7AR5DTTPOCUTFUYRR2UX6Y34MDHAZ4Q, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.533351952 (779h58m23.513704592s) [noise=true , address=185.167.204.205:4019 , piece_id=KFCI6QHNYKKMSPTCW3UHLW4BAGEMJ4UW3533O3E5E5IEWFWLJIGQ, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.534306412 (779h58m23.514659034s) [noise=true , address=185.167.204.206:4066 , piece_id=BSNLFLDRXJOULSYRIID5TLRWMYHTWTCW3WOLRUUV5MTULNCQOI4A, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.535288465 (779h58m23.515641105s) [noise=true , address=81.171.14.70:4046 , piece_id=WOOFVSQOIVEYUBACZ5VB25OTVAQQOIOVNTIM3454DMIKM265PC5Q, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.537352031 (779h58m23.517704654s) [noise=true , address=185.167.204.206:4067 , piece_id=ZCWEF6EMT2FOUKDXANWL3JL4MUF3GHU2W7TRP4DN2A5ODXIQ5GTA, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.537498734 (779h58m23.517851373s) [noise=true , address=89.149.201.147:4001 , piece_id=MG6CMDYJ2YMJXJESV5VEMIH7LBQDM76GDCPAWDBVRUA2FQJIGKGQ, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.537897241 (779h58m23.518249877s) [noise=true , address=81.171.14.70:4044 , piece_id=WGKLEZ5XR24VULXUHGXCBWFQYCJZYAOMY5SSMGQVZCLUNBFB2F6A, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.538034385 (779h58m23.518387007s) [noise=true , address=185.167.204.205:4033 , piece_id=NEZLBBZB5LX4AXIE7H4BCK3KNQVAE33F2TUXDSRAJH3YVSWAQC6A, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.538323050 (779h58m23.51867569s) [noise=true , address=185.167.204.206:4038 , piece_id=OIMOMB7DTTLRNEUN2CNIH3EDBRSKTBZBBW6Y2YVJ2MSUIXQUOBWA, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.541534619 (779h58m23.521887242s) [noise=true , address=185.167.204.206:4054 , piece_id=OOVZMXJSWKCWSUX344RRDG7UEZEAOMQXUG6OV5VXESSJWGWRB2UA, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.541641818 (779h58m23.521994457s) [noise=true , address=89.149.201.147:4002 , piece_id=SSYLTLUQ4FQF6PGLTSTKPJ5IZRQJ4E54VHGD7GXJK62HEBRTO76Q, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.542927272 (779h58m23.523279912s) [noise=true , address=81.171.14.70:4034 , piece_id=CDJTQ4VXBYZZ2LOC67NSP4UE32UNKHXJV2MDMZUWUD7555NHIQZA, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.548025411 (779h58m23.528378057s) [noise=true , address=185.167.204.205:4006 , piece_id=MHY77DOTEOQ3GJAKT4XAPI4T26UPVA4V7GOHP4VXE7XELXWZCBJQ, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.549020976 (779h58m23.529373612s) [noise=true , address=185.167.204.205:4013 , piece_id=PMPDCCOHXJDWJKHW5FEHPFRGMGCPMVGNDV65GJY73E7KVXTLEJKA, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.549553037 (779h58m23.529905673s) [noise=true , address=185.167.204.206:4060 , piece_id=EGSWBQO6LPILNP5DCG5B2NRTAFODYQLXQNPIWA53PMMMYFBL7FPA, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.553488022 (779h58m23.533840671s) [noise=true , address=89.149.201.147:4018 , piece_id=NQSTC75YLUXJMXORNC5Y3HD344Q65BAEEYLKLMG4H5AFLQSVB7YQ, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.560492616 (779h58m23.540845257s) [noise=true , address=185.167.204.205:4021 , piece_id=JIGFGKA2MDSOU4B5K6NSQBVC25AP5BV7O3HWVFEGT3CN6AKZODQQ, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.577759518 (779h58m23.558112158s) [noise=true , address=185.167.204.206:4070 , piece_id=N4WYS4MR4AJANFYA4M23HZQBFTZD4AGDDYLXU6XPK7GHSAOX4OKA, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.577785998 (779h58m23.55813862s) [seg_pos={0 0}, upload=686441]: Segment reached optimal threshold of 26 pieces. Jan 7 03:26:43.578144391 (779h58m23.558497039s) [noise=true , address=81.171.14.70:4037 , piece_id=2FG5V4FO6DQ2SW2654GQ7QNHNFVACXHBZ4H6YCAAWGDGNPLUJQKQ, seg_pos={0 0}, upload=686441]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx): protocol: expected piece hash; context canceled; EOF Jan 7 03:26:43.578155754 (779h58m23.558508401s) [noise=true , address=81.171.14.70:4037 , piece_id=2FG5V4FO6DQ2SW2654GQ7QNHNFVACXHBZ4H6YCAAWGDGNPLUJQKQ, seg_pos={0 0}, upload=686441]: Long tail cancellation due to optimal threshold Jan 7 03:26:43.578572099 (779h58m23.558924744s) [noise=true , address=81.171.14.70:4025 , piece_id=FBE46UULDT5THRXYPUNWLIEAGC4WMXOZP65J3ERVKVJPLTVAMKCQ, seg_pos={0 0}, upload=686441]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU): protocol: expected piece hash; context canceled; EOF Jan 7 03:26:43.578582539 (779h58m23.558935183s) [noise=true , address=81.171.14.70:4025 , piece_id=FBE46UULDT5THRXYPUNWLIEAGC4WMXOZP65J3ERVKVJPLTVAMKCQ, seg_pos={0 0}, upload=686441]: Long tail cancellation due to optimal threshold Jan 7 03:26:43.578616418 (779h58m23.558969042s) [seg_pos={0 0}, upload=686441]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Jan 7 03:26:43.578719775 (779h58m23.559072419s) [upload=686441]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 7 03:26:43.578727146 (779h58m23.55907979s) [upload=686441]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 7 03:26:43.611283195 (779h58m23.591635841s) [upload=686441]: Done uploading segments... Jan 7 03:26:43.611442487 (779h58m23.591795129s) [upload=686441]: Upload finished. err: <nil> Duration: 460.719144ms core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 2.693828964s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:15:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.693864303s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 2.313395739s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:15:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.313432532s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 2.542842006s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:15:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.542870616s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 2.415289379s trace: https://jaeger.dev.storj.io/trace/55aff79aa6016ac2 FAIL (2026-01-07 04:15:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 7 04:12:12.279109725 (780h43m52.25946237s) [upload=687094]: Starting upload Jan 7 04:12:12.279186406 (780h43m52.259539028s) [upload=687094]: Done starting upload Jan 7 04:12:12.279290520 (780h43m52.259643158s) [upload=687094]: Uploading segments... Jan 7 04:12:12.279517431 (780h43m52.25987013s) [upload=687094]: Got next segment. Inline: false Jan 7 04:12:12.279563468 (780h43m52.259916108s) [upload=687094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 7 04:12:12.279569970 (780h43m52.259922606s) [upload=687094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 7 04:12:12.394010718 (780h43m52.374363366s) [seg_pos={0 0}, upload=687094]: Begin upload segment... Jan 7 04:12:12.394156094 (780h43m52.374508746s) [seg_pos={0 0}, upload=687094]: Done begin upload segment. Jan 7 04:12:12.394448797 (780h43m52.374801443s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395295598 (780h43m52.375648227s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.395828331 (780h43m52.376180961s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.397171427 (780h43m52.377524065s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.398004170 (780h43m52.378356797s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.399010338 (780h43m52.379362963s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.400124232 (780h43m52.38047687s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.401181503 (780h43m52.381534126s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.404226165 (780h43m52.384578803s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.406884374 (780h43m52.387237014s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409136352 (780h43m52.389488975s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.409580232 (780h43m52.389932857s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411118382 (780h43m52.391471013s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.411546829 (780h43m52.391899459s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.412979295 (780h43m52.393331924s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:12.414024113 (780h43m52.394376802s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Uploading piece... Jan 7 04:12:13.005774346 (780h43m52.986126982s) [upload=687094]: Next returned nil segment Jan 7 04:12:13.005788645 (780h43m52.986141282s) [upload=687094]: Waiting for error group managing segments... Jan 7 04:12:13.141645151 (780h43m53.121997788s) [noise=true , address=77.90.31.12:4590 , piece_id=7U5CT27FHLZEBHTBVN6YW63UAOWK5VJUQRUFDRBYUFJXNZDFTF7Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142254412 (780h43m53.122607048s) [noise=true , address=145.79.227.37:4565 , piece_id=N5RTMM5R7HX2WFR6NKWILXCMFTXJLKJW6OFY5NRFAOTI4AFUWWHA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142513975 (780h43m53.122866611s) [noise=true , address=145.79.227.41:4087 , piece_id=XEO2LXNR5PXM3S3PDJKTMSOMRKEGCY727N4YRIQNRMQBASR7DNPQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142667267 (780h43m53.123019904s) [noise=true , address=77.90.31.15:4697 , piece_id=6PNBQXI4TFOEESVT5URVRLECUVUIO2EDKR2AX2NBRPRRM67OYDJA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.142887445 (780h43m53.123240085s) [noise=true , address=145.79.227.42:4123 , piece_id=FKNW353UOK7HRJZ7V75LVM762HHXFVQJJJZRF2YEN5T7YBX3ZCCQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144261505 (780h43m53.124614142s) [noise=true , address=145.79.227.48:4302 , piece_id=T6H6AFDITQN3ON7NF2RPO6SYGQH7XTT7VNBZCVVW5SWT7AF5QNMQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.144696993 (780h43m53.125049638s) [noise=true , address=145.79.227.5:4221 , piece_id=FD4RN5TKDXG2B4EGCL6FXPQAAUF7W6EV6NLEYIGGFWSLCURFHCZA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.146284346 (780h43m53.126636983s) [noise=true , address=145.79.227.26:4655 , piece_id=LIBOAKKGTMS4EP4GTPQTW3GTT6YRTYQIKDXHNQDTM53YNJMJIQQQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.149018531 (780h43m53.129371167s) [noise=true , address=145.79.227.7:4293 , piece_id=6ZO2IR2EVKVYEK5TPRTPCX6JAQ4PC2VT2NDLBLTGSHIJKSK5G5RA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.153650309 (780h43m53.134002946s) [noise=true , address=145.79.227.47:4266 , piece_id=DBIZFL3SFVKPB4J3V6JXH7VBDNURGC5BZAJSQYEXNHJKYGOMNV4Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:13.166062242 (780h43m53.146414878s) [noise=true , address=145.79.227.40:4050 , piece_id=MJ6OTBT2PTKCXO5RNOVAZY43YTTJZXQZP6TDFHC6T35YNKTC3PKA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: <nil> Jan 7 04:12:41.968123259 (780h44m21.948475904s) [noise=true , address=45.91.116.13:4154 , piece_id=RHSAPXSMIZIGRU4AWZLGWJRZ2S55JLZ4WP37UDY2IGNEN6CBU73Q, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968200455 (780h44m21.948553117s) [noise=true , address=45.91.116.6:4975 , piece_id=ZDIXVBCHBDO35ZY7GYTFE3M4DLUCAATJRPPWFLHRD44WWSHENGEA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968381362 (780h44m21.94873401s) [noise=true , address=45.91.116.10:4084 , piece_id=XLZT46H4BT3MQWHLNHIF6BLKSBEPOUFU5PKR6OXNEMLC4QF6MFGQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, address:45.91.116.10:4084): protocol: read tcp 139.59.144.22:59932->45.91.116.10:4084: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968524111 (780h44m21.948876763s) [noise=true , address=45.91.116.14:4191 , piece_id=LC7EBFCO2RVSODGHDDDOMC2FZZHZ56VOZPRSPEMXI24Z4Q3HXHDQ, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.968976651 (780h44m21.949329301s) [noise=true , address=45.91.116.9:4262 , piece_id=VTBWMU7HK3AQEUJNBH5IXIGWBSPCU4D7XV3QQOAGVF3HPL2ATONA, seg_pos={0 0}, upload=687094]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 7 04:12:41.969026012 (780h44m21.949378652s) [seg_pos={0 0}, upload=687094]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 7 04:12:42.089798853 (780h44m22.07015148s) [upload=687094]: Done uploading segments... Jan 7 04:12:42.089848843 (780h44m22.070201488s) [upload=687094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.415321739s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 1.079221783s trace: https://jaeger.dev.storj.io/trace/1abf79eb2f5524b8 FAIL (2026-01-07 04:14:45 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 1.079253413s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 1.123437292s trace: https://jaeger.dev.storj.io/trace/7b371892eef4dd63 FAIL (2026-01-07 04:14:40 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== , upload=686441]: Uploading piece... Jan 7 03:26:43.387872805 (779h58m23.368225447s) [noise=true , address=81.171.14.70:4026 , piece_id=D6V7S54H3EHP7OGPZ5QXLL7BTFINC2WXI62SO7JOOQA3DY4AMDLA, seg_pos={0 0}, upload=686441]: Uploading piece... Jan 7 03:26:43.389178235 (779h58m23.369530882s) [noise=true , address=89.149.201.147:4002 , piece_id=SSYLTLUQ4FQF6PGLTSTKPJ5IZRQJ4E54VHGD7GXJK62HEBRTO76Q, seg_pos={0 0}, upload=686441]: Uploading piece... Jan 7 03:26:43.389739604 (779h58m23.370092243s) [noise=true , address=81.171.14.70:4035 , piece_id=WTUPVS7JQZMTPBY6BZDFOFHPX26Y7MZO6BCGYOIQDZJHWWMSYLXQ, seg_pos={0 0}, upload=686441]: Uploading piece... Jan 7 03:26:43.390234914 (779h58m23.370587552s) [noise=true , address=185.167.204.205:4010 , piece_id=JIUNU2MQONHWOCZHMHYP6U7II55LAN3QZOGHGFV6C5UGNNN7WN3Q, seg_pos={0 0}, upload=686441]: Uploading piece... Jan 7 03:26:43.390479507 (779h58m23.370832149s) [noise=true , address=89.149.201.147:4004 , piece_id=ZFRSJDEEINM7XYDXLVEA44IMJPY5L3CCZ7DNDY5YYXVOP3QIXECQ, seg_pos={0 0}, upload=686441]: Uploading piece... Jan 7 03:26:43.390725806 (779h58m23.371078448s) [noise=true , address=89.149.201.147:4009 , piece_id=OIA5BFYG3T3Q562VX7DSN325RRQVO2LCTKOWMSMU7CNYSLSYKIHQ, seg_pos={0 0}, upload=686441]: Uploading piece... Jan 7 03:26:43.391155258 (779h58m23.371507903s) [noise=true , address=185.167.204.206:4038 , piece_id=OIMOMB7DTTLRNEUN2CNIH3EDBRSKTBZBBW6Y2YVJ2MSUIXQUOBWA, seg_pos={0 0}, upload=686441]: Uploading piece... Jan 7 03:26:43.391639750 (779h58m23.371992392s) [noise=true , address=185.167.204.205:4006 , piece_id=MHY77DOTEOQ3GJAKT4XAPI4T26UPVA4V7GOHP4VXE7XELXWZCBJQ, seg_pos={0 0}, upload=686441]: Uploading piece... Jan 7 03:26:43.392116840 (779h58m23.372469479s) [noise=true , address=81.171.14.70:4025 , piece_id=FBE46UULDT5THRXYPUNWLIEAGC4WMXOZP65J3ERVKVJPLTVAMKCQ, seg_pos={0 0}, upload=686441]: Uploading piece... Jan 7 03:26:43.392750335 (779h58m23.373102981s) [noise=true , address=89.149.201.147:4018 , piece_id=NQSTC75YLUXJMXORNC5Y3HD344Q65BAEEYLKLMG4H5AFLQSVB7YQ, seg_pos={0 0}, upload=686441]: Uploading piece... Jan 7 03:26:43.500488982 (779h58m23.480841619s) [upload=686441]: Next returned nil segment Jan 7 03:26:43.500502704 (779h58m23.480855344s) [upload=686441]: Waiting for error group managing segments... Jan 7 03:26:43.519244889 (779h58m23.499597525s) [noise=true , address=89.149.201.147:4009 , piece_id=OIA5BFYG3T3Q562VX7DSN325RRQVO2LCTKOWMSMU7CNYSLSYKIHQ, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.521603031 (779h58m23.501955672s) [noise=true , address=89.149.201.147:4015 , piece_id=7XWP6ZCNJ4Y76PFLX3X4HCJ4FLQUKNPIFBUIFL5AZZHF5TNGXJSA, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.522270327 (779h58m23.502622973s) [noise=true , address=81.171.14.70:4026 , piece_id=D6V7S54H3EHP7OGPZ5QXLL7BTFINC2WXI62SO7JOOQA3DY4AMDLA, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.522813706 (779h58m23.503166363s) [noise=true , address=81.171.14.70:4035 , piece_id=WTUPVS7JQZMTPBY6BZDFOFHPX26Y7MZO6BCGYOIQDZJHWWMSYLXQ, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.523516032 (779h58m23.503868675s) [noise=true , address=89.149.201.147:4004 , piece_id=ZFRSJDEEINM7XYDXLVEA44IMJPY5L3CCZ7DNDY5YYXVOP3QIXECQ, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.523730457 (779h58m23.5040831s) [noise=true , address=81.171.14.70:4043 , piece_id=OOIX4QERWXDD3NLKOKT6JKNTQJ2QDT7Z7C25TME277PUSVINLUWA, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.529666198 (779h58m23.510018838s) [noise=true , address=81.171.14.70:4039 , piece_id=WJYTHK6LCJXQ5OHFVUAHJD5A4YG3ZRABHKLICBAFBQ4ELNDN24KQ, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.531785009 (779h58m23.512137648s) [noise=true , address=185.167.204.205:4010 , piece_id=JIUNU2MQONHWOCZHMHYP6U7II55LAN3QZOGHGFV6C5UGNNN7WN3Q, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.531964414 (779h58m23.512317053s) [noise=true , address=185.167.204.205:4023 , piece_id=ZHIVHZ57WMZVXMB6WR3DE7AR5DTTPOCUTFUYRR2UX6Y34MDHAZ4Q, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.533351952 (779h58m23.513704592s) [noise=true , address=185.167.204.205:4019 , piece_id=KFCI6QHNYKKMSPTCW3UHLW4BAGEMJ4UW3533O3E5E5IEWFWLJIGQ, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.534306412 (779h58m23.514659034s) [noise=true , address=185.167.204.206:4066 , piece_id=BSNLFLDRXJOULSYRIID5TLRWMYHTWTCW3WOLRUUV5MTULNCQOI4A, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.535288465 (779h58m23.515641105s) [noise=true , address=81.171.14.70:4046 , piece_id=WOOFVSQOIVEYUBACZ5VB25OTVAQQOIOVNTIM3454DMIKM265PC5Q, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.537352031 (779h58m23.517704654s) [noise=true , address=185.167.204.206:4067 , piece_id=ZCWEF6EMT2FOUKDXANWL3JL4MUF3GHU2W7TRP4DN2A5ODXIQ5GTA, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.537498734 (779h58m23.517851373s) [noise=true , address=89.149.201.147:4001 , piece_id=MG6CMDYJ2YMJXJESV5VEMIH7LBQDM76GDCPAWDBVRUA2FQJIGKGQ, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.537897241 (779h58m23.518249877s) [noise=true , address=81.171.14.70:4044 , piece_id=WGKLEZ5XR24VULXUHGXCBWFQYCJZYAOMY5SSMGQVZCLUNBFB2F6A, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.538034385 (779h58m23.518387007s) [noise=true , address=185.167.204.205:4033 , piece_id=NEZLBBZB5LX4AXIE7H4BCK3KNQVAE33F2TUXDSRAJH3YVSWAQC6A, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.538323050 (779h58m23.51867569s) [noise=true , address=185.167.204.206:4038 , piece_id=OIMOMB7DTTLRNEUN2CNIH3EDBRSKTBZBBW6Y2YVJ2MSUIXQUOBWA, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.541534619 (779h58m23.521887242s) [noise=true , address=185.167.204.206:4054 , piece_id=OOVZMXJSWKCWSUX344RRDG7UEZEAOMQXUG6OV5VXESSJWGWRB2UA, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.541641818 (779h58m23.521994457s) [noise=true , address=89.149.201.147:4002 , piece_id=SSYLTLUQ4FQF6PGLTSTKPJ5IZRQJ4E54VHGD7GXJK62HEBRTO76Q, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.542927272 (779h58m23.523279912s) [noise=true , address=81.171.14.70:4034 , piece_id=CDJTQ4VXBYZZ2LOC67NSP4UE32UNKHXJV2MDMZUWUD7555NHIQZA, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.548025411 (779h58m23.528378057s) [noise=true , address=185.167.204.205:4006 , piece_id=MHY77DOTEOQ3GJAKT4XAPI4T26UPVA4V7GOHP4VXE7XELXWZCBJQ, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.549020976 (779h58m23.529373612s) [noise=true , address=185.167.204.205:4013 , piece_id=PMPDCCOHXJDWJKHW5FEHPFRGMGCPMVGNDV65GJY73E7KVXTLEJKA, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.549553037 (779h58m23.529905673s) [noise=true , address=185.167.204.206:4060 , piece_id=EGSWBQO6LPILNP5DCG5B2NRTAFODYQLXQNPIWA53PMMMYFBL7FPA, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.553488022 (779h58m23.533840671s) [noise=true , address=89.149.201.147:4018 , piece_id=NQSTC75YLUXJMXORNC5Y3HD344Q65BAEEYLKLMG4H5AFLQSVB7YQ, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.560492616 (779h58m23.540845257s) [noise=true , address=185.167.204.205:4021 , piece_id=JIGFGKA2MDSOU4B5K6NSQBVC25AP5BV7O3HWVFEGT3CN6AKZODQQ, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.577759518 (779h58m23.558112158s) [noise=true , address=185.167.204.206:4070 , piece_id=N4WYS4MR4AJANFYA4M23HZQBFTZD4AGDDYLXU6XPK7GHSAOX4OKA, seg_pos={0 0}, upload=686441]: Done uploading piece. err: <nil> Jan 7 03:26:43.577785998 (779h58m23.55813862s) [seg_pos={0 0}, upload=686441]: Segment reached optimal threshold of 26 pieces. Jan 7 03:26:43.578144391 (779h58m23.558497039s) [noise=true , address=81.171.14.70:4037 , piece_id=2FG5V4FO6DQ2SW2654GQ7QNHNFVACXHBZ4H6YCAAWGDGNPLUJQKQ, seg_pos={0 0}, upload=686441]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx): protocol: expected piece hash; context canceled; EOF Jan 7 03:26:43.578155754 (779h58m23.558508401s) [noise=true , address=81.171.14.70:4037 , piece_id=2FG5V4FO6DQ2SW2654GQ7QNHNFVACXHBZ4H6YCAAWGDGNPLUJQKQ, seg_pos={0 0}, upload=686441]: Long tail cancellation due to optimal threshold Jan 7 03:26:43.578572099 (779h58m23.558924744s) [noise=true , address=81.171.14.70:4025 , piece_id=FBE46UULDT5THRXYPUNWLIEAGC4WMXOZP65J3ERVKVJPLTVAMKCQ, seg_pos={0 0}, upload=686441]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU): protocol: expected piece hash; context canceled; EOF Jan 7 03:26:43.578582539 (779h58m23.558935183s) [noise=true , address=81.171.14.70:4025 , piece_id=FBE46UULDT5THRXYPUNWLIEAGC4WMXOZP65J3ERVKVJPLTVAMKCQ, seg_pos={0 0}, upload=686441]: Long tail cancellation due to optimal threshold Jan 7 03:26:43.578616418 (779h58m23.558969042s) [seg_pos={0 0}, upload=686441]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Jan 7 03:26:43.578719775 (779h58m23.559072419s) [upload=686441]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 7 03:26:43.578727146 (779h58m23.55907979s) [upload=686441]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 7 03:26:43.611283195 (779h58m23.591635841s) [upload=686441]: Done uploading segments... Jan 7 03:26:43.611442487 (779h58m23.591795129s) [upload=686441]: Upload finished. err: <nil> Duration: 1.123463534s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 165.96199ms trace: https://jaeger.dev.storj.io/trace/4b8a4595fd3a13 FAIL (2026-01-07 04:14:40 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 165.990068ms core uplink eu1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 2.710996551s trace: https://jaeger.dev.storj.io/trace/6a8f48d2e29be936 FAIL (2026-01-07 04:14:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.711018352s edge gateway us-select-1

1
...
167
168
169
170
171
...
3959