Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [X..XXX....X.X.X...............X.............................] duration: 2.284259161s FAIL (2025-07-26 22:40:59 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 988 (58h42m45.061082575s) [upload=56308]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:18:14.018433272 (58h42m45.208731856s) [upload=56308]: Done uploading segments... Jul 26 22:18:14.018484438 (58h42m45.208783019s) [upload=56308]: Upload finished. err: <nil> Jul 26 22:18:57.520834389 (58h43m28.711132968s) [upload=56321]: Starting upload Jul 26 22:18:57.520934224 (58h43m28.711232794s) [upload=56321]: Done starting upload Jul 26 22:18:57.521309042 (58h43m28.711607621s) [upload=56321]: Uploading segments... Jul 26 22:18:57.521389036 (58h43m28.711687618s) [upload=56321]: Got next segment. Inline: false Jul 26 22:18:57.521430569 (58h43m28.711729148s) [upload=56321]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:18:57.521435990 (58h43m28.711734568s) [upload=56321]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:18:57.634354903 (58h43m28.824653493s) [seg_pos={0 0}, upload=56321]: Begin upload segment... Jul 26 22:18:57.634433002 (58h43m28.824731593s) [seg_pos={0 0}, upload=56321]: Done begin upload segment. Jul 26 22:18:57.634466006 (58h43m28.824764596s) [noise=true , address=104.36.82.12:4150 , piece_id=4PVRPBKTUNFKUWDJDTXYCIXDKONWG5OPQVB2M6KPKK66SMCYYHGA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.634507350 (58h43m28.82480593s) [noise=true , address=45.91.116.6:4973 , piece_id=S5BZKUNC2KNOVQMFRHSXOZNZZG6BUBK57ECM6W3S4YKN55KPOSFQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635011032 (58h43m28.825309612s) [noise=true , address=45.91.116.4:4938 , piece_id=INUA3CJT7KOAKKRYSBLTNXLX3MKWYYAFZ45BMUOZ6Y6I3B2EHHCQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635154508 (58h43m28.825453097s) [noise=true , address=45.91.116.8:4047 , piece_id=YTLRWKOOJ4XIGA3SKHFT2K66R57LBTW7OOVZSVJMCCV3YISPGWLA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635425419 (58h43m28.825724018s) [noise=true , address=45.91.116.8:4048 , piece_id=BF2LQK3BFGLPNCTDWB2M6HLRBSYBCXLR2AO7CMPERHT4DIBNKC4Q, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635823171 (58h43m28.826121762s) [noise=true , address=77.90.31.12:4590 , piece_id=PZSNEJOE623OVLNPZSRVVTTAGNHGRD7JCBUXCB3FXOMUE4CQNLFA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635848858 (58h43m28.826147438s) [noise=true , address=45.91.116.12:4227 , piece_id=FICCSE6IJQKUGYKTS3JHH4G65CPBOGDSPVIYXT25NR2GKRZQLOXA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.636411603 (58h43m28.826710186s) [noise=true , address=104.36.82.7:4909 , piece_id=HOFM67WVGW5QXUA65MEPQLJF7EV25AXBG4GNMREFYF2RNRGH2XWQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.636540327 (58h43m28.826838918s) [noise=true , address=77.90.31.11:4553 , piece_id=BP6FPURAP7D6RT34G72KZ32UY76QKXDG464WKNDPNUNOD6WS3KFQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.636837624 (58h43m28.827136209s) [noise=true , address=45.91.116.10:4081 , piece_id=2R44TDAMR2O3CNYFRZXOCSMG2ISUWMIQPVDWN3V2ECZWW4JUHNMA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.637107959 (58h43m28.827406542s) [noise=true , address=45.91.116.13:4154 , piece_id=PNBIV7A27TASCKR2CEQT7NJ5ZTG3NUQ2PLXJH7R47BFQQ53FLKUQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.637209450 (58h43m28.827508033s) [noise=true , address=77.90.31.21:4877 , piece_id=RCTFN3VAXG26GTA6GFQDQDIBAHJPOSI3QUSLLKO5APYYA57BSBRQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.637657116 (58h43m28.827955704s) [noise=true , address=45.91.116.7:4010 , piece_id=DZSLTEUQELXNHSH57IUZ6PRMOS56GR5TCWNAEEY7Y6YJ7KNDTRJA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.638167018 (58h43m28.828465602s) [noise=true , address=104.36.82.3:4767 , piece_id=IWRMBBOMGH7ZV3JA326TJ4SWHLEGA2T6CDLUDKJ4RFZBUZMTA3MQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.638722599 (58h43m28.829021185s) [noise=true , address=45.91.116.7:4009 , piece_id=2JHIJGUSNOSVSGOSGHHIYA2RNVLRS2P7MIKYHLSX4PTQOWNZKKGQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.639240448 (58h43m28.82953902s) [noise=true , address=104.36.82.11:4114 , piece_id=ZUBONJLMV4QRTG7DLIHWZICNNQ5OOF3HAPN26Q5AMK4TY4OYTXNA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:58.247247885 (58h43m29.437546466s) [upload=56321]: Next returned nil segment Jul 26 22:18:58.247273951 (58h43m29.437572537s) [upload=56321]: Waiting for error group managing segments... Jul 26 22:18:58.383663350 (58h43m29.573961931s) [noise=true , address=45.91.116.13:4154 , piece_id=PNBIV7A27TASCKR2CEQT7NJ5ZTG3NUQ2PLXJH7R47BFQQ53FLKUQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.384857324 (58h43m29.575155903s) [noise=true , address=45.91.116.8:4047 , piece_id=YTLRWKOOJ4XIGA3SKHFT2K66R57LBTW7OOVZSVJMCCV3YISPGWLA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.384981909 (58h43m29.575280495s) [noise=true , address=45.91.116.6:4973 , piece_id=S5BZKUNC2KNOVQMFRHSXOZNZZG6BUBK57ECM6W3S4YKN55KPOSFQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.387318149 (58h43m29.577616722s) [noise=true , address=45.91.116.7:4009 , piece_id=2JHIJGUSNOSVSGOSGHHIYA2RNVLRS2P7MIKYHLSX4PTQOWNZKKGQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.395512660 (58h43m29.585811243s) [noise=true , address=45.91.116.12:4227 , piece_id=FICCSE6IJQKUGYKTS3JHH4G65CPBOGDSPVIYXT25NR2GKRZQLOXA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.401895917 (58h43m29.592194499s) [noise=true , address=45.91.116.10:4081 , piece_id=2R44TDAMR2O3CNYFRZXOCSMG2ISUWMIQPVDWN3V2ECZWW4JUHNMA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.402135621 (58h43m29.59243419s) [noise=true , address=77.90.31.11:4553 , piece_id=BP6FPURAP7D6RT34G72KZ32UY76QKXDG464WKNDPNUNOD6WS3KFQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.403028120 (58h43m29.593326693s) [noise=true , address=45.91.116.8:4048 , piece_id=BF2LQK3BFGLPNCTDWB2M6HLRBSYBCXLR2AO7CMPERHT4DIBNKC4Q, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.404431307 (58h43m29.594729891s) [noise=true , address=45.91.116.4:4938 , piece_id=INUA3CJT7KOAKKRYSBLTNXLX3MKWYYAFZ45BMUOZ6Y6I3B2EHHCQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.409536042 (58h43m29.599834622s) [noise=true , address=45.91.116.7:4010 , piece_id=DZSLTEUQELXNHSH57IUZ6PRMOS56GR5TCWNAEEY7Y6YJ7KNDTRJA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:59.425708084 (58h43m30.616006681s) [noise=true , address=77.90.31.21:4877 , piece_id=RCTFN3VAXG26GTA6GFQDQDIBAHJPOSI3QUSLLKO5APYYA57BSBRQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:19:00.437139003 (58h43m31.627437582s) [noise=true , address=77.90.31.12:4590 , piece_id=PZSNEJOE623OVLNPZSRVVTTAGNHGRD7JCBUXCB3FXOMUE4CQNLFA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:19:27.246703337 (58h43m58.437001916s) [noise=true , address=104.36.82.7:4909 , piece_id=HOFM67WVGW5QXUA65MEPQLJF7EV25AXBG4GNMREFYF2RNRGH2XWQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: write tcp 139.59.144.22:57728->104.36.82.7:4909: use of closed network connection; write tcp 139.59.144.22:57728->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57728->104.36.82.7:4909: use of closed network connection Jul 26 22:19:27.246791447 (58h43m58.437090027s) [noise=true , address=104.36.82.11:4114 , piece_id=ZUBONJLMV4QRTG7DLIHWZICNNQ5OOF3HAPN26Q5AMK4TY4OYTXNA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 139.59.144.22:44152->104.36.82.11:4114: use of closed network connection; write tcp 139.59.144.22:44152->104.36.82.11:4114: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44152->104.36.82.11:4114: use of closed network connection Jul 26 22:19:27.246848250 (58h43m58.437146829s) [noise=true , address=104.36.82.3:4767 , piece_id=IWRMBBOMGH7ZV3JA326TJ4SWHLEGA2T6CDLUDKJ4RFZBUZMTA3MQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:52976->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:52976->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52976->104.36.82.3:4767: use of closed network connection Jul 26 22:19:27.246885606 (58h43m58.4371843s) [noise=true , address=104.36.82.12:4150 , piece_id=4PVRPBKTUNFKUWDJDTXYCIXDKONWG5OPQVB2M6KPKK66SMCYYHGA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:104.36.82.12:4150): protocol: write tcp 139.59.144.22:43452->104.36.82.12:4150: use of closed network connection; write tcp 139.59.144.22:43452->104.36.82.12:4150: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43452->104.36.82.12:4150: use of closed network connection Jul 26 22:19:27.246926689 (58h43m58.437225269s) [seg_pos={0 0}, upload=56321]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 22:19:27.354450344 (58h43m58.544748918s) [upload=56321]: Done uploading segments... Jul 26 22:19:27.354479247 (58h43m58.544777827s) [upload=56321]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 2.28429805s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 20 out of 60 is failed (threshold=3) [..................X.............XX...XXXXX.XXXX.XXXXX..XXX..] duration: 29.517871249s FAIL (2025-07-26 22:40:57 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 6 22:38:23.917529311 (59h2m55.107827904s) [upload=56631]: Done starting upload Jul 26 22:38:23.917846807 (59h2m55.108145388s) [upload=56631]: Uploading segments... Jul 26 22:38:23.917959648 (59h2m55.108258232s) [upload=56631]: Got next segment. Inline: false Jul 26 22:38:23.917998371 (59h2m55.108296951s) [upload=56631]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:38:23.918003592 (59h2m55.108302159s) [upload=56631]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:38:24.126845246 (59h2m55.317143827s) [seg_pos={0 0}, upload=56631]: Begin upload segment... Jul 26 22:38:24.126946250 (59h2m55.317244817s) [seg_pos={0 0}, upload=56631]: Done begin upload segment. Jul 26 22:38:24.126983024 (59h2m55.317281613s) [noise=true , address=79.127.205.234:4613 , piece_id=PB57BJOZXUAWC75ONCQFPIGV3F43VD5QGR5O53IUEY3SJLOYPYPQ, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.127171415 (59h2m55.317470008s) [noise=true , address=79.127.219.48:4090 , piece_id=TUWO3DINZR5GBD6V6UMNETD5SQORVNADN6Y7CVG7GD2VXYUPUQ5Q, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.127653251 (59h2m55.317951845s) [noise=true , address=45.91.116.10:4100 , piece_id=BB577NEP76YKWDZI3X7MZXJEVB4BGSZZLQU5AKOWVGFO5F5ZOXFQ, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.127862315 (59h2m55.318160904s) [noise=true , address=79.127.219.51:4210 , piece_id=WBHIJ3ETU4NQFVTVQLMCRC3KORGCP246B5TBKAVKHLA3QISKHIAA, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.128042586 (59h2m55.318341175s) [noise=true , address=77.90.31.21:4880 , piece_id=R2O7OT3PYZSZV3OZVXFUBOVASKM3CJTFLHMSGBYU42O4VM5IKEDQ, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.128393003 (59h2m55.318691592s) [noise=true , address=79.127.220.101:4048 , piece_id=EAATBR22YZLUAGBQBT2TLZIMJJWKG3VQRIZZHRGCH533MDUHIK6A, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.128427794 (59h2m55.318726374s) [noise=true , address=77.90.31.17:4792 , piece_id=QBKUGW647ZUJCAT3UQBS76GOYSGQFXTF2SXA3OSTRD5QMXUOAFOA, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.128812762 (59h2m55.319111353s) [noise=true , address=77.90.31.10:4543 , piece_id=PUEXSYUSSZ6HYO7KFSPKQTN56VD356OH65FGWDZZNJNMVYBBQ5CA, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.128938664 (59h2m55.319237252s) [noise=true , address=104.36.82.10:4101 , piece_id=BV3EQW57Y4G274PPJM2WS67TTIVXT2DWCOV4U6XVB2RPACQNJQUA, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.129160636 (59h2m55.319459225s) [noise=true , address=104.36.82.45:4925 , piece_id=OFKLTRXDEPOGW7CUKDT7U6DUCJBP7H2T3KYHQUBSD5E35IM4SGRQ, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.129309084 (59h2m55.319607673s) [noise=true , address=77.90.31.12:4611 , piece_id=W6HPFILAQM2GUYNGFAVVVZMM3J7SPIQY4VGBFCO5U6IIYP2WLHJQ, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.129493745 (59h2m55.319792336s) [noise=true , address=64.49.13.140:4516 , piece_id=DPN4UUJLBJ3EEQFACSZQKJCJGGSLH6WB673M4DAREN2EF3FQNWMQ, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.129803640 (59h2m55.32010223s) [noise=true , address=64.49.13.170:4721 , piece_id=IZWGA6PE57DVSRNG6PSUMWWH5S2STZMREIQGOKCD6HI6DFZWKNXQ, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.129817888 (59h2m55.320116478s) [noise=true , address=156.146.43.225:4448 , piece_id=XIFOWF3YDDQNP5JQS7NPYW5ARQOURRLHCEK6PU5J3AEUL43T4BYA, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.130310600 (59h2m55.320609186s) [noise=true , address=45.91.116.12:4229 , piece_id=KPDDGJ5SGXZGJFWTVUE7M4MISTLEL5DIGHNPEWSMLSTUUQFEZS5Q, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.134915854 (59h2m55.325214436s) [noise=true , address=79.127.205.235:4689 , piece_id=Q7DIHD25XP42FK6CYCO3BEGD632YPLUMYPBJKEZCBHS4YZWVJVFQ, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.135399255 (59h2m55.325697837s) [noise=true , address=79.127.219.38:4013 , piece_id=HP5XJHQM6W5KOYDMVH2JQN7XWCRRG5ZLKGRE22HS65MFFXVNZU4Q, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.135794642 (59h2m55.32609323s) [noise=true , address=79.127.205.246:4270 , piece_id=AIM5XOGJTJ7QQ3PPDCQ25ERTFBGRJI6UXNJAG6LDOGV5CKUGJI3A, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.550256489 (59h2m55.740555068s) [noise=true , address=79.127.219.51:4210 , piece_id=WBHIJ3ETU4NQFVTVQLMCRC3KORGCP246B5TBKAVKHLA3QISKHIAA, seg_pos={0 0}, upload=56631]: Done uploading piece. err: ecclient: upload failed (node:1yCKELQfksAs1CLfacoYZgpHGzRM3UTGak3eWPu9ds3qn1a9G2, address:79.127.219.51:4210): protocol: storage node overloaded, request limit: 2000 Jul 26 22:38:24.687416075 (59h2m55.877714644s) [upload=56631]: Next returned nil segment Jul 26 22:38:24.687434636 (59h2m55.877733205s) [upload=56631]: Waiting for error group managing segments... Jul 26 22:38:24.869879720 (59h2m56.060178307s) [noise=true , address=77.90.31.17:4792 , piece_id=QBKUGW647ZUJCAT3UQBS76GOYSGQFXTF2SXA3OSTRD5QMXUOAFOA, seg_pos={0 0}, upload=56631]: Done uploading piece. err: <nil> Jul 26 22:38:24.870055848 (59h2m56.060354452s) [noise=true , address=77.90.31.10:4543 , piece_id=PUEXSYUSSZ6HYO7KFSPKQTN56VD356OH65FGWDZZNJNMVYBBQ5CA, seg_pos={0 0}, upload=56631]: Done uploading piece. err: <nil> Jul 26 22:38:24.878082696 (59h2m56.068381283s) [noise=true , address=77.90.31.12:4611 , piece_id=W6HPFILAQM2GUYNGFAVVVZMM3J7SPIQY4VGBFCO5U6IIYP2WLHJQ, seg_pos={0 0}, upload=56631]: Done uploading piece. err: <nil> Jul 26 22:38:24.904762702 (59h2m56.095061281s) [noise=true , address=64.49.13.170:4721 , piece_id=IZWGA6PE57DVSRNG6PSUMWWH5S2STZMREIQGOKCD6HI6DFZWKNXQ, seg_pos={0 0}, upload=56631]: Done uploading piece. err: ecclient: upload failed (node:1MYa3uS5XnUo4DEvPvXbjcKdCqddUUKbi7L6HV9q6VgyzehyD3, address:64.49.13.170:4721): protocol: storage node overloaded, request limit: 2000 Jul 26 22:38:24.908276832 (59h2m56.098575415s) [noise=true , address=79.127.219.38:4013 , piece_id=HP5XJHQM6W5KOYDMVH2JQN7XWCRRG5ZLKGRE22HS65MFFXVNZU4Q, seg_pos={0 0}, upload=56631]: Done uploading piece. err: <nil> Jul 26 22:38:24.949670169 (59h2m56.139968738s) [noise=true , address=77.90.31.21:4880 , piece_id=R2O7OT3PYZSZV3OZVXFUBOVASKM3CJTFLHMSGBYU42O4VM5IKEDQ, seg_pos={0 0}, upload=56631]: Done uploading piece. err: <nil> Jul 26 22:38:25.347251577 (59h2m56.537550146s) [noise=true , address=79.127.205.234:4613 , piece_id=PB57BJOZXUAWC75ONCQFPIGV3F43VD5QGR5O53IUEY3SJLOYPYPQ, seg_pos={0 0}, upload=56631]: Done uploading piece. err: <nil> Jul 26 22:38:25.439562747 (59h2m56.629861316s) [noise=true , address=79.127.205.235:4689 , piece_id=Q7DIHD25XP42FK6CYCO3BEGD632YPLUMYPBJKEZCBHS4YZWVJVFQ, seg_pos={0 0}, upload=56631]: Done uploading piece. err: <nil> Jul 26 22:38:25.725950048 (59h2m56.916248627s) [noise=true , address=79.127.205.246:4270 , piece_id=AIM5XOGJTJ7QQ3PPDCQ25ERTFBGRJI6UXNJAG6LDOGV5CKUGJI3A, seg_pos={0 0}, upload=56631]: Done uploading piece. err: <nil> Jul 26 22:38:27.210185703 (59h2m58.400484272s) [noise=true , address=79.127.219.48:4090 , piece_id=TUWO3DINZR5GBD6V6UMNETD5SQORVNADN6Y7CVG7GD2VXYUPUQ5Q, seg_pos={0 0}, upload=56631]: Done uploading piece. err: <nil> Jul 26 22:38:27.709038123 (59h2m58.899336692s) [noise=true , address=156.146.43.225:4448 , piece_id=XIFOWF3YDDQNP5JQS7NPYW5ARQOURRLHCEK6PU5J3AEUL43T4BYA, seg_pos={0 0}, upload=56631]: Done uploading piece. err: <nil> Jul 26 22:38:28.776879313 (59h2m59.967177906s) [noise=true , address=79.127.220.101:4048 , piece_id=EAATBR22YZLUAGBQBT2TLZIMJJWKG3VQRIZZHRGCH533MDUHIK6A, seg_pos={0 0}, upload=56631]: Done uploading piece. err: <nil> Jul 26 22:38:29.935859494 (59h3m1.126158078s) [noise=true , address=64.49.13.140:4516 , piece_id=DPN4UUJLBJ3EEQFACSZQKJCJGGSLH6WB673M4DAREN2EF3FQNWMQ, seg_pos={0 0}, upload=56631]: Done uploading piece. err: <nil> Jul 26 22:38:32.767735980 (59h3m3.95803455s) [noise=true , address=45.91.116.12:4229 , piece_id=KPDDGJ5SGXZGJFWTVUE7M4MISTLEL5DIGHNPEWSMLSTUUQFEZS5Q, seg_pos={0 0}, upload=56631]: Done uploading piece. err: ecclient: upload failed (node:12rfuAG41swAn82EM5ZTAN31NhGp4UKMHFiZiT57NXmSAi3MHe2, address:45.91.116.12:4229): protocol: storage node overloaded, request limit: 2000 Jul 26 22:38:54.325490826 (59h3m25.515789399s) [noise=true , address=104.36.82.45:4925 , piece_id=OFKLTRXDEPOGW7CUKDT7U6DUCJBP7H2T3KYHQUBSD5E35IM4SGRQ, seg_pos={0 0}, upload=56631]: Done uploading piece. err: <nil> Jul 26 22:39:23.643352570 (59h3m54.833651156s) [noise=true , address=104.36.82.10:4101 , piece_id=BV3EQW57Y4G274PPJM2WS67TTIVXT2DWCOV4U6XVB2RPACQNJQUA, seg_pos={0 0}, upload=56631]: Done uploading piece. err: ecclient: upload failed (node:1RmTDQrWoQBSpHrcUsb2PYijpcFdCyRya7VrMkxhWeQXdGeQpp, address:104.36.82.10:4101): protocol: write tcp 139.59.144.22:45976->104.36.82.10:4101: use of closed network connection; write tcp 139.59.144.22:45976->104.36.82.10:4101: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:45976->104.36.82.10:4101: use of closed network connection Jul 26 22:39:23.643418875 (59h3m54.833717453s) [noise=true , address=45.91.116.10:4100 , piece_id=BB577NEP76YKWDZI3X7MZXJEVB4BGSZZLQU5AKOWVGFO5F5ZOXFQ, seg_pos={0 0}, upload=56631]: Done uploading piece. err: ecclient: upload failed (node:12PQzzS3ZUKXCpiPHqTokRAC9MUuU8QQp7fJTDydmYehhs21YBn, address:45.91.116.10:4100): protocol: write tcp 139.59.144.22:33428->45.91.116.10:4100: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 22:39:23.643454601 (59h3m54.833753171s) [seg_pos={0 0}, upload=56631]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 26 22:39:23.749096004 (59h3m54.939394577s) [upload=56631]: Done uploading segments... Jul 26 22:39:23.749123123 (59h3m54.939421691s) [upload=56631]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 29.517906293s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 10 out of 60 is failed (threshold=3) [...............................X.......X..X...X.XX...X..X.XX] duration: 1m0.001371467s FAIL (2025-07-26 22:40:43 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001389067s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [.X..XXX....X.X.X...............X............................] duration: 3.660662812s FAIL (2025-07-26 22:40:21 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 988 (58h42m45.061082575s) [upload=56308]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:18:14.018433272 (58h42m45.208731856s) [upload=56308]: Done uploading segments... Jul 26 22:18:14.018484438 (58h42m45.208783019s) [upload=56308]: Upload finished. err: <nil> Jul 26 22:18:57.520834389 (58h43m28.711132968s) [upload=56321]: Starting upload Jul 26 22:18:57.520934224 (58h43m28.711232794s) [upload=56321]: Done starting upload Jul 26 22:18:57.521309042 (58h43m28.711607621s) [upload=56321]: Uploading segments... Jul 26 22:18:57.521389036 (58h43m28.711687618s) [upload=56321]: Got next segment. Inline: false Jul 26 22:18:57.521430569 (58h43m28.711729148s) [upload=56321]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:18:57.521435990 (58h43m28.711734568s) [upload=56321]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:18:57.634354903 (58h43m28.824653493s) [seg_pos={0 0}, upload=56321]: Begin upload segment... Jul 26 22:18:57.634433002 (58h43m28.824731593s) [seg_pos={0 0}, upload=56321]: Done begin upload segment. Jul 26 22:18:57.634466006 (58h43m28.824764596s) [noise=true , address=104.36.82.12:4150 , piece_id=4PVRPBKTUNFKUWDJDTXYCIXDKONWG5OPQVB2M6KPKK66SMCYYHGA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.634507350 (58h43m28.82480593s) [noise=true , address=45.91.116.6:4973 , piece_id=S5BZKUNC2KNOVQMFRHSXOZNZZG6BUBK57ECM6W3S4YKN55KPOSFQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635011032 (58h43m28.825309612s) [noise=true , address=45.91.116.4:4938 , piece_id=INUA3CJT7KOAKKRYSBLTNXLX3MKWYYAFZ45BMUOZ6Y6I3B2EHHCQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635154508 (58h43m28.825453097s) [noise=true , address=45.91.116.8:4047 , piece_id=YTLRWKOOJ4XIGA3SKHFT2K66R57LBTW7OOVZSVJMCCV3YISPGWLA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635425419 (58h43m28.825724018s) [noise=true , address=45.91.116.8:4048 , piece_id=BF2LQK3BFGLPNCTDWB2M6HLRBSYBCXLR2AO7CMPERHT4DIBNKC4Q, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635823171 (58h43m28.826121762s) [noise=true , address=77.90.31.12:4590 , piece_id=PZSNEJOE623OVLNPZSRVVTTAGNHGRD7JCBUXCB3FXOMUE4CQNLFA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635848858 (58h43m28.826147438s) [noise=true , address=45.91.116.12:4227 , piece_id=FICCSE6IJQKUGYKTS3JHH4G65CPBOGDSPVIYXT25NR2GKRZQLOXA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.636411603 (58h43m28.826710186s) [noise=true , address=104.36.82.7:4909 , piece_id=HOFM67WVGW5QXUA65MEPQLJF7EV25AXBG4GNMREFYF2RNRGH2XWQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.636540327 (58h43m28.826838918s) [noise=true , address=77.90.31.11:4553 , piece_id=BP6FPURAP7D6RT34G72KZ32UY76QKXDG464WKNDPNUNOD6WS3KFQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.636837624 (58h43m28.827136209s) [noise=true , address=45.91.116.10:4081 , piece_id=2R44TDAMR2O3CNYFRZXOCSMG2ISUWMIQPVDWN3V2ECZWW4JUHNMA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.637107959 (58h43m28.827406542s) [noise=true , address=45.91.116.13:4154 , piece_id=PNBIV7A27TASCKR2CEQT7NJ5ZTG3NUQ2PLXJH7R47BFQQ53FLKUQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.637209450 (58h43m28.827508033s) [noise=true , address=77.90.31.21:4877 , piece_id=RCTFN3VAXG26GTA6GFQDQDIBAHJPOSI3QUSLLKO5APYYA57BSBRQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.637657116 (58h43m28.827955704s) [noise=true , address=45.91.116.7:4010 , piece_id=DZSLTEUQELXNHSH57IUZ6PRMOS56GR5TCWNAEEY7Y6YJ7KNDTRJA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.638167018 (58h43m28.828465602s) [noise=true , address=104.36.82.3:4767 , piece_id=IWRMBBOMGH7ZV3JA326TJ4SWHLEGA2T6CDLUDKJ4RFZBUZMTA3MQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.638722599 (58h43m28.829021185s) [noise=true , address=45.91.116.7:4009 , piece_id=2JHIJGUSNOSVSGOSGHHIYA2RNVLRS2P7MIKYHLSX4PTQOWNZKKGQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.639240448 (58h43m28.82953902s) [noise=true , address=104.36.82.11:4114 , piece_id=ZUBONJLMV4QRTG7DLIHWZICNNQ5OOF3HAPN26Q5AMK4TY4OYTXNA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:58.247247885 (58h43m29.437546466s) [upload=56321]: Next returned nil segment Jul 26 22:18:58.247273951 (58h43m29.437572537s) [upload=56321]: Waiting for error group managing segments... Jul 26 22:18:58.383663350 (58h43m29.573961931s) [noise=true , address=45.91.116.13:4154 , piece_id=PNBIV7A27TASCKR2CEQT7NJ5ZTG3NUQ2PLXJH7R47BFQQ53FLKUQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.384857324 (58h43m29.575155903s) [noise=true , address=45.91.116.8:4047 , piece_id=YTLRWKOOJ4XIGA3SKHFT2K66R57LBTW7OOVZSVJMCCV3YISPGWLA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.384981909 (58h43m29.575280495s) [noise=true , address=45.91.116.6:4973 , piece_id=S5BZKUNC2KNOVQMFRHSXOZNZZG6BUBK57ECM6W3S4YKN55KPOSFQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.387318149 (58h43m29.577616722s) [noise=true , address=45.91.116.7:4009 , piece_id=2JHIJGUSNOSVSGOSGHHIYA2RNVLRS2P7MIKYHLSX4PTQOWNZKKGQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.395512660 (58h43m29.585811243s) [noise=true , address=45.91.116.12:4227 , piece_id=FICCSE6IJQKUGYKTS3JHH4G65CPBOGDSPVIYXT25NR2GKRZQLOXA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.401895917 (58h43m29.592194499s) [noise=true , address=45.91.116.10:4081 , piece_id=2R44TDAMR2O3CNYFRZXOCSMG2ISUWMIQPVDWN3V2ECZWW4JUHNMA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.402135621 (58h43m29.59243419s) [noise=true , address=77.90.31.11:4553 , piece_id=BP6FPURAP7D6RT34G72KZ32UY76QKXDG464WKNDPNUNOD6WS3KFQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.403028120 (58h43m29.593326693s) [noise=true , address=45.91.116.8:4048 , piece_id=BF2LQK3BFGLPNCTDWB2M6HLRBSYBCXLR2AO7CMPERHT4DIBNKC4Q, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.404431307 (58h43m29.594729891s) [noise=true , address=45.91.116.4:4938 , piece_id=INUA3CJT7KOAKKRYSBLTNXLX3MKWYYAFZ45BMUOZ6Y6I3B2EHHCQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.409536042 (58h43m29.599834622s) [noise=true , address=45.91.116.7:4010 , piece_id=DZSLTEUQELXNHSH57IUZ6PRMOS56GR5TCWNAEEY7Y6YJ7KNDTRJA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:59.425708084 (58h43m30.616006681s) [noise=true , address=77.90.31.21:4877 , piece_id=RCTFN3VAXG26GTA6GFQDQDIBAHJPOSI3QUSLLKO5APYYA57BSBRQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:19:00.437139003 (58h43m31.627437582s) [noise=true , address=77.90.31.12:4590 , piece_id=PZSNEJOE623OVLNPZSRVVTTAGNHGRD7JCBUXCB3FXOMUE4CQNLFA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:19:27.246703337 (58h43m58.437001916s) [noise=true , address=104.36.82.7:4909 , piece_id=HOFM67WVGW5QXUA65MEPQLJF7EV25AXBG4GNMREFYF2RNRGH2XWQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: write tcp 139.59.144.22:57728->104.36.82.7:4909: use of closed network connection; write tcp 139.59.144.22:57728->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57728->104.36.82.7:4909: use of closed network connection Jul 26 22:19:27.246791447 (58h43m58.437090027s) [noise=true , address=104.36.82.11:4114 , piece_id=ZUBONJLMV4QRTG7DLIHWZICNNQ5OOF3HAPN26Q5AMK4TY4OYTXNA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 139.59.144.22:44152->104.36.82.11:4114: use of closed network connection; write tcp 139.59.144.22:44152->104.36.82.11:4114: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44152->104.36.82.11:4114: use of closed network connection Jul 26 22:19:27.246848250 (58h43m58.437146829s) [noise=true , address=104.36.82.3:4767 , piece_id=IWRMBBOMGH7ZV3JA326TJ4SWHLEGA2T6CDLUDKJ4RFZBUZMTA3MQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:52976->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:52976->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52976->104.36.82.3:4767: use of closed network connection Jul 26 22:19:27.246885606 (58h43m58.4371843s) [noise=true , address=104.36.82.12:4150 , piece_id=4PVRPBKTUNFKUWDJDTXYCIXDKONWG5OPQVB2M6KPKK66SMCYYHGA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:104.36.82.12:4150): protocol: write tcp 139.59.144.22:43452->104.36.82.12:4150: use of closed network connection; write tcp 139.59.144.22:43452->104.36.82.12:4150: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43452->104.36.82.12:4150: use of closed network connection Jul 26 22:19:27.246926689 (58h43m58.437225269s) [seg_pos={0 0}, upload=56321]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 22:19:27.354450344 (58h43m58.544748918s) [upload=56321]: Done uploading segments... Jul 26 22:19:27.354479247 (58h43m58.544777827s) [upload=56321]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 3.660711261s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 921.772176ms FAIL (2025-07-26 22:40:11 UTC) orbiter warning: digest mismatch: after b0275bfe409329e90caa68828ca9cf99d935619c40adf20794b6b4a165f10dac != before 689768e222479cddedfe8605c20bdb7052e22d65868c907a97a7972f7764c1dd Duration: 921.80725ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........X.X.........................X.....................] duration: 2.402701413s FAIL (2025-07-26 22:39:59 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.402726847s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [..X..XXX....X.X.X...............X...........................] duration: 2.319501235s FAIL (2025-07-26 22:39:54 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 988 (58h42m45.061082575s) [upload=56308]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:18:14.018433272 (58h42m45.208731856s) [upload=56308]: Done uploading segments... Jul 26 22:18:14.018484438 (58h42m45.208783019s) [upload=56308]: Upload finished. err: <nil> Jul 26 22:18:57.520834389 (58h43m28.711132968s) [upload=56321]: Starting upload Jul 26 22:18:57.520934224 (58h43m28.711232794s) [upload=56321]: Done starting upload Jul 26 22:18:57.521309042 (58h43m28.711607621s) [upload=56321]: Uploading segments... Jul 26 22:18:57.521389036 (58h43m28.711687618s) [upload=56321]: Got next segment. Inline: false Jul 26 22:18:57.521430569 (58h43m28.711729148s) [upload=56321]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:18:57.521435990 (58h43m28.711734568s) [upload=56321]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:18:57.634354903 (58h43m28.824653493s) [seg_pos={0 0}, upload=56321]: Begin upload segment... Jul 26 22:18:57.634433002 (58h43m28.824731593s) [seg_pos={0 0}, upload=56321]: Done begin upload segment. Jul 26 22:18:57.634466006 (58h43m28.824764596s) [noise=true , address=104.36.82.12:4150 , piece_id=4PVRPBKTUNFKUWDJDTXYCIXDKONWG5OPQVB2M6KPKK66SMCYYHGA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.634507350 (58h43m28.82480593s) [noise=true , address=45.91.116.6:4973 , piece_id=S5BZKUNC2KNOVQMFRHSXOZNZZG6BUBK57ECM6W3S4YKN55KPOSFQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635011032 (58h43m28.825309612s) [noise=true , address=45.91.116.4:4938 , piece_id=INUA3CJT7KOAKKRYSBLTNXLX3MKWYYAFZ45BMUOZ6Y6I3B2EHHCQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635154508 (58h43m28.825453097s) [noise=true , address=45.91.116.8:4047 , piece_id=YTLRWKOOJ4XIGA3SKHFT2K66R57LBTW7OOVZSVJMCCV3YISPGWLA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635425419 (58h43m28.825724018s) [noise=true , address=45.91.116.8:4048 , piece_id=BF2LQK3BFGLPNCTDWB2M6HLRBSYBCXLR2AO7CMPERHT4DIBNKC4Q, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635823171 (58h43m28.826121762s) [noise=true , address=77.90.31.12:4590 , piece_id=PZSNEJOE623OVLNPZSRVVTTAGNHGRD7JCBUXCB3FXOMUE4CQNLFA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635848858 (58h43m28.826147438s) [noise=true , address=45.91.116.12:4227 , piece_id=FICCSE6IJQKUGYKTS3JHH4G65CPBOGDSPVIYXT25NR2GKRZQLOXA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.636411603 (58h43m28.826710186s) [noise=true , address=104.36.82.7:4909 , piece_id=HOFM67WVGW5QXUA65MEPQLJF7EV25AXBG4GNMREFYF2RNRGH2XWQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.636540327 (58h43m28.826838918s) [noise=true , address=77.90.31.11:4553 , piece_id=BP6FPURAP7D6RT34G72KZ32UY76QKXDG464WKNDPNUNOD6WS3KFQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.636837624 (58h43m28.827136209s) [noise=true , address=45.91.116.10:4081 , piece_id=2R44TDAMR2O3CNYFRZXOCSMG2ISUWMIQPVDWN3V2ECZWW4JUHNMA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.637107959 (58h43m28.827406542s) [noise=true , address=45.91.116.13:4154 , piece_id=PNBIV7A27TASCKR2CEQT7NJ5ZTG3NUQ2PLXJH7R47BFQQ53FLKUQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.637209450 (58h43m28.827508033s) [noise=true , address=77.90.31.21:4877 , piece_id=RCTFN3VAXG26GTA6GFQDQDIBAHJPOSI3QUSLLKO5APYYA57BSBRQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.637657116 (58h43m28.827955704s) [noise=true , address=45.91.116.7:4010 , piece_id=DZSLTEUQELXNHSH57IUZ6PRMOS56GR5TCWNAEEY7Y6YJ7KNDTRJA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.638167018 (58h43m28.828465602s) [noise=true , address=104.36.82.3:4767 , piece_id=IWRMBBOMGH7ZV3JA326TJ4SWHLEGA2T6CDLUDKJ4RFZBUZMTA3MQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.638722599 (58h43m28.829021185s) [noise=true , address=45.91.116.7:4009 , piece_id=2JHIJGUSNOSVSGOSGHHIYA2RNVLRS2P7MIKYHLSX4PTQOWNZKKGQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.639240448 (58h43m28.82953902s) [noise=true , address=104.36.82.11:4114 , piece_id=ZUBONJLMV4QRTG7DLIHWZICNNQ5OOF3HAPN26Q5AMK4TY4OYTXNA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:58.247247885 (58h43m29.437546466s) [upload=56321]: Next returned nil segment Jul 26 22:18:58.247273951 (58h43m29.437572537s) [upload=56321]: Waiting for error group managing segments... Jul 26 22:18:58.383663350 (58h43m29.573961931s) [noise=true , address=45.91.116.13:4154 , piece_id=PNBIV7A27TASCKR2CEQT7NJ5ZTG3NUQ2PLXJH7R47BFQQ53FLKUQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.384857324 (58h43m29.575155903s) [noise=true , address=45.91.116.8:4047 , piece_id=YTLRWKOOJ4XIGA3SKHFT2K66R57LBTW7OOVZSVJMCCV3YISPGWLA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.384981909 (58h43m29.575280495s) [noise=true , address=45.91.116.6:4973 , piece_id=S5BZKUNC2KNOVQMFRHSXOZNZZG6BUBK57ECM6W3S4YKN55KPOSFQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.387318149 (58h43m29.577616722s) [noise=true , address=45.91.116.7:4009 , piece_id=2JHIJGUSNOSVSGOSGHHIYA2RNVLRS2P7MIKYHLSX4PTQOWNZKKGQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.395512660 (58h43m29.585811243s) [noise=true , address=45.91.116.12:4227 , piece_id=FICCSE6IJQKUGYKTS3JHH4G65CPBOGDSPVIYXT25NR2GKRZQLOXA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.401895917 (58h43m29.592194499s) [noise=true , address=45.91.116.10:4081 , piece_id=2R44TDAMR2O3CNYFRZXOCSMG2ISUWMIQPVDWN3V2ECZWW4JUHNMA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.402135621 (58h43m29.59243419s) [noise=true , address=77.90.31.11:4553 , piece_id=BP6FPURAP7D6RT34G72KZ32UY76QKXDG464WKNDPNUNOD6WS3KFQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.403028120 (58h43m29.593326693s) [noise=true , address=45.91.116.8:4048 , piece_id=BF2LQK3BFGLPNCTDWB2M6HLRBSYBCXLR2AO7CMPERHT4DIBNKC4Q, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.404431307 (58h43m29.594729891s) [noise=true , address=45.91.116.4:4938 , piece_id=INUA3CJT7KOAKKRYSBLTNXLX3MKWYYAFZ45BMUOZ6Y6I3B2EHHCQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.409536042 (58h43m29.599834622s) [noise=true , address=45.91.116.7:4010 , piece_id=DZSLTEUQELXNHSH57IUZ6PRMOS56GR5TCWNAEEY7Y6YJ7KNDTRJA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:59.425708084 (58h43m30.616006681s) [noise=true , address=77.90.31.21:4877 , piece_id=RCTFN3VAXG26GTA6GFQDQDIBAHJPOSI3QUSLLKO5APYYA57BSBRQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:19:00.437139003 (58h43m31.627437582s) [noise=true , address=77.90.31.12:4590 , piece_id=PZSNEJOE623OVLNPZSRVVTTAGNHGRD7JCBUXCB3FXOMUE4CQNLFA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:19:27.246703337 (58h43m58.437001916s) [noise=true , address=104.36.82.7:4909 , piece_id=HOFM67WVGW5QXUA65MEPQLJF7EV25AXBG4GNMREFYF2RNRGH2XWQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: write tcp 139.59.144.22:57728->104.36.82.7:4909: use of closed network connection; write tcp 139.59.144.22:57728->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57728->104.36.82.7:4909: use of closed network connection Jul 26 22:19:27.246791447 (58h43m58.437090027s) [noise=true , address=104.36.82.11:4114 , piece_id=ZUBONJLMV4QRTG7DLIHWZICNNQ5OOF3HAPN26Q5AMK4TY4OYTXNA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 139.59.144.22:44152->104.36.82.11:4114: use of closed network connection; write tcp 139.59.144.22:44152->104.36.82.11:4114: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44152->104.36.82.11:4114: use of closed network connection Jul 26 22:19:27.246848250 (58h43m58.437146829s) [noise=true , address=104.36.82.3:4767 , piece_id=IWRMBBOMGH7ZV3JA326TJ4SWHLEGA2T6CDLUDKJ4RFZBUZMTA3MQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:52976->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:52976->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52976->104.36.82.3:4767: use of closed network connection Jul 26 22:19:27.246885606 (58h43m58.4371843s) [noise=true , address=104.36.82.12:4150 , piece_id=4PVRPBKTUNFKUWDJDTXYCIXDKONWG5OPQVB2M6KPKK66SMCYYHGA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:104.36.82.12:4150): protocol: write tcp 139.59.144.22:43452->104.36.82.12:4150: use of closed network connection; write tcp 139.59.144.22:43452->104.36.82.12:4150: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43452->104.36.82.12:4150: use of closed network connection Jul 26 22:19:27.246926689 (58h43m58.437225269s) [seg_pos={0 0}, upload=56321]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 22:19:27.354450344 (58h43m58.544748918s) [upload=56321]: Done uploading segments... Jul 26 22:19:27.354479247 (58h43m58.544777827s) [upload=56321]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 2.319529901s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 20 out of 60 is failed (threshold=3) [...................X.............XX...XXXXX.XXXX.XXXXX..XXX.] duration: 22.772983243s FAIL (2025-07-26 22:39:46 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 6 22:38:23.917529311 (59h2m55.107827904s) [upload=56631]: Done starting upload Jul 26 22:38:23.917846807 (59h2m55.108145388s) [upload=56631]: Uploading segments... Jul 26 22:38:23.917959648 (59h2m55.108258232s) [upload=56631]: Got next segment. Inline: false Jul 26 22:38:23.917998371 (59h2m55.108296951s) [upload=56631]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:38:23.918003592 (59h2m55.108302159s) [upload=56631]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:38:24.126845246 (59h2m55.317143827s) [seg_pos={0 0}, upload=56631]: Begin upload segment... Jul 26 22:38:24.126946250 (59h2m55.317244817s) [seg_pos={0 0}, upload=56631]: Done begin upload segment. Jul 26 22:38:24.126983024 (59h2m55.317281613s) [noise=true , address=79.127.205.234:4613 , piece_id=PB57BJOZXUAWC75ONCQFPIGV3F43VD5QGR5O53IUEY3SJLOYPYPQ, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.127171415 (59h2m55.317470008s) [noise=true , address=79.127.219.48:4090 , piece_id=TUWO3DINZR5GBD6V6UMNETD5SQORVNADN6Y7CVG7GD2VXYUPUQ5Q, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.127653251 (59h2m55.317951845s) [noise=true , address=45.91.116.10:4100 , piece_id=BB577NEP76YKWDZI3X7MZXJEVB4BGSZZLQU5AKOWVGFO5F5ZOXFQ, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.127862315 (59h2m55.318160904s) [noise=true , address=79.127.219.51:4210 , piece_id=WBHIJ3ETU4NQFVTVQLMCRC3KORGCP246B5TBKAVKHLA3QISKHIAA, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.128042586 (59h2m55.318341175s) [noise=true , address=77.90.31.21:4880 , piece_id=R2O7OT3PYZSZV3OZVXFUBOVASKM3CJTFLHMSGBYU42O4VM5IKEDQ, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.128393003 (59h2m55.318691592s) [noise=true , address=79.127.220.101:4048 , piece_id=EAATBR22YZLUAGBQBT2TLZIMJJWKG3VQRIZZHRGCH533MDUHIK6A, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.128427794 (59h2m55.318726374s) [noise=true , address=77.90.31.17:4792 , piece_id=QBKUGW647ZUJCAT3UQBS76GOYSGQFXTF2SXA3OSTRD5QMXUOAFOA, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.128812762 (59h2m55.319111353s) [noise=true , address=77.90.31.10:4543 , piece_id=PUEXSYUSSZ6HYO7KFSPKQTN56VD356OH65FGWDZZNJNMVYBBQ5CA, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.128938664 (59h2m55.319237252s) [noise=true , address=104.36.82.10:4101 , piece_id=BV3EQW57Y4G274PPJM2WS67TTIVXT2DWCOV4U6XVB2RPACQNJQUA, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.129160636 (59h2m55.319459225s) [noise=true , address=104.36.82.45:4925 , piece_id=OFKLTRXDEPOGW7CUKDT7U6DUCJBP7H2T3KYHQUBSD5E35IM4SGRQ, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.129309084 (59h2m55.319607673s) [noise=true , address=77.90.31.12:4611 , piece_id=W6HPFILAQM2GUYNGFAVVVZMM3J7SPIQY4VGBFCO5U6IIYP2WLHJQ, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.129493745 (59h2m55.319792336s) [noise=true , address=64.49.13.140:4516 , piece_id=DPN4UUJLBJ3EEQFACSZQKJCJGGSLH6WB673M4DAREN2EF3FQNWMQ, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.129803640 (59h2m55.32010223s) [noise=true , address=64.49.13.170:4721 , piece_id=IZWGA6PE57DVSRNG6PSUMWWH5S2STZMREIQGOKCD6HI6DFZWKNXQ, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.129817888 (59h2m55.320116478s) [noise=true , address=156.146.43.225:4448 , piece_id=XIFOWF3YDDQNP5JQS7NPYW5ARQOURRLHCEK6PU5J3AEUL43T4BYA, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.130310600 (59h2m55.320609186s) [noise=true , address=45.91.116.12:4229 , piece_id=KPDDGJ5SGXZGJFWTVUE7M4MISTLEL5DIGHNPEWSMLSTUUQFEZS5Q, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.134915854 (59h2m55.325214436s) [noise=true , address=79.127.205.235:4689 , piece_id=Q7DIHD25XP42FK6CYCO3BEGD632YPLUMYPBJKEZCBHS4YZWVJVFQ, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.135399255 (59h2m55.325697837s) [noise=true , address=79.127.219.38:4013 , piece_id=HP5XJHQM6W5KOYDMVH2JQN7XWCRRG5ZLKGRE22HS65MFFXVNZU4Q, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.135794642 (59h2m55.32609323s) [noise=true , address=79.127.205.246:4270 , piece_id=AIM5XOGJTJ7QQ3PPDCQ25ERTFBGRJI6UXNJAG6LDOGV5CKUGJI3A, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.550256489 (59h2m55.740555068s) [noise=true , address=79.127.219.51:4210 , piece_id=WBHIJ3ETU4NQFVTVQLMCRC3KORGCP246B5TBKAVKHLA3QISKHIAA, seg_pos={0 0}, upload=56631]: Done uploading piece. err: ecclient: upload failed (node:1yCKELQfksAs1CLfacoYZgpHGzRM3UTGak3eWPu9ds3qn1a9G2, address:79.127.219.51:4210): protocol: storage node overloaded, request limit: 2000 Jul 26 22:38:24.687416075 (59h2m55.877714644s) [upload=56631]: Next returned nil segment Jul 26 22:38:24.687434636 (59h2m55.877733205s) [upload=56631]: Waiting for error group managing segments... Jul 26 22:38:24.869879720 (59h2m56.060178307s) [noise=true , address=77.90.31.17:4792 , piece_id=QBKUGW647ZUJCAT3UQBS76GOYSGQFXTF2SXA3OSTRD5QMXUOAFOA, seg_pos={0 0}, upload=56631]: Done uploading piece. err: <nil> Jul 26 22:38:24.870055848 (59h2m56.060354452s) [noise=true , address=77.90.31.10:4543 , piece_id=PUEXSYUSSZ6HYO7KFSPKQTN56VD356OH65FGWDZZNJNMVYBBQ5CA, seg_pos={0 0}, upload=56631]: Done uploading piece. err: <nil> Jul 26 22:38:24.878082696 (59h2m56.068381283s) [noise=true , address=77.90.31.12:4611 , piece_id=W6HPFILAQM2GUYNGFAVVVZMM3J7SPIQY4VGBFCO5U6IIYP2WLHJQ, seg_pos={0 0}, upload=56631]: Done uploading piece. err: <nil> Jul 26 22:38:24.904762702 (59h2m56.095061281s) [noise=true , address=64.49.13.170:4721 , piece_id=IZWGA6PE57DVSRNG6PSUMWWH5S2STZMREIQGOKCD6HI6DFZWKNXQ, seg_pos={0 0}, upload=56631]: Done uploading piece. err: ecclient: upload failed (node:1MYa3uS5XnUo4DEvPvXbjcKdCqddUUKbi7L6HV9q6VgyzehyD3, address:64.49.13.170:4721): protocol: storage node overloaded, request limit: 2000 Jul 26 22:38:24.908276832 (59h2m56.098575415s) [noise=true , address=79.127.219.38:4013 , piece_id=HP5XJHQM6W5KOYDMVH2JQN7XWCRRG5ZLKGRE22HS65MFFXVNZU4Q, seg_pos={0 0}, upload=56631]: Done uploading piece. err: <nil> Jul 26 22:38:24.949670169 (59h2m56.139968738s) [noise=true , address=77.90.31.21:4880 , piece_id=R2O7OT3PYZSZV3OZVXFUBOVASKM3CJTFLHMSGBYU42O4VM5IKEDQ, seg_pos={0 0}, upload=56631]: Done uploading piece. err: <nil> Jul 26 22:38:25.347251577 (59h2m56.537550146s) [noise=true , address=79.127.205.234:4613 , piece_id=PB57BJOZXUAWC75ONCQFPIGV3F43VD5QGR5O53IUEY3SJLOYPYPQ, seg_pos={0 0}, upload=56631]: Done uploading piece. err: <nil> Jul 26 22:38:25.439562747 (59h2m56.629861316s) [noise=true , address=79.127.205.235:4689 , piece_id=Q7DIHD25XP42FK6CYCO3BEGD632YPLUMYPBJKEZCBHS4YZWVJVFQ, seg_pos={0 0}, upload=56631]: Done uploading piece. err: <nil> Jul 26 22:38:25.725950048 (59h2m56.916248627s) [noise=true , address=79.127.205.246:4270 , piece_id=AIM5XOGJTJ7QQ3PPDCQ25ERTFBGRJI6UXNJAG6LDOGV5CKUGJI3A, seg_pos={0 0}, upload=56631]: Done uploading piece. err: <nil> Jul 26 22:38:27.210185703 (59h2m58.400484272s) [noise=true , address=79.127.219.48:4090 , piece_id=TUWO3DINZR5GBD6V6UMNETD5SQORVNADN6Y7CVG7GD2VXYUPUQ5Q, seg_pos={0 0}, upload=56631]: Done uploading piece. err: <nil> Jul 26 22:38:27.709038123 (59h2m58.899336692s) [noise=true , address=156.146.43.225:4448 , piece_id=XIFOWF3YDDQNP5JQS7NPYW5ARQOURRLHCEK6PU5J3AEUL43T4BYA, seg_pos={0 0}, upload=56631]: Done uploading piece. err: <nil> Jul 26 22:38:28.776879313 (59h2m59.967177906s) [noise=true , address=79.127.220.101:4048 , piece_id=EAATBR22YZLUAGBQBT2TLZIMJJWKG3VQRIZZHRGCH533MDUHIK6A, seg_pos={0 0}, upload=56631]: Done uploading piece. err: <nil> Jul 26 22:38:29.935859494 (59h3m1.126158078s) [noise=true , address=64.49.13.140:4516 , piece_id=DPN4UUJLBJ3EEQFACSZQKJCJGGSLH6WB673M4DAREN2EF3FQNWMQ, seg_pos={0 0}, upload=56631]: Done uploading piece. err: <nil> Jul 26 22:38:32.767735980 (59h3m3.95803455s) [noise=true , address=45.91.116.12:4229 , piece_id=KPDDGJ5SGXZGJFWTVUE7M4MISTLEL5DIGHNPEWSMLSTUUQFEZS5Q, seg_pos={0 0}, upload=56631]: Done uploading piece. err: ecclient: upload failed (node:12rfuAG41swAn82EM5ZTAN31NhGp4UKMHFiZiT57NXmSAi3MHe2, address:45.91.116.12:4229): protocol: storage node overloaded, request limit: 2000 Jul 26 22:38:54.325490826 (59h3m25.515789399s) [noise=true , address=104.36.82.45:4925 , piece_id=OFKLTRXDEPOGW7CUKDT7U6DUCJBP7H2T3KYHQUBSD5E35IM4SGRQ, seg_pos={0 0}, upload=56631]: Done uploading piece. err: <nil> Jul 26 22:39:23.643352570 (59h3m54.833651156s) [noise=true , address=104.36.82.10:4101 , piece_id=BV3EQW57Y4G274PPJM2WS67TTIVXT2DWCOV4U6XVB2RPACQNJQUA, seg_pos={0 0}, upload=56631]: Done uploading piece. err: ecclient: upload failed (node:1RmTDQrWoQBSpHrcUsb2PYijpcFdCyRya7VrMkxhWeQXdGeQpp, address:104.36.82.10:4101): protocol: write tcp 139.59.144.22:45976->104.36.82.10:4101: use of closed network connection; write tcp 139.59.144.22:45976->104.36.82.10:4101: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:45976->104.36.82.10:4101: use of closed network connection Jul 26 22:39:23.643418875 (59h3m54.833717453s) [noise=true , address=45.91.116.10:4100 , piece_id=BB577NEP76YKWDZI3X7MZXJEVB4BGSZZLQU5AKOWVGFO5F5ZOXFQ, seg_pos={0 0}, upload=56631]: Done uploading piece. err: ecclient: upload failed (node:12PQzzS3ZUKXCpiPHqTokRAC9MUuU8QQp7fJTDydmYehhs21YBn, address:45.91.116.10:4100): protocol: write tcp 139.59.144.22:33428->45.91.116.10:4100: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 22:39:23.643454601 (59h3m54.833753171s) [seg_pos={0 0}, upload=56631]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 26 22:39:23.749096004 (59h3m54.939394577s) [upload=56631]: Done uploading segments... Jul 26 22:39:23.749123123 (59h3m54.939421691s) [upload=56631]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 22.773007891s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........X.X.........................X....................] duration: 8.853367599s FAIL (2025-07-26 22:39:44 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 8.85340058s core uplink us1 vivint
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 9 out of 60 is failed (threshold=3) [................................X.......X..X...X.XX...X..X.X] duration: 1m0.001635893s FAIL (2025-07-26 22:39:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001669708s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 20 out of 60 is failed (threshold=3) [....................X.............XX...XXXXX.XXXX.XXXXX..XXX] duration: 1m0.107484842s FAIL (2025-07-26 22:39:23 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 6 22:38:23.917529311 (59h2m55.107827904s) [upload=56631]: Done starting upload Jul 26 22:38:23.917846807 (59h2m55.108145388s) [upload=56631]: Uploading segments... Jul 26 22:38:23.917959648 (59h2m55.108258232s) [upload=56631]: Got next segment. Inline: false Jul 26 22:38:23.917998371 (59h2m55.108296951s) [upload=56631]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:38:23.918003592 (59h2m55.108302159s) [upload=56631]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:38:24.126845246 (59h2m55.317143827s) [seg_pos={0 0}, upload=56631]: Begin upload segment... Jul 26 22:38:24.126946250 (59h2m55.317244817s) [seg_pos={0 0}, upload=56631]: Done begin upload segment. Jul 26 22:38:24.126983024 (59h2m55.317281613s) [noise=true , address=79.127.205.234:4613 , piece_id=PB57BJOZXUAWC75ONCQFPIGV3F43VD5QGR5O53IUEY3SJLOYPYPQ, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.127171415 (59h2m55.317470008s) [noise=true , address=79.127.219.48:4090 , piece_id=TUWO3DINZR5GBD6V6UMNETD5SQORVNADN6Y7CVG7GD2VXYUPUQ5Q, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.127653251 (59h2m55.317951845s) [noise=true , address=45.91.116.10:4100 , piece_id=BB577NEP76YKWDZI3X7MZXJEVB4BGSZZLQU5AKOWVGFO5F5ZOXFQ, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.127862315 (59h2m55.318160904s) [noise=true , address=79.127.219.51:4210 , piece_id=WBHIJ3ETU4NQFVTVQLMCRC3KORGCP246B5TBKAVKHLA3QISKHIAA, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.128042586 (59h2m55.318341175s) [noise=true , address=77.90.31.21:4880 , piece_id=R2O7OT3PYZSZV3OZVXFUBOVASKM3CJTFLHMSGBYU42O4VM5IKEDQ, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.128393003 (59h2m55.318691592s) [noise=true , address=79.127.220.101:4048 , piece_id=EAATBR22YZLUAGBQBT2TLZIMJJWKG3VQRIZZHRGCH533MDUHIK6A, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.128427794 (59h2m55.318726374s) [noise=true , address=77.90.31.17:4792 , piece_id=QBKUGW647ZUJCAT3UQBS76GOYSGQFXTF2SXA3OSTRD5QMXUOAFOA, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.128812762 (59h2m55.319111353s) [noise=true , address=77.90.31.10:4543 , piece_id=PUEXSYUSSZ6HYO7KFSPKQTN56VD356OH65FGWDZZNJNMVYBBQ5CA, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.128938664 (59h2m55.319237252s) [noise=true , address=104.36.82.10:4101 , piece_id=BV3EQW57Y4G274PPJM2WS67TTIVXT2DWCOV4U6XVB2RPACQNJQUA, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.129160636 (59h2m55.319459225s) [noise=true , address=104.36.82.45:4925 , piece_id=OFKLTRXDEPOGW7CUKDT7U6DUCJBP7H2T3KYHQUBSD5E35IM4SGRQ, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.129309084 (59h2m55.319607673s) [noise=true , address=77.90.31.12:4611 , piece_id=W6HPFILAQM2GUYNGFAVVVZMM3J7SPIQY4VGBFCO5U6IIYP2WLHJQ, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.129493745 (59h2m55.319792336s) [noise=true , address=64.49.13.140:4516 , piece_id=DPN4UUJLBJ3EEQFACSZQKJCJGGSLH6WB673M4DAREN2EF3FQNWMQ, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.129803640 (59h2m55.32010223s) [noise=true , address=64.49.13.170:4721 , piece_id=IZWGA6PE57DVSRNG6PSUMWWH5S2STZMREIQGOKCD6HI6DFZWKNXQ, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.129817888 (59h2m55.320116478s) [noise=true , address=156.146.43.225:4448 , piece_id=XIFOWF3YDDQNP5JQS7NPYW5ARQOURRLHCEK6PU5J3AEUL43T4BYA, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.130310600 (59h2m55.320609186s) [noise=true , address=45.91.116.12:4229 , piece_id=KPDDGJ5SGXZGJFWTVUE7M4MISTLEL5DIGHNPEWSMLSTUUQFEZS5Q, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.134915854 (59h2m55.325214436s) [noise=true , address=79.127.205.235:4689 , piece_id=Q7DIHD25XP42FK6CYCO3BEGD632YPLUMYPBJKEZCBHS4YZWVJVFQ, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.135399255 (59h2m55.325697837s) [noise=true , address=79.127.219.38:4013 , piece_id=HP5XJHQM6W5KOYDMVH2JQN7XWCRRG5ZLKGRE22HS65MFFXVNZU4Q, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.135794642 (59h2m55.32609323s) [noise=true , address=79.127.205.246:4270 , piece_id=AIM5XOGJTJ7QQ3PPDCQ25ERTFBGRJI6UXNJAG6LDOGV5CKUGJI3A, seg_pos={0 0}, upload=56631]: Uploading piece... Jul 26 22:38:24.550256489 (59h2m55.740555068s) [noise=true , address=79.127.219.51:4210 , piece_id=WBHIJ3ETU4NQFVTVQLMCRC3KORGCP246B5TBKAVKHLA3QISKHIAA, seg_pos={0 0}, upload=56631]: Done uploading piece. err: ecclient: upload failed (node:1yCKELQfksAs1CLfacoYZgpHGzRM3UTGak3eWPu9ds3qn1a9G2, address:79.127.219.51:4210): protocol: storage node overloaded, request limit: 2000 Jul 26 22:38:24.687416075 (59h2m55.877714644s) [upload=56631]: Next returned nil segment Jul 26 22:38:24.687434636 (59h2m55.877733205s) [upload=56631]: Waiting for error group managing segments... Jul 26 22:38:24.869879720 (59h2m56.060178307s) [noise=true , address=77.90.31.17:4792 , piece_id=QBKUGW647ZUJCAT3UQBS76GOYSGQFXTF2SXA3OSTRD5QMXUOAFOA, seg_pos={0 0}, upload=56631]: Done uploading piece. err: <nil> Jul 26 22:38:24.870055848 (59h2m56.060354452s) [noise=true , address=77.90.31.10:4543 , piece_id=PUEXSYUSSZ6HYO7KFSPKQTN56VD356OH65FGWDZZNJNMVYBBQ5CA, seg_pos={0 0}, upload=56631]: Done uploading piece. err: <nil> Jul 26 22:38:24.878082696 (59h2m56.068381283s) [noise=true , address=77.90.31.12:4611 , piece_id=W6HPFILAQM2GUYNGFAVVVZMM3J7SPIQY4VGBFCO5U6IIYP2WLHJQ, seg_pos={0 0}, upload=56631]: Done uploading piece. err: <nil> Jul 26 22:38:24.904762702 (59h2m56.095061281s) [noise=true , address=64.49.13.170:4721 , piece_id=IZWGA6PE57DVSRNG6PSUMWWH5S2STZMREIQGOKCD6HI6DFZWKNXQ, seg_pos={0 0}, upload=56631]: Done uploading piece. err: ecclient: upload failed (node:1MYa3uS5XnUo4DEvPvXbjcKdCqddUUKbi7L6HV9q6VgyzehyD3, address:64.49.13.170:4721): protocol: storage node overloaded, request limit: 2000 Jul 26 22:38:24.908276832 (59h2m56.098575415s) [noise=true , address=79.127.219.38:4013 , piece_id=HP5XJHQM6W5KOYDMVH2JQN7XWCRRG5ZLKGRE22HS65MFFXVNZU4Q, seg_pos={0 0}, upload=56631]: Done uploading piece. err: <nil> Jul 26 22:38:24.949670169 (59h2m56.139968738s) [noise=true , address=77.90.31.21:4880 , piece_id=R2O7OT3PYZSZV3OZVXFUBOVASKM3CJTFLHMSGBYU42O4VM5IKEDQ, seg_pos={0 0}, upload=56631]: Done uploading piece. err: <nil> Jul 26 22:38:25.347251577 (59h2m56.537550146s) [noise=true , address=79.127.205.234:4613 , piece_id=PB57BJOZXUAWC75ONCQFPIGV3F43VD5QGR5O53IUEY3SJLOYPYPQ, seg_pos={0 0}, upload=56631]: Done uploading piece. err: <nil> Jul 26 22:38:25.439562747 (59h2m56.629861316s) [noise=true , address=79.127.205.235:4689 , piece_id=Q7DIHD25XP42FK6CYCO3BEGD632YPLUMYPBJKEZCBHS4YZWVJVFQ, seg_pos={0 0}, upload=56631]: Done uploading piece. err: <nil> Jul 26 22:38:25.725950048 (59h2m56.916248627s) [noise=true , address=79.127.205.246:4270 , piece_id=AIM5XOGJTJ7QQ3PPDCQ25ERTFBGRJI6UXNJAG6LDOGV5CKUGJI3A, seg_pos={0 0}, upload=56631]: Done uploading piece. err: <nil> Jul 26 22:38:27.210185703 (59h2m58.400484272s) [noise=true , address=79.127.219.48:4090 , piece_id=TUWO3DINZR5GBD6V6UMNETD5SQORVNADN6Y7CVG7GD2VXYUPUQ5Q, seg_pos={0 0}, upload=56631]: Done uploading piece. err: <nil> Jul 26 22:38:27.709038123 (59h2m58.899336692s) [noise=true , address=156.146.43.225:4448 , piece_id=XIFOWF3YDDQNP5JQS7NPYW5ARQOURRLHCEK6PU5J3AEUL43T4BYA, seg_pos={0 0}, upload=56631]: Done uploading piece. err: <nil> Jul 26 22:38:28.776879313 (59h2m59.967177906s) [noise=true , address=79.127.220.101:4048 , piece_id=EAATBR22YZLUAGBQBT2TLZIMJJWKG3VQRIZZHRGCH533MDUHIK6A, seg_pos={0 0}, upload=56631]: Done uploading piece. err: <nil> Jul 26 22:38:29.935859494 (59h3m1.126158078s) [noise=true , address=64.49.13.140:4516 , piece_id=DPN4UUJLBJ3EEQFACSZQKJCJGGSLH6WB673M4DAREN2EF3FQNWMQ, seg_pos={0 0}, upload=56631]: Done uploading piece. err: <nil> Jul 26 22:38:32.767735980 (59h3m3.95803455s) [noise=true , address=45.91.116.12:4229 , piece_id=KPDDGJ5SGXZGJFWTVUE7M4MISTLEL5DIGHNPEWSMLSTUUQFEZS5Q, seg_pos={0 0}, upload=56631]: Done uploading piece. err: ecclient: upload failed (node:12rfuAG41swAn82EM5ZTAN31NhGp4UKMHFiZiT57NXmSAi3MHe2, address:45.91.116.12:4229): protocol: storage node overloaded, request limit: 2000 Jul 26 22:38:54.325490826 (59h3m25.515789399s) [noise=true , address=104.36.82.45:4925 , piece_id=OFKLTRXDEPOGW7CUKDT7U6DUCJBP7H2T3KYHQUBSD5E35IM4SGRQ, seg_pos={0 0}, upload=56631]: Done uploading piece. err: <nil> Jul 26 22:39:23.643352570 (59h3m54.833651156s) [noise=true , address=104.36.82.10:4101 , piece_id=BV3EQW57Y4G274PPJM2WS67TTIVXT2DWCOV4U6XVB2RPACQNJQUA, seg_pos={0 0}, upload=56631]: Done uploading piece. err: ecclient: upload failed (node:1RmTDQrWoQBSpHrcUsb2PYijpcFdCyRya7VrMkxhWeQXdGeQpp, address:104.36.82.10:4101): protocol: write tcp 139.59.144.22:45976->104.36.82.10:4101: use of closed network connection; write tcp 139.59.144.22:45976->104.36.82.10:4101: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:45976->104.36.82.10:4101: use of closed network connection Jul 26 22:39:23.643418875 (59h3m54.833717453s) [noise=true , address=45.91.116.10:4100 , piece_id=BB577NEP76YKWDZI3X7MZXJEVB4BGSZZLQU5AKOWVGFO5F5ZOXFQ, seg_pos={0 0}, upload=56631]: Done uploading piece. err: ecclient: upload failed (node:12PQzzS3ZUKXCpiPHqTokRAC9MUuU8QQp7fJTDydmYehhs21YBn, address:45.91.116.10:4100): protocol: write tcp 139.59.144.22:33428->45.91.116.10:4100: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 22:39:23.643454601 (59h3m54.833753171s) [seg_pos={0 0}, upload=56631]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 26 22:39:23.749096004 (59h3m54.939394577s) [upload=56631]: Done uploading segments... Jul 26 22:39:23.749123123 (59h3m54.939421691s) [upload=56631]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1m0.107501402s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 866.292154ms FAIL (2025-07-26 22:39:17 UTC) orbiter warning: digest mismatch: after b0275bfe409329e90caa68828ca9cf99d935619c40adf20794b6b4a165f10dac != before 689768e222479cddedfe8605c20bdb7052e22d65868c907a97a7972f7764c1dd Duration: 866.321582ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [............X.X.........................X...................] duration: 2.255273596s FAIL (2025-07-26 22:39:10 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.255307675s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [X..X..XXX....X.X.X...............X..........................] duration: 2.253997671s FAIL (2025-07-26 22:38:45 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 988 (58h42m45.061082575s) [upload=56308]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:18:14.018433272 (58h42m45.208731856s) [upload=56308]: Done uploading segments... Jul 26 22:18:14.018484438 (58h42m45.208783019s) [upload=56308]: Upload finished. err: <nil> Jul 26 22:18:57.520834389 (58h43m28.711132968s) [upload=56321]: Starting upload Jul 26 22:18:57.520934224 (58h43m28.711232794s) [upload=56321]: Done starting upload Jul 26 22:18:57.521309042 (58h43m28.711607621s) [upload=56321]: Uploading segments... Jul 26 22:18:57.521389036 (58h43m28.711687618s) [upload=56321]: Got next segment. Inline: false Jul 26 22:18:57.521430569 (58h43m28.711729148s) [upload=56321]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:18:57.521435990 (58h43m28.711734568s) [upload=56321]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:18:57.634354903 (58h43m28.824653493s) [seg_pos={0 0}, upload=56321]: Begin upload segment... Jul 26 22:18:57.634433002 (58h43m28.824731593s) [seg_pos={0 0}, upload=56321]: Done begin upload segment. Jul 26 22:18:57.634466006 (58h43m28.824764596s) [noise=true , address=104.36.82.12:4150 , piece_id=4PVRPBKTUNFKUWDJDTXYCIXDKONWG5OPQVB2M6KPKK66SMCYYHGA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.634507350 (58h43m28.82480593s) [noise=true , address=45.91.116.6:4973 , piece_id=S5BZKUNC2KNOVQMFRHSXOZNZZG6BUBK57ECM6W3S4YKN55KPOSFQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635011032 (58h43m28.825309612s) [noise=true , address=45.91.116.4:4938 , piece_id=INUA3CJT7KOAKKRYSBLTNXLX3MKWYYAFZ45BMUOZ6Y6I3B2EHHCQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635154508 (58h43m28.825453097s) [noise=true , address=45.91.116.8:4047 , piece_id=YTLRWKOOJ4XIGA3SKHFT2K66R57LBTW7OOVZSVJMCCV3YISPGWLA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635425419 (58h43m28.825724018s) [noise=true , address=45.91.116.8:4048 , piece_id=BF2LQK3BFGLPNCTDWB2M6HLRBSYBCXLR2AO7CMPERHT4DIBNKC4Q, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635823171 (58h43m28.826121762s) [noise=true , address=77.90.31.12:4590 , piece_id=PZSNEJOE623OVLNPZSRVVTTAGNHGRD7JCBUXCB3FXOMUE4CQNLFA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635848858 (58h43m28.826147438s) [noise=true , address=45.91.116.12:4227 , piece_id=FICCSE6IJQKUGYKTS3JHH4G65CPBOGDSPVIYXT25NR2GKRZQLOXA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.636411603 (58h43m28.826710186s) [noise=true , address=104.36.82.7:4909 , piece_id=HOFM67WVGW5QXUA65MEPQLJF7EV25AXBG4GNMREFYF2RNRGH2XWQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.636540327 (58h43m28.826838918s) [noise=true , address=77.90.31.11:4553 , piece_id=BP6FPURAP7D6RT34G72KZ32UY76QKXDG464WKNDPNUNOD6WS3KFQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.636837624 (58h43m28.827136209s) [noise=true , address=45.91.116.10:4081 , piece_id=2R44TDAMR2O3CNYFRZXOCSMG2ISUWMIQPVDWN3V2ECZWW4JUHNMA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.637107959 (58h43m28.827406542s) [noise=true , address=45.91.116.13:4154 , piece_id=PNBIV7A27TASCKR2CEQT7NJ5ZTG3NUQ2PLXJH7R47BFQQ53FLKUQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.637209450 (58h43m28.827508033s) [noise=true , address=77.90.31.21:4877 , piece_id=RCTFN3VAXG26GTA6GFQDQDIBAHJPOSI3QUSLLKO5APYYA57BSBRQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.637657116 (58h43m28.827955704s) [noise=true , address=45.91.116.7:4010 , piece_id=DZSLTEUQELXNHSH57IUZ6PRMOS56GR5TCWNAEEY7Y6YJ7KNDTRJA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.638167018 (58h43m28.828465602s) [noise=true , address=104.36.82.3:4767 , piece_id=IWRMBBOMGH7ZV3JA326TJ4SWHLEGA2T6CDLUDKJ4RFZBUZMTA3MQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.638722599 (58h43m28.829021185s) [noise=true , address=45.91.116.7:4009 , piece_id=2JHIJGUSNOSVSGOSGHHIYA2RNVLRS2P7MIKYHLSX4PTQOWNZKKGQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.639240448 (58h43m28.82953902s) [noise=true , address=104.36.82.11:4114 , piece_id=ZUBONJLMV4QRTG7DLIHWZICNNQ5OOF3HAPN26Q5AMK4TY4OYTXNA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:58.247247885 (58h43m29.437546466s) [upload=56321]: Next returned nil segment Jul 26 22:18:58.247273951 (58h43m29.437572537s) [upload=56321]: Waiting for error group managing segments... Jul 26 22:18:58.383663350 (58h43m29.573961931s) [noise=true , address=45.91.116.13:4154 , piece_id=PNBIV7A27TASCKR2CEQT7NJ5ZTG3NUQ2PLXJH7R47BFQQ53FLKUQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.384857324 (58h43m29.575155903s) [noise=true , address=45.91.116.8:4047 , piece_id=YTLRWKOOJ4XIGA3SKHFT2K66R57LBTW7OOVZSVJMCCV3YISPGWLA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.384981909 (58h43m29.575280495s) [noise=true , address=45.91.116.6:4973 , piece_id=S5BZKUNC2KNOVQMFRHSXOZNZZG6BUBK57ECM6W3S4YKN55KPOSFQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.387318149 (58h43m29.577616722s) [noise=true , address=45.91.116.7:4009 , piece_id=2JHIJGUSNOSVSGOSGHHIYA2RNVLRS2P7MIKYHLSX4PTQOWNZKKGQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.395512660 (58h43m29.585811243s) [noise=true , address=45.91.116.12:4227 , piece_id=FICCSE6IJQKUGYKTS3JHH4G65CPBOGDSPVIYXT25NR2GKRZQLOXA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.401895917 (58h43m29.592194499s) [noise=true , address=45.91.116.10:4081 , piece_id=2R44TDAMR2O3CNYFRZXOCSMG2ISUWMIQPVDWN3V2ECZWW4JUHNMA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.402135621 (58h43m29.59243419s) [noise=true , address=77.90.31.11:4553 , piece_id=BP6FPURAP7D6RT34G72KZ32UY76QKXDG464WKNDPNUNOD6WS3KFQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.403028120 (58h43m29.593326693s) [noise=true , address=45.91.116.8:4048 , piece_id=BF2LQK3BFGLPNCTDWB2M6HLRBSYBCXLR2AO7CMPERHT4DIBNKC4Q, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.404431307 (58h43m29.594729891s) [noise=true , address=45.91.116.4:4938 , piece_id=INUA3CJT7KOAKKRYSBLTNXLX3MKWYYAFZ45BMUOZ6Y6I3B2EHHCQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.409536042 (58h43m29.599834622s) [noise=true , address=45.91.116.7:4010 , piece_id=DZSLTEUQELXNHSH57IUZ6PRMOS56GR5TCWNAEEY7Y6YJ7KNDTRJA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:59.425708084 (58h43m30.616006681s) [noise=true , address=77.90.31.21:4877 , piece_id=RCTFN3VAXG26GTA6GFQDQDIBAHJPOSI3QUSLLKO5APYYA57BSBRQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:19:00.437139003 (58h43m31.627437582s) [noise=true , address=77.90.31.12:4590 , piece_id=PZSNEJOE623OVLNPZSRVVTTAGNHGRD7JCBUXCB3FXOMUE4CQNLFA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:19:27.246703337 (58h43m58.437001916s) [noise=true , address=104.36.82.7:4909 , piece_id=HOFM67WVGW5QXUA65MEPQLJF7EV25AXBG4GNMREFYF2RNRGH2XWQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: write tcp 139.59.144.22:57728->104.36.82.7:4909: use of closed network connection; write tcp 139.59.144.22:57728->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57728->104.36.82.7:4909: use of closed network connection Jul 26 22:19:27.246791447 (58h43m58.437090027s) [noise=true , address=104.36.82.11:4114 , piece_id=ZUBONJLMV4QRTG7DLIHWZICNNQ5OOF3HAPN26Q5AMK4TY4OYTXNA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 139.59.144.22:44152->104.36.82.11:4114: use of closed network connection; write tcp 139.59.144.22:44152->104.36.82.11:4114: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44152->104.36.82.11:4114: use of closed network connection Jul 26 22:19:27.246848250 (58h43m58.437146829s) [noise=true , address=104.36.82.3:4767 , piece_id=IWRMBBOMGH7ZV3JA326TJ4SWHLEGA2T6CDLUDKJ4RFZBUZMTA3MQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:52976->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:52976->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52976->104.36.82.3:4767: use of closed network connection Jul 26 22:19:27.246885606 (58h43m58.4371843s) [noise=true , address=104.36.82.12:4150 , piece_id=4PVRPBKTUNFKUWDJDTXYCIXDKONWG5OPQVB2M6KPKK66SMCYYHGA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:104.36.82.12:4150): protocol: write tcp 139.59.144.22:43452->104.36.82.12:4150: use of closed network connection; write tcp 139.59.144.22:43452->104.36.82.12:4150: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43452->104.36.82.12:4150: use of closed network connection Jul 26 22:19:27.246926689 (58h43m58.437225269s) [seg_pos={0 0}, upload=56321]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 22:19:27.354450344 (58h43m58.544748918s) [upload=56321]: Done uploading segments... Jul 26 22:19:27.354479247 (58h43m58.544777827s) [upload=56321]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 2.254024593s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 825.779157ms FAIL (2025-07-26 22:38:36 UTC) orbiter warning: digest mismatch: after b0275bfe409329e90caa68828ca9cf99d935619c40adf20794b6b4a165f10dac != before 689768e222479cddedfe8605c20bdb7052e22d65868c907a97a7972f7764c1dd Duration: 825.811016ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 19 out of 60 is failed (threshold=3) [.....................X.............XX...XXXXX.XXXX.XXXXX..XX] duration: 1m0.111391094s FAIL (2025-07-26 22:38:23 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== h1m54.855615318s) [upload=56617]: Got next segment. Inline: false Jul 26 22:37:23.665355019 (59h1m54.855653598s) [upload=56617]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:37:23.665360447 (59h1m54.855659025s) [upload=56617]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:37:23.895245808 (59h1m55.08554439s) [seg_pos={0 0}, upload=56617]: Begin upload segment... Jul 26 22:37:23.895307401 (59h1m55.085605973s) [seg_pos={0 0}, upload=56617]: Done begin upload segment. Jul 26 22:37:23.895350191 (59h1m55.085648779s) [noise=true , address=77.90.31.16:4743 , piece_id=33ROCTBNII7DVT6T3QJOERAL7PG2VHCTK7TM5SDZ7IUYME54SLOA, seg_pos={0 0}, upload=56617]: Uploading piece... Jul 26 22:37:23.895365577 (59h1m55.085664163s) [noise=true , address=77.90.31.19:4866 , piece_id=LACLNWIEZ24RHRH22KMOITFBGLOJ4PKRXJSSN4PEY5F7E2C426EA, seg_pos={0 0}, upload=56617]: Uploading piece... Jul 26 22:37:23.895768371 (59h1m55.086066956s) [noise=true , address=77.90.31.17:4780 , piece_id=42IQQNDMDR7VAR7EBWBQ6ZQ3LKNVTDAG4QMCGYDSRSKQDYJSG7QQ, seg_pos={0 0}, upload=56617]: Uploading piece... Jul 26 22:37:23.895814340 (59h1m55.086112929s) [noise=true , address=79.127.201.215:4360 , piece_id=GOYQFSFVLDRYCKSBALDKRXTCST3I5L3VZCDRHECPZZ5FXGO7D4NA, seg_pos={0 0}, upload=56617]: Uploading piece... Jul 26 22:37:23.896166084 (59h1m55.086464668s) [noise=true , address=79.127.219.47:4069 , piece_id=57DHMEKAKLNB5BZF6RT2JVHR4GFCV77XRJ2W2JDQQ4IKHZG4HWOQ, seg_pos={0 0}, upload=56617]: Uploading piece... Jul 26 22:37:23.896213342 (59h1m55.086511915s) [noise=true , address=79.127.219.51:4210 , piece_id=24BLXSYZO6JDXZH3LTQV7YJNBEGWD2HQYGKR5B2Z4R2SJMCV37RQ, seg_pos={0 0}, upload=56617]: Uploading piece... Jul 26 22:37:23.896719857 (59h1m55.087018446s) [noise=true , address=77.90.31.18:4832 , piece_id=3FIOP5YXX2KZURYZPTQB77ZVFCCF2CMDP4QEOPBGWRDYP34M4DDQ, seg_pos={0 0}, upload=56617]: Uploading piece... Jul 26 22:37:23.897441940 (59h1m55.087740509s) [noise=true , address=77.90.31.21:4891 , piece_id=ELBDSI3MEF4MB6KSLWKP4TVC6EUR4QTL7C7BFZG3PN6QX5S2H2CA, seg_pos={0 0}, upload=56617]: Uploading piece... Jul 26 22:37:23.897930018 (59h1m55.088228603s) [noise=true , address=77.90.31.10:4530 , piece_id=NBSDQWZ2WMW6XZYFLGW4G4YPUKCKQDL3DX6AIDJ75O5URTDSB7CA, seg_pos={0 0}, upload=56617]: Uploading piece... Jul 26 22:37:23.898322278 (59h1m55.088620861s) [noise=true , address=156.146.43.225:4434 , piece_id=YWXCQN7FHD5GOM44XL33ONAZ2I7YXQTM3UHEBR5ZH4VDSQICHIOQ, seg_pos={0 0}, upload=56617]: Uploading piece... Jul 26 22:37:23.898762492 (59h1m55.089061074s) [noise=true , address=79.127.205.246:4266 , piece_id=RTRAAGJCRYLNTD4GA65G7O7LMSNWUPRETENGOZK3AMEKGCVIUQNQ, seg_pos={0 0}, upload=56617]: Uploading piece... Jul 26 22:37:23.899099338 (59h1m55.08939792s) [noise=true , address=79.127.220.101:4048 , piece_id=SBFQFZOFQCEZOXHM6Q3Z6MZEL2UDAEW2ANBGB2Z2AUJWQE6VREAQ, seg_pos={0 0}, upload=56617]: Uploading piece... Jul 26 22:37:23.900843715 (59h1m55.091142298s) [noise=true , address=45.91.116.9:4268 , piece_id=ET3OJ7G5JPBJ5HMVCTNOUJOLBGTR36676Q4CSG4MCD3AD7VLBOQA, seg_pos={0 0}, upload=56617]: Uploading piece... Jul 26 22:37:23.901958111 (59h1m55.092256698s) [noise=true , address=104.36.82.36:4632 , piece_id=G3YWDKCVPLGTI2WT2OD72PBK6Z7SN52WFW7B2XZHCL6AV2IJOCIA, seg_pos={0 0}, upload=56617]: Uploading piece... Jul 26 22:37:23.902281481 (59h1m55.092580066s) [noise=true , address=79.127.205.245:4225 , piece_id=6H5CO53X4BERFTSSFBGHKVOLW7DT5JTQCIVPTJFTZSD32F5IJ44A, seg_pos={0 0}, upload=56617]: Uploading piece... Jul 26 22:37:23.902722074 (59h1m55.093020664s) [noise=true , address=104.36.82.59:4315 , piece_id=3FBI6GSI5BGPR5GD3R7BMFQYY2FMLVXDK4QKSGRNWOD6PSJINQWQ, seg_pos={0 0}, upload=56617]: Uploading piece... Jul 26 22:37:23.903306195 (59h1m55.093604779s) [noise=true , address=79.127.219.38:4005 , piece_id=ENALAIBSWNA427Q3RZFASWIMVWAJIHSJ7ZZTYANJN3FR2CKSHT4Q, seg_pos={0 0}, upload=56617]: Uploading piece... Jul 26 22:37:23.903864664 (59h1m55.094163253s) [noise=true , address=79.127.205.235:4677 , piece_id=MP6W4QGRB22VGVTGXQTQTCVJOXWMHEORMHUMZUAGXXJGJWMURRFQ, seg_pos={0 0}, upload=56617]: Uploading piece... Jul 26 22:37:24.307664893 (59h1m55.497963474s) [noise=true , address=79.127.219.47:4069 , piece_id=57DHMEKAKLNB5BZF6RT2JVHR4GFCV77XRJ2W2JDQQ4IKHZG4HWOQ, seg_pos={0 0}, upload=56617]: Done uploading piece. err: ecclient: upload failed (node:18dK6RxbMxU4a46pW4YPiQW87xgpeoTb1ouqE9fzTdzTAFAa3W, address:79.127.219.47:4069): protocol: storage node overloaded, request limit: 2000 Jul 26 22:37:24.434174932 (59h1m55.62447352s) [noise=true , address=156.146.43.225:4434 , piece_id=YWXCQN7FHD5GOM44XL33ONAZ2I7YXQTM3UHEBR5ZH4VDSQICHIOQ, seg_pos={0 0}, upload=56617]: Done uploading piece. err: ecclient: upload failed (node:12WJ1b6sKJG4NcqmNnvSLVw1e8Us6phnN5UpJY2tvo38pdmhFLp, address:156.146.43.225:4434): protocol: storage node overloaded, request limit: 2000 Jul 26 22:37:24.476986738 (59h1m55.667285323s) [upload=56617]: Next returned nil segment Jul 26 22:37:24.477011605 (59h1m55.667310197s) [upload=56617]: Waiting for error group managing segments... Jul 26 22:37:24.644567335 (59h1m55.834865907s) [noise=true , address=77.90.31.21:4891 , piece_id=ELBDSI3MEF4MB6KSLWKP4TVC6EUR4QTL7C7BFZG3PN6QX5S2H2CA, seg_pos={0 0}, upload=56617]: Done uploading piece. err: <nil> Jul 26 22:37:24.664774529 (59h1m55.855073112s) [noise=true , address=104.36.82.59:4315 , piece_id=3FBI6GSI5BGPR5GD3R7BMFQYY2FMLVXDK4QKSGRNWOD6PSJINQWQ, seg_pos={0 0}, upload=56617]: Done uploading piece. err: <nil> Jul 26 22:37:24.696712084 (59h1m55.887010654s) [noise=true , address=77.90.31.10:4530 , piece_id=NBSDQWZ2WMW6XZYFLGW4G4YPUKCKQDL3DX6AIDJ75O5URTDSB7CA, seg_pos={0 0}, upload=56617]: Done uploading piece. err: <nil> Jul 26 22:37:24.701192983 (59h1m55.891491563s) [noise=true , address=77.90.31.19:4866 , piece_id=LACLNWIEZ24RHRH22KMOITFBGLOJ4PKRXJSSN4PEY5F7E2C426EA, seg_pos={0 0}, upload=56617]: Done uploading piece. err: ecclient: upload failed (node:12uCbmPHVXpukjQahs2G6oh7VY2zDMooMAbdgSyHN7Biuesmz3A, address:77.90.31.19:4866): protocol: storage node overloaded, request limit: 2000 Jul 26 22:37:24.995195640 (59h1m56.185494209s) [noise=true , address=77.90.31.17:4780 , piece_id=42IQQNDMDR7VAR7EBWBQ6ZQ3LKNVTDAG4QMCGYDSRSKQDYJSG7QQ, seg_pos={0 0}, upload=56617]: Done uploading piece. err: <nil> Jul 26 22:37:25.155746863 (59h1m56.346045431s) [noise=true , address=79.127.219.38:4005 , piece_id=ENALAIBSWNA427Q3RZFASWIMVWAJIHSJ7ZZTYANJN3FR2CKSHT4Q, seg_pos={0 0}, upload=56617]: Done uploading piece. err: <nil> Jul 26 22:37:25.241750748 (59h1m56.432049337s) [noise=true , address=79.127.205.245:4225 , piece_id=6H5CO53X4BERFTSSFBGHKVOLW7DT5JTQCIVPTJFTZSD32F5IJ44A, seg_pos={0 0}, upload=56617]: Done uploading piece. err: <nil> Jul 26 22:37:25.252820718 (59h1m56.443119291s) [noise=true , address=79.127.205.235:4677 , piece_id=MP6W4QGRB22VGVTGXQTQTCVJOXWMHEORMHUMZUAGXXJGJWMURRFQ, seg_pos={0 0}, upload=56617]: Done uploading piece. err: <nil> Jul 26 22:37:25.325012628 (59h1m56.515311214s) [noise=true , address=79.127.205.246:4266 , piece_id=RTRAAGJCRYLNTD4GA65G7O7LMSNWUPRETENGOZK3AMEKGCVIUQNQ, seg_pos={0 0}, upload=56617]: Done uploading piece. err: <nil> Jul 26 22:37:27.202093280 (59h1m58.392391864s) [noise=true , address=79.127.219.51:4210 , piece_id=24BLXSYZO6JDXZH3LTQV7YJNBEGWD2HQYGKR5B2Z4R2SJMCV37RQ, seg_pos={0 0}, upload=56617]: Done uploading piece. err: <nil> Jul 26 22:37:27.368446946 (59h1m58.558745518s) [noise=true , address=79.127.201.215:4360 , piece_id=GOYQFSFVLDRYCKSBALDKRXTCST3I5L3VZCDRHECPZZ5FXGO7D4NA, seg_pos={0 0}, upload=56617]: Done uploading piece. err: <nil> Jul 26 22:37:29.598459964 (59h2m0.788758535s) [noise=true , address=79.127.220.101:4048 , piece_id=SBFQFZOFQCEZOXHM6Q3Z6MZEL2UDAEW2ANBGB2Z2AUJWQE6VREAQ, seg_pos={0 0}, upload=56617]: Done uploading piece. err: <nil> Jul 26 22:37:34.816794838 (59h2m6.007093404s) [noise=true , address=45.91.116.9:4268 , piece_id=ET3OJ7G5JPBJ5HMVCTNOUJOLBGTR36676Q4CSG4MCD3AD7VLBOQA, seg_pos={0 0}, upload=56617]: Done uploading piece. err: <nil> Jul 26 22:38:23.395201199 (59h2m54.585499779s) [noise=true , address=77.90.31.16:4743 , piece_id=33ROCTBNII7DVT6T3QJOERAL7PG2VHCTK7TM5SDZ7IUYME54SLOA, seg_pos={0 0}, upload=56617]: Done uploading piece. err: ecclient: upload failed (node:12XLPBKV4R1CLgVK32nSQffKHxjhU7XsGo8MEsHaAbDGiusTTAv, address:77.90.31.16:4743): protocol: rpc: dial tcp 77.90.31.16:4743: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 22:38:23.395267705 (59h2m54.585566284s) [noise=true , address=77.90.31.18:4832 , piece_id=3FIOP5YXX2KZURYZPTQB77ZVFCCF2CMDP4QEOPBGWRDYP34M4DDQ, seg_pos={0 0}, upload=56617]: Done uploading piece. err: ecclient: upload failed (node:12QnZ3q9oe4zPuLpS3V1DCBhyaU1umS5B7bpuAqPkaaY96ErWbo, address:77.90.31.18:4832): protocol: rpc: dial tcp 77.90.31.18:4832: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Jul 26 22:38:23.395327930 (59h2m54.58562651s) [noise=true , address=104.36.82.36:4632 , piece_id=G3YWDKCVPLGTI2WT2OD72PBK6Z7SN52WFW7B2XZHCL6AV2IJOCIA, seg_pos={0 0}, upload=56617]: Done uploading piece. err: ecclient: upload failed (node:1B7uLDPBrYvc1N81dKm8bcYnhYwjA5mRdRfeQpWqKqrJjmQJgC, address:104.36.82.36:4632): protocol: write tcp 139.59.144.22:33550->104.36.82.36:4632: use of closed network connection; write tcp 139.59.144.22:33550->104.36.82.36:4632: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33550->104.36.82.36:4632: use of closed network connection Jul 26 22:38:23.395384077 (59h2m54.585682671s) [seg_pos={0 0}, upload=56617]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Jul 26 22:38:23.504591228 (59h2m54.694889815s) [upload=56617]: Done uploading segments... Jul 26 22:38:23.504630341 (59h2m54.694928924s) [upload=56617]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 1m0.111414349s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............X.X.........................X..................] duration: 2.369814215s FAIL (2025-07-26 22:38:14 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.369867264s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [.X..X..XXX....X.X.X...............X.........................] duration: 3.371433615s FAIL (2025-07-26 22:38:03 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 988 (58h42m45.061082575s) [upload=56308]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:18:14.018433272 (58h42m45.208731856s) [upload=56308]: Done uploading segments... Jul 26 22:18:14.018484438 (58h42m45.208783019s) [upload=56308]: Upload finished. err: <nil> Jul 26 22:18:57.520834389 (58h43m28.711132968s) [upload=56321]: Starting upload Jul 26 22:18:57.520934224 (58h43m28.711232794s) [upload=56321]: Done starting upload Jul 26 22:18:57.521309042 (58h43m28.711607621s) [upload=56321]: Uploading segments... Jul 26 22:18:57.521389036 (58h43m28.711687618s) [upload=56321]: Got next segment. Inline: false Jul 26 22:18:57.521430569 (58h43m28.711729148s) [upload=56321]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:18:57.521435990 (58h43m28.711734568s) [upload=56321]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:18:57.634354903 (58h43m28.824653493s) [seg_pos={0 0}, upload=56321]: Begin upload segment... Jul 26 22:18:57.634433002 (58h43m28.824731593s) [seg_pos={0 0}, upload=56321]: Done begin upload segment. Jul 26 22:18:57.634466006 (58h43m28.824764596s) [noise=true , address=104.36.82.12:4150 , piece_id=4PVRPBKTUNFKUWDJDTXYCIXDKONWG5OPQVB2M6KPKK66SMCYYHGA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.634507350 (58h43m28.82480593s) [noise=true , address=45.91.116.6:4973 , piece_id=S5BZKUNC2KNOVQMFRHSXOZNZZG6BUBK57ECM6W3S4YKN55KPOSFQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635011032 (58h43m28.825309612s) [noise=true , address=45.91.116.4:4938 , piece_id=INUA3CJT7KOAKKRYSBLTNXLX3MKWYYAFZ45BMUOZ6Y6I3B2EHHCQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635154508 (58h43m28.825453097s) [noise=true , address=45.91.116.8:4047 , piece_id=YTLRWKOOJ4XIGA3SKHFT2K66R57LBTW7OOVZSVJMCCV3YISPGWLA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635425419 (58h43m28.825724018s) [noise=true , address=45.91.116.8:4048 , piece_id=BF2LQK3BFGLPNCTDWB2M6HLRBSYBCXLR2AO7CMPERHT4DIBNKC4Q, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635823171 (58h43m28.826121762s) [noise=true , address=77.90.31.12:4590 , piece_id=PZSNEJOE623OVLNPZSRVVTTAGNHGRD7JCBUXCB3FXOMUE4CQNLFA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635848858 (58h43m28.826147438s) [noise=true , address=45.91.116.12:4227 , piece_id=FICCSE6IJQKUGYKTS3JHH4G65CPBOGDSPVIYXT25NR2GKRZQLOXA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.636411603 (58h43m28.826710186s) [noise=true , address=104.36.82.7:4909 , piece_id=HOFM67WVGW5QXUA65MEPQLJF7EV25AXBG4GNMREFYF2RNRGH2XWQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.636540327 (58h43m28.826838918s) [noise=true , address=77.90.31.11:4553 , piece_id=BP6FPURAP7D6RT34G72KZ32UY76QKXDG464WKNDPNUNOD6WS3KFQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.636837624 (58h43m28.827136209s) [noise=true , address=45.91.116.10:4081 , piece_id=2R44TDAMR2O3CNYFRZXOCSMG2ISUWMIQPVDWN3V2ECZWW4JUHNMA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.637107959 (58h43m28.827406542s) [noise=true , address=45.91.116.13:4154 , piece_id=PNBIV7A27TASCKR2CEQT7NJ5ZTG3NUQ2PLXJH7R47BFQQ53FLKUQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.637209450 (58h43m28.827508033s) [noise=true , address=77.90.31.21:4877 , piece_id=RCTFN3VAXG26GTA6GFQDQDIBAHJPOSI3QUSLLKO5APYYA57BSBRQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.637657116 (58h43m28.827955704s) [noise=true , address=45.91.116.7:4010 , piece_id=DZSLTEUQELXNHSH57IUZ6PRMOS56GR5TCWNAEEY7Y6YJ7KNDTRJA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.638167018 (58h43m28.828465602s) [noise=true , address=104.36.82.3:4767 , piece_id=IWRMBBOMGH7ZV3JA326TJ4SWHLEGA2T6CDLUDKJ4RFZBUZMTA3MQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.638722599 (58h43m28.829021185s) [noise=true , address=45.91.116.7:4009 , piece_id=2JHIJGUSNOSVSGOSGHHIYA2RNVLRS2P7MIKYHLSX4PTQOWNZKKGQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.639240448 (58h43m28.82953902s) [noise=true , address=104.36.82.11:4114 , piece_id=ZUBONJLMV4QRTG7DLIHWZICNNQ5OOF3HAPN26Q5AMK4TY4OYTXNA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:58.247247885 (58h43m29.437546466s) [upload=56321]: Next returned nil segment Jul 26 22:18:58.247273951 (58h43m29.437572537s) [upload=56321]: Waiting for error group managing segments... Jul 26 22:18:58.383663350 (58h43m29.573961931s) [noise=true , address=45.91.116.13:4154 , piece_id=PNBIV7A27TASCKR2CEQT7NJ5ZTG3NUQ2PLXJH7R47BFQQ53FLKUQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.384857324 (58h43m29.575155903s) [noise=true , address=45.91.116.8:4047 , piece_id=YTLRWKOOJ4XIGA3SKHFT2K66R57LBTW7OOVZSVJMCCV3YISPGWLA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.384981909 (58h43m29.575280495s) [noise=true , address=45.91.116.6:4973 , piece_id=S5BZKUNC2KNOVQMFRHSXOZNZZG6BUBK57ECM6W3S4YKN55KPOSFQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.387318149 (58h43m29.577616722s) [noise=true , address=45.91.116.7:4009 , piece_id=2JHIJGUSNOSVSGOSGHHIYA2RNVLRS2P7MIKYHLSX4PTQOWNZKKGQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.395512660 (58h43m29.585811243s) [noise=true , address=45.91.116.12:4227 , piece_id=FICCSE6IJQKUGYKTS3JHH4G65CPBOGDSPVIYXT25NR2GKRZQLOXA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.401895917 (58h43m29.592194499s) [noise=true , address=45.91.116.10:4081 , piece_id=2R44TDAMR2O3CNYFRZXOCSMG2ISUWMIQPVDWN3V2ECZWW4JUHNMA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.402135621 (58h43m29.59243419s) [noise=true , address=77.90.31.11:4553 , piece_id=BP6FPURAP7D6RT34G72KZ32UY76QKXDG464WKNDPNUNOD6WS3KFQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.403028120 (58h43m29.593326693s) [noise=true , address=45.91.116.8:4048 , piece_id=BF2LQK3BFGLPNCTDWB2M6HLRBSYBCXLR2AO7CMPERHT4DIBNKC4Q, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.404431307 (58h43m29.594729891s) [noise=true , address=45.91.116.4:4938 , piece_id=INUA3CJT7KOAKKRYSBLTNXLX3MKWYYAFZ45BMUOZ6Y6I3B2EHHCQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.409536042 (58h43m29.599834622s) [noise=true , address=45.91.116.7:4010 , piece_id=DZSLTEUQELXNHSH57IUZ6PRMOS56GR5TCWNAEEY7Y6YJ7KNDTRJA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:59.425708084 (58h43m30.616006681s) [noise=true , address=77.90.31.21:4877 , piece_id=RCTFN3VAXG26GTA6GFQDQDIBAHJPOSI3QUSLLKO5APYYA57BSBRQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:19:00.437139003 (58h43m31.627437582s) [noise=true , address=77.90.31.12:4590 , piece_id=PZSNEJOE623OVLNPZSRVVTTAGNHGRD7JCBUXCB3FXOMUE4CQNLFA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:19:27.246703337 (58h43m58.437001916s) [noise=true , address=104.36.82.7:4909 , piece_id=HOFM67WVGW5QXUA65MEPQLJF7EV25AXBG4GNMREFYF2RNRGH2XWQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: write tcp 139.59.144.22:57728->104.36.82.7:4909: use of closed network connection; write tcp 139.59.144.22:57728->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57728->104.36.82.7:4909: use of closed network connection Jul 26 22:19:27.246791447 (58h43m58.437090027s) [noise=true , address=104.36.82.11:4114 , piece_id=ZUBONJLMV4QRTG7DLIHWZICNNQ5OOF3HAPN26Q5AMK4TY4OYTXNA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 139.59.144.22:44152->104.36.82.11:4114: use of closed network connection; write tcp 139.59.144.22:44152->104.36.82.11:4114: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44152->104.36.82.11:4114: use of closed network connection Jul 26 22:19:27.246848250 (58h43m58.437146829s) [noise=true , address=104.36.82.3:4767 , piece_id=IWRMBBOMGH7ZV3JA326TJ4SWHLEGA2T6CDLUDKJ4RFZBUZMTA3MQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:52976->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:52976->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52976->104.36.82.3:4767: use of closed network connection Jul 26 22:19:27.246885606 (58h43m58.4371843s) [noise=true , address=104.36.82.12:4150 , piece_id=4PVRPBKTUNFKUWDJDTXYCIXDKONWG5OPQVB2M6KPKK66SMCYYHGA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:104.36.82.12:4150): protocol: write tcp 139.59.144.22:43452->104.36.82.12:4150: use of closed network connection; write tcp 139.59.144.22:43452->104.36.82.12:4150: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43452->104.36.82.12:4150: use of closed network connection Jul 26 22:19:27.246926689 (58h43m58.437225269s) [seg_pos={0 0}, upload=56321]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 22:19:27.354450344 (58h43m58.544748918s) [upload=56321]: Done uploading segments... Jul 26 22:19:27.354479247 (58h43m58.544777827s) [upload=56321]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 3.371465018s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............X.X.........................X.................] duration: 2.346420513s FAIL (2025-07-26 22:37:59 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.346452739s core uplink us1 vivint
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 8 out of 60 is failed (threshold=3) [.................................X.......X..X...X.XX...X..X.] duration: 23.485760024s FAIL (2025-07-26 22:37:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 23.485796662s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 842.824233ms FAIL (2025-07-26 22:37:27 UTC) orbiter warning: digest mismatch: after b0275bfe409329e90caa68828ca9cf99d935619c40adf20794b6b4a165f10dac != before 689768e222479cddedfe8605c20bdb7052e22d65868c907a97a7972f7764c1dd Duration: 842.856198ms core uplink us1 vivint
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 8 out of 60 is failed (threshold=3) [..................................X.......X..X...X.XX...X..X] duration: 1m0.000698932s FAIL (2025-07-26 22:37:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000723058s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 18 out of 60 is failed (threshold=3) [......................X.............XX...XXXXX.XXXX.XXXXX..X] duration: 1m0.10958496s FAIL (2025-07-26 22:37:12 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== : Done starting upload Jul 26 22:36:12.234786203 (59h0m43.425084773s) [upload=56599]: Uploading segments... Jul 26 22:36:12.234845988 (59h0m43.425144576s) [upload=56599]: Got next segment. Inline: false Jul 26 22:36:12.234897291 (59h0m43.42519587s) [upload=56599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:36:12.234903331 (59h0m43.425201902s) [upload=56599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:36:12.439478433 (59h0m43.629777023s) [seg_pos={0 0}, upload=56599]: Begin upload segment... Jul 26 22:36:12.439560018 (59h0m43.629858597s) [seg_pos={0 0}, upload=56599]: Done begin upload segment. Jul 26 22:36:12.439614891 (59h0m43.629913471s) [noise=true , address=79.127.205.235:4685 , piece_id=4JQBKSAILRACTGUYF5O7OFZTLYCPCJ4RTCWBP7WMSWJEJNIJXVQA, seg_pos={0 0}, upload=56599]: Uploading piece... Jul 26 22:36:12.439628027 (59h0m43.629926613s) [noise=true , address=77.90.31.19:4875 , piece_id=SGYAJYPLA3CGLHWTU4X3DKGCENITO7P7WENNM5Y6WIUSLMXZRF6Q, seg_pos={0 0}, upload=56599]: Uploading piece... Jul 26 22:36:12.440021320 (59h0m43.630319909s) [noise=true , address=77.90.31.13:4648 , piece_id=C5DBTLFHYM7BHH7DC3TJTSL7FQIIZENDQQCA6TEJHLIWPWU4WWOA, seg_pos={0 0}, upload=56599]: Uploading piece... Jul 26 22:36:12.440093618 (59h0m43.630392189s) [noise=true , address=79.127.205.246:4258 , piece_id=HHQKJ5HRUDLW65S6JIBKZSGL7TBIVCVBAKYZHCEHKLOKMIOZVDGA, seg_pos={0 0}, upload=56599]: Uploading piece... Jul 26 22:36:12.440558711 (59h0m43.630857301s) [noise=true , address=104.36.82.51:4082 , piece_id=CPPQ7UPWE4YSPOK2FMOXPQFLJQ3BZHLIVZ4AZNKZW5ETQLCS3RTQ, seg_pos={0 0}, upload=56599]: Uploading piece... Jul 26 22:36:12.440985386 (59h0m43.631283975s) [noise=true , address=77.90.31.16:4760 , piece_id=HHS2FUWA4MQMFRQK5GSPDWMVV5OVIIY5LD6NVUKVK7CPKSVRXYZQ, seg_pos={0 0}, upload=56599]: Uploading piece... Jul 26 22:36:12.441368830 (59h0m43.631667419s) [noise=true , address=77.90.31.10:4550 , piece_id=BTZ2KE523G5U2CEKZYBQAHA7TEAMAWDI3L4ZRWMCOTJP5NHRBW5Q, seg_pos={0 0}, upload=56599]: Uploading piece... Jul 26 22:36:12.441756386 (59h0m43.632054968s) [noise=true , address=77.90.31.18:4821 , piece_id=SWOWYMFGC72SF2YWPTX4UG4OJH4HYOLLHUIZGXADQKOEF2EMKU4Q, seg_pos={0 0}, upload=56599]: Uploading piece... Jul 26 22:36:12.442104949 (59h0m43.632403556s) [noise=true , address=104.36.82.3:4769 , piece_id=K6IAAGSAT3U5FOGVC6EODKFIHHUYKQ5MI3T4M3RKUZ7G7OVQCVSQ, seg_pos={0 0}, upload=56599]: Uploading piece... Jul 26 22:36:12.443050626 (59h0m43.633349215s) [noise=true , address=77.90.31.17:4779 , piece_id=GHWGKUUDX32IYF5JLO74XTICI64VALC3CORU4MYPVXJXETKP7MIQ, seg_pos={0 0}, upload=56599]: Uploading piece... Jul 26 22:36:12.443871067 (59h0m43.634169656s) [noise=true , address=64.49.13.140:4508 , piece_id=VHIE23HEY37IBJPIOFUIBPD24KMIUNY6KK4J36PCMVWMNHTT5BUQ, seg_pos={0 0}, upload=56599]: Uploading piece... Jul 26 22:36:12.444278996 (59h0m43.634577587s) [noise=true , address=64.49.13.138:4482 , piece_id=5LDVQSRIWHNJGBDH33DSFE6VEAMSDMG35EDVTLBBFZKBYN6LHSWQ, seg_pos={0 0}, upload=56599]: Uploading piece... Jul 26 22:36:12.444707114 (59h0m43.635005704s) [noise=true , address=79.127.219.38:4004 , piece_id=AWIQ35WXG7COCMIIBZWG4XUFPXX56N7NDH557P75WJODOFXXPR3Q, seg_pos={0 0}, upload=56599]: Uploading piece... Jul 26 22:36:12.445078254 (59h0m43.635376845s) [noise=true , address=79.127.205.233:4643 , piece_id=APLDJGHS27PL6CO54NBPXAIMTUTSVBWCD3REBYFCUG7HD7GQ4ILQ, seg_pos={0 0}, upload=56599]: Uploading piece... Jul 26 22:36:12.445771128 (59h0m43.636069713s) [noise=true , address=45.91.116.11:4146 , piece_id=RDOWY3FQR5SAI4AQAGW7OGBH2UHW7QJLYXP3X4VFXRRRK56IEVMQ, seg_pos={0 0}, upload=56599]: Uploading piece... Jul 26 22:36:12.446245114 (59h0m43.636543693s) [noise=true , address=79.127.205.234:4610 , piece_id=Z6E6WOW54X4IPTXPBFTVR7PCSPYHL6RE3TE462WGU6VEM5VNVZRQ, seg_pos={0 0}, upload=56599]: Uploading piece... Jul 26 22:36:12.446615655 (59h0m43.636914238s) [noise=true , address=79.127.205.245:4249 , piece_id=ZLTCHPKEK7YLHGPARB5RYBGURB3JYXI643L7CRLX7MBAFVQS6BQQ, seg_pos={0 0}, upload=56599]: Uploading piece... Jul 26 22:36:12.447213049 (59h0m43.637511632s) [noise=true , address=79.127.201.215:4359 , piece_id=PL6KYEJRUBPZ7TVIQX6ZRUP6FZ4556KWOEZNZIWGK2XJFA7F5JWQ, seg_pos={0 0}, upload=56599]: Uploading piece... Jul 26 22:36:13.021521918 (59h0m44.211820487s) [upload=56599]: Next returned nil segment Jul 26 22:36:13.021541143 (59h0m44.211839716s) [upload=56599]: Waiting for error group managing segments... Jul 26 22:36:13.184686035 (59h0m44.374984616s) [noise=true , address=77.90.31.13:4648 , piece_id=C5DBTLFHYM7BHH7DC3TJTSL7FQIIZENDQQCA6TEJHLIWPWU4WWOA, seg_pos={0 0}, upload=56599]: Done uploading piece. err: <nil> Jul 26 22:36:13.364199068 (59h0m44.554497648s) [noise=true , address=79.127.219.38:4004 , piece_id=AWIQ35WXG7COCMIIBZWG4XUFPXX56N7NDH557P75WJODOFXXPR3Q, seg_pos={0 0}, upload=56599]: Done uploading piece. err: <nil> Jul 26 22:36:13.670779313 (59h0m44.861077893s) [noise=true , address=64.49.13.138:4482 , piece_id=5LDVQSRIWHNJGBDH33DSFE6VEAMSDMG35EDVTLBBFZKBYN6LHSWQ, seg_pos={0 0}, upload=56599]: Done uploading piece. err: ecclient: upload failed (node:12joW2m4Uc87FSH65Cep1UKivUB1HG1119CQj14eWEKPcwaPsD5, address:64.49.13.138:4482): protocol: storage node overloaded, request limit: 2000 Jul 26 22:36:13.765699620 (59h0m44.955998208s) [noise=true , address=79.127.205.235:4685 , piece_id=4JQBKSAILRACTGUYF5O7OFZTLYCPCJ4RTCWBP7WMSWJEJNIJXVQA, seg_pos={0 0}, upload=56599]: Done uploading piece. err: <nil> Jul 26 22:36:13.782472445 (59h0m44.972771011s) [noise=true , address=79.127.205.245:4249 , piece_id=ZLTCHPKEK7YLHGPARB5RYBGURB3JYXI643L7CRLX7MBAFVQS6BQQ, seg_pos={0 0}, upload=56599]: Done uploading piece. err: <nil> Jul 26 22:36:13.783009822 (59h0m44.97330841s) [noise=true , address=79.127.205.234:4610 , piece_id=Z6E6WOW54X4IPTXPBFTVR7PCSPYHL6RE3TE462WGU6VEM5VNVZRQ, seg_pos={0 0}, upload=56599]: Done uploading piece. err: <nil> Jul 26 22:36:13.825147049 (59h0m45.015445615s) [noise=true , address=77.90.31.10:4550 , piece_id=BTZ2KE523G5U2CEKZYBQAHA7TEAMAWDI3L4ZRWMCOTJP5NHRBW5Q, seg_pos={0 0}, upload=56599]: Done uploading piece. err: <nil> Jul 26 22:36:13.867385487 (59h0m45.057684064s) [noise=true , address=79.127.205.246:4258 , piece_id=HHQKJ5HRUDLW65S6JIBKZSGL7TBIVCVBAKYZHCEHKLOKMIOZVDGA, seg_pos={0 0}, upload=56599]: Done uploading piece. err: <nil> Jul 26 22:36:13.890527042 (59h0m45.080825607s) [noise=true , address=79.127.205.233:4643 , piece_id=APLDJGHS27PL6CO54NBPXAIMTUTSVBWCD3REBYFCUG7HD7GQ4ILQ, seg_pos={0 0}, upload=56599]: Done uploading piece. err: <nil> Jul 26 22:36:18.644594737 (59h0m49.83489331s) [noise=true , address=77.90.31.17:4779 , piece_id=GHWGKUUDX32IYF5JLO74XTICI64VALC3CORU4MYPVXJXETKP7MIQ, seg_pos={0 0}, upload=56599]: Done uploading piece. err: <nil> Jul 26 22:36:19.205591684 (59h0m50.395890256s) [noise=true , address=64.49.13.140:4508 , piece_id=VHIE23HEY37IBJPIOFUIBPD24KMIUNY6KK4J36PCMVWMNHTT5BUQ, seg_pos={0 0}, upload=56599]: Done uploading piece. err: <nil> Jul 26 22:36:21.161339700 (59h0m52.351638273s) [noise=true , address=45.91.116.11:4146 , piece_id=RDOWY3FQR5SAI4AQAGW7OGBH2UHW7QJLYXP3X4VFXRRRK56IEVMQ, seg_pos={0 0}, upload=56599]: Done uploading piece. err: <nil> Jul 26 22:36:23.425491706 (59h0m54.615790286s) [noise=true , address=77.90.31.16:4760 , piece_id=HHS2FUWA4MQMFRQK5GSPDWMVV5OVIIY5LD6NVUKVK7CPKSVRXYZQ, seg_pos={0 0}, upload=56599]: Done uploading piece. err: ecclient: upload failed (node:1e6p1TD5CqUyWdZjkrxh2kRqMuyDiMUr9eimfeayDAKwhtBrZ8, address:77.90.31.16:4760): protocol: storage node overloaded, request limit: 2000 Jul 26 22:36:55.349289209 (59h1m26.539587775s) [noise=true , address=79.127.201.215:4359 , piece_id=PL6KYEJRUBPZ7TVIQX6ZRUP6FZ4556KWOEZNZIWGK2XJFA7F5JWQ, seg_pos={0 0}, upload=56599]: Done uploading piece. err: <nil> Jul 26 22:37:07.525108280 (59h1m38.71540686s) [noise=true , address=104.36.82.3:4769 , piece_id=K6IAAGSAT3U5FOGVC6EODKFIHHUYKQ5MI3T4M3RKUZ7G7OVQCVSQ, seg_pos={0 0}, upload=56599]: Done uploading piece. err: <nil> Jul 26 22:37:11.961214279 (59h1m43.15151286s) [noise=true , address=104.36.82.51:4082 , piece_id=CPPQ7UPWE4YSPOK2FMOXPQFLJQ3BZHLIVZ4AZNKZW5ETQLCS3RTQ, seg_pos={0 0}, upload=56599]: Done uploading piece. err: ecclient: upload failed (node:1m5NjTfYFQoyaftkARd75HMFZDW1jpnVj7H8LaUZWZcZYozHru, address:104.36.82.51:4082): protocol: write tcp 139.59.144.22:50476->104.36.82.51:4082: use of closed network connection; write tcp 139.59.144.22:50476->104.36.82.51:4082: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50476->104.36.82.51:4082: use of closed network connection Jul 26 22:37:11.961309591 (59h1m43.151608169s) [noise=true , address=77.90.31.18:4821 , piece_id=SWOWYMFGC72SF2YWPTX4UG4OJH4HYOLLHUIZGXADQKOEF2EMKU4Q, seg_pos={0 0}, upload=56599]: Done uploading piece. err: ecclient: upload failed (node:12dSS7NcpaDB98oZdBM6vFe56mejA8fJZKA5YvERXTD9qqXVPdH, address:77.90.31.18:4821): protocol: rpc: dial tcp 77.90.31.18:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 22:37:11.961349041 (59h1m43.15164762s) [noise=true , address=77.90.31.19:4875 , piece_id=SGYAJYPLA3CGLHWTU4X3DKGCENITO7P7WENNM5Y6WIUSLMXZRF6Q, seg_pos={0 0}, upload=56599]: Done uploading piece. err: ecclient: upload failed (node:1xSFQaW3aVnAVDYE5r5ADmCGU95vWnshuZyte32W2YAj94avNn, address:77.90.31.19:4875): protocol: write tcp 139.59.144.22:59064->77.90.31.19:4875: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 22:37:11.961373486 (59h1m43.151672066s) [seg_pos={0 0}, upload=56599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 26 22:37:12.069302764 (59h1m43.25960133s) [upload=56599]: Done uploading segments... Jul 26 22:37:12.069330870 (59h1m43.259629449s) [upload=56599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1m0.109608176s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............X.X.........................X................] duration: 2.265603066s FAIL (2025-07-26 22:37:09 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.265650346s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [X.X..X..XXX....X.X.X...............X........................] duration: 4.298235097s FAIL (2025-07-26 22:36:56 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 988 (58h42m45.061082575s) [upload=56308]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:18:14.018433272 (58h42m45.208731856s) [upload=56308]: Done uploading segments... Jul 26 22:18:14.018484438 (58h42m45.208783019s) [upload=56308]: Upload finished. err: <nil> Jul 26 22:18:57.520834389 (58h43m28.711132968s) [upload=56321]: Starting upload Jul 26 22:18:57.520934224 (58h43m28.711232794s) [upload=56321]: Done starting upload Jul 26 22:18:57.521309042 (58h43m28.711607621s) [upload=56321]: Uploading segments... Jul 26 22:18:57.521389036 (58h43m28.711687618s) [upload=56321]: Got next segment. Inline: false Jul 26 22:18:57.521430569 (58h43m28.711729148s) [upload=56321]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:18:57.521435990 (58h43m28.711734568s) [upload=56321]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:18:57.634354903 (58h43m28.824653493s) [seg_pos={0 0}, upload=56321]: Begin upload segment... Jul 26 22:18:57.634433002 (58h43m28.824731593s) [seg_pos={0 0}, upload=56321]: Done begin upload segment. Jul 26 22:18:57.634466006 (58h43m28.824764596s) [noise=true , address=104.36.82.12:4150 , piece_id=4PVRPBKTUNFKUWDJDTXYCIXDKONWG5OPQVB2M6KPKK66SMCYYHGA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.634507350 (58h43m28.82480593s) [noise=true , address=45.91.116.6:4973 , piece_id=S5BZKUNC2KNOVQMFRHSXOZNZZG6BUBK57ECM6W3S4YKN55KPOSFQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635011032 (58h43m28.825309612s) [noise=true , address=45.91.116.4:4938 , piece_id=INUA3CJT7KOAKKRYSBLTNXLX3MKWYYAFZ45BMUOZ6Y6I3B2EHHCQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635154508 (58h43m28.825453097s) [noise=true , address=45.91.116.8:4047 , piece_id=YTLRWKOOJ4XIGA3SKHFT2K66R57LBTW7OOVZSVJMCCV3YISPGWLA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635425419 (58h43m28.825724018s) [noise=true , address=45.91.116.8:4048 , piece_id=BF2LQK3BFGLPNCTDWB2M6HLRBSYBCXLR2AO7CMPERHT4DIBNKC4Q, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635823171 (58h43m28.826121762s) [noise=true , address=77.90.31.12:4590 , piece_id=PZSNEJOE623OVLNPZSRVVTTAGNHGRD7JCBUXCB3FXOMUE4CQNLFA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635848858 (58h43m28.826147438s) [noise=true , address=45.91.116.12:4227 , piece_id=FICCSE6IJQKUGYKTS3JHH4G65CPBOGDSPVIYXT25NR2GKRZQLOXA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.636411603 (58h43m28.826710186s) [noise=true , address=104.36.82.7:4909 , piece_id=HOFM67WVGW5QXUA65MEPQLJF7EV25AXBG4GNMREFYF2RNRGH2XWQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.636540327 (58h43m28.826838918s) [noise=true , address=77.90.31.11:4553 , piece_id=BP6FPURAP7D6RT34G72KZ32UY76QKXDG464WKNDPNUNOD6WS3KFQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.636837624 (58h43m28.827136209s) [noise=true , address=45.91.116.10:4081 , piece_id=2R44TDAMR2O3CNYFRZXOCSMG2ISUWMIQPVDWN3V2ECZWW4JUHNMA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.637107959 (58h43m28.827406542s) [noise=true , address=45.91.116.13:4154 , piece_id=PNBIV7A27TASCKR2CEQT7NJ5ZTG3NUQ2PLXJH7R47BFQQ53FLKUQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.637209450 (58h43m28.827508033s) [noise=true , address=77.90.31.21:4877 , piece_id=RCTFN3VAXG26GTA6GFQDQDIBAHJPOSI3QUSLLKO5APYYA57BSBRQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.637657116 (58h43m28.827955704s) [noise=true , address=45.91.116.7:4010 , piece_id=DZSLTEUQELXNHSH57IUZ6PRMOS56GR5TCWNAEEY7Y6YJ7KNDTRJA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.638167018 (58h43m28.828465602s) [noise=true , address=104.36.82.3:4767 , piece_id=IWRMBBOMGH7ZV3JA326TJ4SWHLEGA2T6CDLUDKJ4RFZBUZMTA3MQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.638722599 (58h43m28.829021185s) [noise=true , address=45.91.116.7:4009 , piece_id=2JHIJGUSNOSVSGOSGHHIYA2RNVLRS2P7MIKYHLSX4PTQOWNZKKGQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.639240448 (58h43m28.82953902s) [noise=true , address=104.36.82.11:4114 , piece_id=ZUBONJLMV4QRTG7DLIHWZICNNQ5OOF3HAPN26Q5AMK4TY4OYTXNA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:58.247247885 (58h43m29.437546466s) [upload=56321]: Next returned nil segment Jul 26 22:18:58.247273951 (58h43m29.437572537s) [upload=56321]: Waiting for error group managing segments... Jul 26 22:18:58.383663350 (58h43m29.573961931s) [noise=true , address=45.91.116.13:4154 , piece_id=PNBIV7A27TASCKR2CEQT7NJ5ZTG3NUQ2PLXJH7R47BFQQ53FLKUQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.384857324 (58h43m29.575155903s) [noise=true , address=45.91.116.8:4047 , piece_id=YTLRWKOOJ4XIGA3SKHFT2K66R57LBTW7OOVZSVJMCCV3YISPGWLA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.384981909 (58h43m29.575280495s) [noise=true , address=45.91.116.6:4973 , piece_id=S5BZKUNC2KNOVQMFRHSXOZNZZG6BUBK57ECM6W3S4YKN55KPOSFQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.387318149 (58h43m29.577616722s) [noise=true , address=45.91.116.7:4009 , piece_id=2JHIJGUSNOSVSGOSGHHIYA2RNVLRS2P7MIKYHLSX4PTQOWNZKKGQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.395512660 (58h43m29.585811243s) [noise=true , address=45.91.116.12:4227 , piece_id=FICCSE6IJQKUGYKTS3JHH4G65CPBOGDSPVIYXT25NR2GKRZQLOXA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.401895917 (58h43m29.592194499s) [noise=true , address=45.91.116.10:4081 , piece_id=2R44TDAMR2O3CNYFRZXOCSMG2ISUWMIQPVDWN3V2ECZWW4JUHNMA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.402135621 (58h43m29.59243419s) [noise=true , address=77.90.31.11:4553 , piece_id=BP6FPURAP7D6RT34G72KZ32UY76QKXDG464WKNDPNUNOD6WS3KFQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.403028120 (58h43m29.593326693s) [noise=true , address=45.91.116.8:4048 , piece_id=BF2LQK3BFGLPNCTDWB2M6HLRBSYBCXLR2AO7CMPERHT4DIBNKC4Q, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.404431307 (58h43m29.594729891s) [noise=true , address=45.91.116.4:4938 , piece_id=INUA3CJT7KOAKKRYSBLTNXLX3MKWYYAFZ45BMUOZ6Y6I3B2EHHCQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.409536042 (58h43m29.599834622s) [noise=true , address=45.91.116.7:4010 , piece_id=DZSLTEUQELXNHSH57IUZ6PRMOS56GR5TCWNAEEY7Y6YJ7KNDTRJA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:59.425708084 (58h43m30.616006681s) [noise=true , address=77.90.31.21:4877 , piece_id=RCTFN3VAXG26GTA6GFQDQDIBAHJPOSI3QUSLLKO5APYYA57BSBRQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:19:00.437139003 (58h43m31.627437582s) [noise=true , address=77.90.31.12:4590 , piece_id=PZSNEJOE623OVLNPZSRVVTTAGNHGRD7JCBUXCB3FXOMUE4CQNLFA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:19:27.246703337 (58h43m58.437001916s) [noise=true , address=104.36.82.7:4909 , piece_id=HOFM67WVGW5QXUA65MEPQLJF7EV25AXBG4GNMREFYF2RNRGH2XWQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: write tcp 139.59.144.22:57728->104.36.82.7:4909: use of closed network connection; write tcp 139.59.144.22:57728->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57728->104.36.82.7:4909: use of closed network connection Jul 26 22:19:27.246791447 (58h43m58.437090027s) [noise=true , address=104.36.82.11:4114 , piece_id=ZUBONJLMV4QRTG7DLIHWZICNNQ5OOF3HAPN26Q5AMK4TY4OYTXNA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 139.59.144.22:44152->104.36.82.11:4114: use of closed network connection; write tcp 139.59.144.22:44152->104.36.82.11:4114: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44152->104.36.82.11:4114: use of closed network connection Jul 26 22:19:27.246848250 (58h43m58.437146829s) [noise=true , address=104.36.82.3:4767 , piece_id=IWRMBBOMGH7ZV3JA326TJ4SWHLEGA2T6CDLUDKJ4RFZBUZMTA3MQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:52976->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:52976->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52976->104.36.82.3:4767: use of closed network connection Jul 26 22:19:27.246885606 (58h43m58.4371843s) [noise=true , address=104.36.82.12:4150 , piece_id=4PVRPBKTUNFKUWDJDTXYCIXDKONWG5OPQVB2M6KPKK66SMCYYHGA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:104.36.82.12:4150): protocol: write tcp 139.59.144.22:43452->104.36.82.12:4150: use of closed network connection; write tcp 139.59.144.22:43452->104.36.82.12:4150: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43452->104.36.82.12:4150: use of closed network connection Jul 26 22:19:27.246926689 (58h43m58.437225269s) [seg_pos={0 0}, upload=56321]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 22:19:27.354450344 (58h43m58.544748918s) [upload=56321]: Done uploading segments... Jul 26 22:19:27.354479247 (58h43m58.544777827s) [upload=56321]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 4.298272681s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [................X.X.........................X...............] duration: 2.266850901s FAIL (2025-07-26 22:36:46 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.266891918s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 829.642459ms FAIL (2025-07-26 22:36:31 UTC) orbiter warning: digest mismatch: after b0275bfe409329e90caa68828ca9cf99d935619c40adf20794b6b4a165f10dac != before 689768e222479cddedfe8605c20bdb7052e22d65868c907a97a7972f7764c1dd Duration: 829.681984ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 830.202852ms FAIL (2025-07-26 22:36:10 UTC) orbiter warning: digest mismatch: after b0275bfe409329e90caa68828ca9cf99d935619c40adf20794b6b4a165f10dac != before 689768e222479cddedfe8605c20bdb7052e22d65868c907a97a7972f7764c1dd Duration: 830.245405ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 17 out of 60 is failed (threshold=3) [.......................X.............XX...XXXXX.XXXX.XXXXX..] duration: 22.115765931s FAIL (2025-07-26 22:36:09 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 600 (58h58m42.315297198s) [upload=56565]: Uploading segments... Jul 26 22:34:11.125202148 (58h58m42.315500741s) [upload=56565]: Got next segment. Inline: false Jul 26 22:34:11.125253419 (58h58m42.31555201s) [upload=56565]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:34:11.125260194 (58h58m42.315558785s) [upload=56565]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:34:11.322095960 (58h58m42.51239454s) [seg_pos={0 0}, upload=56565]: Begin upload segment... Jul 26 22:34:11.322152447 (58h58m42.512451017s) [seg_pos={0 0}, upload=56565]: Done begin upload segment. Jul 26 22:34:11.322180304 (58h58m42.51247889s) [noise=true , address=77.90.31.16:4751 , piece_id=LR2DDIJUVM2BXIAJKD3LJFKABHQTQ42Y3RNAQYRWKPKXQY3A4C4A, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.322195626 (58h58m42.512494215s) [noise=true , address=64.49.13.170:4718 , piece_id=ETDQXE55JNTEUNXSS3CJK7DUSRYPGUCFI2F6H4A3UZURBKEZ5NXQ, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.322587864 (58h58m42.512886453s) [noise=true , address=79.127.205.245:4251 , piece_id=HXJ4SYW37W6NLZP2MQLO22JCK6TEGTRIPFYNNUFXVGESZFBWYMUA, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.322814415 (58h58m42.513113001s) [noise=true , address=156.146.43.225:4449 , piece_id=R346CJ2RFMJZLYYD7TGJUYZG4F4QBOCAN3VK62JS3VY6XV4R3KFA, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.322920678 (58h58m42.513219267s) [noise=true , address=77.90.31.19:4867 , piece_id=25JAFIHTYBGPUT5JFBA2WTXOQYB2RP5DMTXKEYKJV3WZ7OQXBSNQ, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.323255183 (58h58m42.513553765s) [noise=true , address=77.90.31.18:4828 , piece_id=LTMMF2PK7F5VJOX4FGY5SA4HTN3OBSEPDXP43YRDIXIVWBN2ZGSA, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.324172222 (58h58m42.514470811s) [noise=true , address=79.127.219.38:4006 , piece_id=P5CGL67CZK4I5XZHYV3MLMCSSB3A3OE7MRN5Y2BOQZBQAJ6QOO3A, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.324541666 (58h58m42.514840249s) [noise=true , address=79.127.220.97:4361 , piece_id=TPIJYKAADUSKUPBGVPXSBIAISV7CZYXTJTFE2XMAVFAWOTEJM2RA, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.324857179 (58h58m42.515155771s) [noise=true , address=64.49.13.138:4482 , piece_id=M7PX5EAZLT5R6YUPADHQHPHTS7JTDLA2Q24YT3DGWDJB44E4CIJQ, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.325289211 (58h58m42.515587802s) [noise=true , address=79.127.205.246:4286 , piece_id=EU4B36TTO4LNKPMPBJE3BSY4VIAZD6U7T7GJA7PU7JNPIDGDUP6Q, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.325665458 (58h58m42.515964043s) [noise=true , address=79.127.205.234:4603 , piece_id=FZ3SCGNCEGBJ63EYLLZ2ZCIVCEO2GTRAG437HPCW6WHZWWLZTI5Q, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.326231548 (58h58m42.51653015s) [noise=true , address=45.91.116.11:4146 , piece_id=ONZDCEH3EL4U32I7ZLX7WNIGDS6LBJKMQ2BM5HPGM73PI4KW5SHQ, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.326620079 (58h58m42.516918661s) [noise=true , address=77.90.31.11:4560 , piece_id=XNKEYPHFRAG7576QHO6DLFKLX7H6ELTPMP6NLWMPAL2WILGBVUJA, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.326976175 (58h58m42.517274764s) [noise=true , address=79.127.201.215:4347 , piece_id=KEQBM35UNQ4MX7P75XTWPIVL23J6RUYGRTNNXWK4TP4YBD3XA7RA, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.327299882 (58h58m42.517598464s) [noise=true , address=64.49.13.140:4514 , piece_id=UMBGV3GMSPBFDANS7JTETCZI3LRSONWRFUUOIJUFB65UPSHFN5IA, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.327607309 (58h58m42.517905891s) [noise=true , address=79.127.219.51:4212 , piece_id=A7PRZXFXLDQJ27GAY5RBDCYPJPXUPINHHAJVL4A3XCKVPBT2ZC7Q, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.327959291 (58h58m42.51825787s) [noise=true , address=79.127.219.47:4069 , piece_id=G55S75BXMTYQ5RWIPWL2BT64ODX2P4E4OKNJ2KI6IMKU4NP4ZY7Q, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.328438540 (58h58m42.518737122s) [noise=true , address=104.36.82.34:4565 , piece_id=LFCJWIDOVUZLJMXTTQ7XKWL4E6ANKEKRGJYSA3QZJF2UIR67LDFA, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.778832969 (58h58m42.969131559s) [noise=true , address=79.127.219.47:4069 , piece_id=G55S75BXMTYQ5RWIPWL2BT64ODX2P4E4OKNJ2KI6IMKU4NP4ZY7Q, seg_pos={0 0}, upload=56565]: Done uploading piece. err: ecclient: upload failed (node:18dK6RxbMxU4a46pW4YPiQW87xgpeoTb1ouqE9fzTdzTAFAa3W, address:79.127.219.47:4069): protocol: storage node overloaded, request limit: 2000 Jul 26 22:34:11.870845338 (58h58m43.061143929s) [noise=true , address=79.127.220.97:4361 , piece_id=TPIJYKAADUSKUPBGVPXSBIAISV7CZYXTJTFE2XMAVFAWOTEJM2RA, seg_pos={0 0}, upload=56565]: Done uploading piece. err: ecclient: upload failed (node:1ojv5zf5QshKyMqGypaEaYmGvzt96UM9kZgyGeFVPMFk8Twzyn, address:79.127.220.97:4361): protocol: storage node overloaded, request limit: 2000 Jul 26 22:34:11.911516886 (58h58m43.101815467s) [upload=56565]: Next returned nil segment Jul 26 22:34:11.911535331 (58h58m43.101833922s) [upload=56565]: Waiting for error group managing segments... Jul 26 22:34:12.023817302 (58h58m43.214115883s) [noise=true , address=64.49.13.138:4482 , piece_id=M7PX5EAZLT5R6YUPADHQHPHTS7JTDLA2Q24YT3DGWDJB44E4CIJQ, seg_pos={0 0}, upload=56565]: Done uploading piece. err: ecclient: upload failed (node:12joW2m4Uc87FSH65Cep1UKivUB1HG1119CQj14eWEKPcwaPsD5, address:64.49.13.138:4482): protocol: storage node overloaded, request limit: 2000 Jul 26 22:34:12.064253960 (58h58m43.25455264s) [noise=true , address=77.90.31.11:4560 , piece_id=XNKEYPHFRAG7576QHO6DLFKLX7H6ELTPMP6NLWMPAL2WILGBVUJA, seg_pos={0 0}, upload=56565]: Done uploading piece. err: <nil> Jul 26 22:34:12.085636218 (58h58m43.275934793s) [noise=true , address=79.127.219.38:4006 , piece_id=P5CGL67CZK4I5XZHYV3MLMCSSB3A3OE7MRN5Y2BOQZBQAJ6QOO3A, seg_pos={0 0}, upload=56565]: Done uploading piece. err: <nil> Jul 26 22:34:12.547021607 (58h58m43.737320188s) [noise=true , address=45.91.116.11:4146 , piece_id=ONZDCEH3EL4U32I7ZLX7WNIGDS6LBJKMQ2BM5HPGM73PI4KW5SHQ, seg_pos={0 0}, upload=56565]: Done uploading piece. err: <nil> Jul 26 22:34:12.608354679 (58h58m43.798653269s) [noise=true , address=64.49.13.140:4514 , piece_id=UMBGV3GMSPBFDANS7JTETCZI3LRSONWRFUUOIJUFB65UPSHFN5IA, seg_pos={0 0}, upload=56565]: Done uploading piece. err: <nil> Jul 26 22:34:12.628670275 (58h58m43.818968858s) [noise=true , address=79.127.205.246:4286 , piece_id=EU4B36TTO4LNKPMPBJE3BSY4VIAZD6U7T7GJA7PU7JNPIDGDUP6Q, seg_pos={0 0}, upload=56565]: Done uploading piece. err: <nil> Jul 26 22:34:12.653038575 (58h58m43.843337142s) [noise=true , address=79.127.205.245:4251 , piece_id=HXJ4SYW37W6NLZP2MQLO22JCK6TEGTRIPFYNNUFXVGESZFBWYMUA, seg_pos={0 0}, upload=56565]: Done uploading piece. err: <nil> Jul 26 22:34:12.680005647 (58h58m43.870304236s) [noise=true , address=64.49.13.170:4718 , piece_id=ETDQXE55JNTEUNXSS3CJK7DUSRYPGUCFI2F6H4A3UZURBKEZ5NXQ, seg_pos={0 0}, upload=56565]: Done uploading piece. err: <nil> Jul 26 22:34:12.987584251 (58h58m44.177882822s) [noise=true , address=79.127.205.234:4603 , piece_id=FZ3SCGNCEGBJ63EYLLZ2ZCIVCEO2GTRAG437HPCW6WHZWWLZTI5Q, seg_pos={0 0}, upload=56565]: Done uploading piece. err: <nil> Jul 26 22:34:13.171498895 (58h58m44.361797468s) [noise=true , address=156.146.43.225:4449 , piece_id=R346CJ2RFMJZLYYD7TGJUYZG4F4QBOCAN3VK62JS3VY6XV4R3KFA, seg_pos={0 0}, upload=56565]: Done uploading piece. err: <nil> Jul 26 22:34:14.205159349 (58h58m45.395457939s) [noise=true , address=79.127.219.51:4212 , piece_id=A7PRZXFXLDQJ27GAY5RBDCYPJPXUPINHHAJVL4A3XCKVPBT2ZC7Q, seg_pos={0 0}, upload=56565]: Done uploading piece. err: <nil> Jul 26 22:34:21.265563919 (58h58m52.455862501s) [noise=true , address=79.127.201.215:4347 , piece_id=KEQBM35UNQ4MX7P75XTWPIVL23J6RUYGRTNNXWK4TP4YBD3XA7RA, seg_pos={0 0}, upload=56565]: Done uploading piece. err: <nil> Jul 26 22:34:29.546831777 (58h59m0.737130356s) [noise=true , address=104.36.82.34:4565 , piece_id=LFCJWIDOVUZLJMXTTQ7XKWL4E6ANKEKRGJYSA3QZJF2UIR67LDFA, seg_pos={0 0}, upload=56565]: Done uploading piece. err: <nil> Jul 26 22:35:10.854110455 (58h59m42.044409045s) [noise=true , address=77.90.31.16:4751 , piece_id=LR2DDIJUVM2BXIAJKD3LJFKABHQTQ42Y3RNAQYRWKPKXQY3A4C4A, seg_pos={0 0}, upload=56565]: Done uploading piece. err: ecclient: upload failed (node:12Sj7nPQiL1EmwvnVCpRv1HEeDYFnknTXLNZTvhAUmHDcP2Wpuf, address:77.90.31.16:4751): protocol: rpc: dial tcp 77.90.31.16:4751: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 22:35:10.854232997 (58h59m42.044531588s) [noise=true , address=77.90.31.19:4867 , piece_id=25JAFIHTYBGPUT5JFBA2WTXOQYB2RP5DMTXKEYKJV3WZ7OQXBSNQ, seg_pos={0 0}, upload=56565]: Done uploading piece. err: ecclient: upload failed (node:1gczGyJMCtqUfaac5oy7o4CTwypVF8AZjrAAjoJweU3kBFbuXw, address:77.90.31.19:4867): protocol: rpc: dial tcp 77.90.31.19:4867: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 22:35:10.854402880 (58h59m42.044701459s) [noise=true , address=77.90.31.18:4828 , piece_id=LTMMF2PK7F5VJOX4FGY5SA4HTN3OBSEPDXP43YRDIXIVWBN2ZGSA, seg_pos={0 0}, upload=56565]: Done uploading piece. err: ecclient: upload failed (node:1jwTDXPShXfNdfMx3UXi22gQphBpFre1MhRsHyjq6xRz4wwTr9, address:77.90.31.18:4828): protocol: write tcp 139.59.144.22:39748->77.90.31.18:4828: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Jul 26 22:35:10.854431573 (58h59m42.044730153s) [seg_pos={0 0}, upload=56565]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Jul 26 22:35:10.969025541 (58h59m42.159324121s) [upload=56565]: Done uploading segments... Jul 26 22:35:10.969054726 (58h59m42.159353292s) [upload=56565]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 22.115796018s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................X.X.........................X..............] duration: 2.23842816s FAIL (2025-07-26 22:35:57 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.238503127s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 842.261941ms FAIL (2025-07-26 22:35:48 UTC) orbiter warning: digest mismatch: after b0275bfe409329e90caa68828ca9cf99d935619c40adf20794b6b4a165f10dac != before 689768e222479cddedfe8605c20bdb7052e22d65868c907a97a7972f7764c1dd Duration: 842.322085ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.X.X..X..XXX....X.X.X...............X.......................] duration: 2.432904499s FAIL (2025-07-26 22:35:47 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 988 (58h42m45.061082575s) [upload=56308]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:18:14.018433272 (58h42m45.208731856s) [upload=56308]: Done uploading segments... Jul 26 22:18:14.018484438 (58h42m45.208783019s) [upload=56308]: Upload finished. err: <nil> Jul 26 22:18:57.520834389 (58h43m28.711132968s) [upload=56321]: Starting upload Jul 26 22:18:57.520934224 (58h43m28.711232794s) [upload=56321]: Done starting upload Jul 26 22:18:57.521309042 (58h43m28.711607621s) [upload=56321]: Uploading segments... Jul 26 22:18:57.521389036 (58h43m28.711687618s) [upload=56321]: Got next segment. Inline: false Jul 26 22:18:57.521430569 (58h43m28.711729148s) [upload=56321]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:18:57.521435990 (58h43m28.711734568s) [upload=56321]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:18:57.634354903 (58h43m28.824653493s) [seg_pos={0 0}, upload=56321]: Begin upload segment... Jul 26 22:18:57.634433002 (58h43m28.824731593s) [seg_pos={0 0}, upload=56321]: Done begin upload segment. Jul 26 22:18:57.634466006 (58h43m28.824764596s) [noise=true , address=104.36.82.12:4150 , piece_id=4PVRPBKTUNFKUWDJDTXYCIXDKONWG5OPQVB2M6KPKK66SMCYYHGA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.634507350 (58h43m28.82480593s) [noise=true , address=45.91.116.6:4973 , piece_id=S5BZKUNC2KNOVQMFRHSXOZNZZG6BUBK57ECM6W3S4YKN55KPOSFQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635011032 (58h43m28.825309612s) [noise=true , address=45.91.116.4:4938 , piece_id=INUA3CJT7KOAKKRYSBLTNXLX3MKWYYAFZ45BMUOZ6Y6I3B2EHHCQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635154508 (58h43m28.825453097s) [noise=true , address=45.91.116.8:4047 , piece_id=YTLRWKOOJ4XIGA3SKHFT2K66R57LBTW7OOVZSVJMCCV3YISPGWLA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635425419 (58h43m28.825724018s) [noise=true , address=45.91.116.8:4048 , piece_id=BF2LQK3BFGLPNCTDWB2M6HLRBSYBCXLR2AO7CMPERHT4DIBNKC4Q, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635823171 (58h43m28.826121762s) [noise=true , address=77.90.31.12:4590 , piece_id=PZSNEJOE623OVLNPZSRVVTTAGNHGRD7JCBUXCB3FXOMUE4CQNLFA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635848858 (58h43m28.826147438s) [noise=true , address=45.91.116.12:4227 , piece_id=FICCSE6IJQKUGYKTS3JHH4G65CPBOGDSPVIYXT25NR2GKRZQLOXA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.636411603 (58h43m28.826710186s) [noise=true , address=104.36.82.7:4909 , piece_id=HOFM67WVGW5QXUA65MEPQLJF7EV25AXBG4GNMREFYF2RNRGH2XWQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.636540327 (58h43m28.826838918s) [noise=true , address=77.90.31.11:4553 , piece_id=BP6FPURAP7D6RT34G72KZ32UY76QKXDG464WKNDPNUNOD6WS3KFQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.636837624 (58h43m28.827136209s) [noise=true , address=45.91.116.10:4081 , piece_id=2R44TDAMR2O3CNYFRZXOCSMG2ISUWMIQPVDWN3V2ECZWW4JUHNMA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.637107959 (58h43m28.827406542s) [noise=true , address=45.91.116.13:4154 , piece_id=PNBIV7A27TASCKR2CEQT7NJ5ZTG3NUQ2PLXJH7R47BFQQ53FLKUQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.637209450 (58h43m28.827508033s) [noise=true , address=77.90.31.21:4877 , piece_id=RCTFN3VAXG26GTA6GFQDQDIBAHJPOSI3QUSLLKO5APYYA57BSBRQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.637657116 (58h43m28.827955704s) [noise=true , address=45.91.116.7:4010 , piece_id=DZSLTEUQELXNHSH57IUZ6PRMOS56GR5TCWNAEEY7Y6YJ7KNDTRJA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.638167018 (58h43m28.828465602s) [noise=true , address=104.36.82.3:4767 , piece_id=IWRMBBOMGH7ZV3JA326TJ4SWHLEGA2T6CDLUDKJ4RFZBUZMTA3MQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.638722599 (58h43m28.829021185s) [noise=true , address=45.91.116.7:4009 , piece_id=2JHIJGUSNOSVSGOSGHHIYA2RNVLRS2P7MIKYHLSX4PTQOWNZKKGQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.639240448 (58h43m28.82953902s) [noise=true , address=104.36.82.11:4114 , piece_id=ZUBONJLMV4QRTG7DLIHWZICNNQ5OOF3HAPN26Q5AMK4TY4OYTXNA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:58.247247885 (58h43m29.437546466s) [upload=56321]: Next returned nil segment Jul 26 22:18:58.247273951 (58h43m29.437572537s) [upload=56321]: Waiting for error group managing segments... Jul 26 22:18:58.383663350 (58h43m29.573961931s) [noise=true , address=45.91.116.13:4154 , piece_id=PNBIV7A27TASCKR2CEQT7NJ5ZTG3NUQ2PLXJH7R47BFQQ53FLKUQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.384857324 (58h43m29.575155903s) [noise=true , address=45.91.116.8:4047 , piece_id=YTLRWKOOJ4XIGA3SKHFT2K66R57LBTW7OOVZSVJMCCV3YISPGWLA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.384981909 (58h43m29.575280495s) [noise=true , address=45.91.116.6:4973 , piece_id=S5BZKUNC2KNOVQMFRHSXOZNZZG6BUBK57ECM6W3S4YKN55KPOSFQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.387318149 (58h43m29.577616722s) [noise=true , address=45.91.116.7:4009 , piece_id=2JHIJGUSNOSVSGOSGHHIYA2RNVLRS2P7MIKYHLSX4PTQOWNZKKGQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.395512660 (58h43m29.585811243s) [noise=true , address=45.91.116.12:4227 , piece_id=FICCSE6IJQKUGYKTS3JHH4G65CPBOGDSPVIYXT25NR2GKRZQLOXA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.401895917 (58h43m29.592194499s) [noise=true , address=45.91.116.10:4081 , piece_id=2R44TDAMR2O3CNYFRZXOCSMG2ISUWMIQPVDWN3V2ECZWW4JUHNMA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.402135621 (58h43m29.59243419s) [noise=true , address=77.90.31.11:4553 , piece_id=BP6FPURAP7D6RT34G72KZ32UY76QKXDG464WKNDPNUNOD6WS3KFQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.403028120 (58h43m29.593326693s) [noise=true , address=45.91.116.8:4048 , piece_id=BF2LQK3BFGLPNCTDWB2M6HLRBSYBCXLR2AO7CMPERHT4DIBNKC4Q, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.404431307 (58h43m29.594729891s) [noise=true , address=45.91.116.4:4938 , piece_id=INUA3CJT7KOAKKRYSBLTNXLX3MKWYYAFZ45BMUOZ6Y6I3B2EHHCQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.409536042 (58h43m29.599834622s) [noise=true , address=45.91.116.7:4010 , piece_id=DZSLTEUQELXNHSH57IUZ6PRMOS56GR5TCWNAEEY7Y6YJ7KNDTRJA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:59.425708084 (58h43m30.616006681s) [noise=true , address=77.90.31.21:4877 , piece_id=RCTFN3VAXG26GTA6GFQDQDIBAHJPOSI3QUSLLKO5APYYA57BSBRQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:19:00.437139003 (58h43m31.627437582s) [noise=true , address=77.90.31.12:4590 , piece_id=PZSNEJOE623OVLNPZSRVVTTAGNHGRD7JCBUXCB3FXOMUE4CQNLFA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:19:27.246703337 (58h43m58.437001916s) [noise=true , address=104.36.82.7:4909 , piece_id=HOFM67WVGW5QXUA65MEPQLJF7EV25AXBG4GNMREFYF2RNRGH2XWQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: write tcp 139.59.144.22:57728->104.36.82.7:4909: use of closed network connection; write tcp 139.59.144.22:57728->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57728->104.36.82.7:4909: use of closed network connection Jul 26 22:19:27.246791447 (58h43m58.437090027s) [noise=true , address=104.36.82.11:4114 , piece_id=ZUBONJLMV4QRTG7DLIHWZICNNQ5OOF3HAPN26Q5AMK4TY4OYTXNA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 139.59.144.22:44152->104.36.82.11:4114: use of closed network connection; write tcp 139.59.144.22:44152->104.36.82.11:4114: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44152->104.36.82.11:4114: use of closed network connection Jul 26 22:19:27.246848250 (58h43m58.437146829s) [noise=true , address=104.36.82.3:4767 , piece_id=IWRMBBOMGH7ZV3JA326TJ4SWHLEGA2T6CDLUDKJ4RFZBUZMTA3MQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:52976->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:52976->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52976->104.36.82.3:4767: use of closed network connection Jul 26 22:19:27.246885606 (58h43m58.4371843s) [noise=true , address=104.36.82.12:4150 , piece_id=4PVRPBKTUNFKUWDJDTXYCIXDKONWG5OPQVB2M6KPKK66SMCYYHGA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:104.36.82.12:4150): protocol: write tcp 139.59.144.22:43452->104.36.82.12:4150: use of closed network connection; write tcp 139.59.144.22:43452->104.36.82.12:4150: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43452->104.36.82.12:4150: use of closed network connection Jul 26 22:19:27.246926689 (58h43m58.437225269s) [seg_pos={0 0}, upload=56321]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 22:19:27.354450344 (58h43m58.544748918s) [upload=56321]: Done uploading segments... Jul 26 22:19:27.354479247 (58h43m58.544777827s) [upload=56321]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 2.432936369s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 7 out of 60 is failed (threshold=3) [...................................X.......X..X...X.XX...X..] duration: 5.387679422s FAIL (2025-07-26 22:35:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.387777901s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 17 out of 60 is failed (threshold=3) [........................X.............XX...XXXXX.XXXX.XXXXX.] duration: 24.962503665s FAIL (2025-07-26 22:35:43 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 600 (58h58m42.315297198s) [upload=56565]: Uploading segments... Jul 26 22:34:11.125202148 (58h58m42.315500741s) [upload=56565]: Got next segment. Inline: false Jul 26 22:34:11.125253419 (58h58m42.31555201s) [upload=56565]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:34:11.125260194 (58h58m42.315558785s) [upload=56565]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:34:11.322095960 (58h58m42.51239454s) [seg_pos={0 0}, upload=56565]: Begin upload segment... Jul 26 22:34:11.322152447 (58h58m42.512451017s) [seg_pos={0 0}, upload=56565]: Done begin upload segment. Jul 26 22:34:11.322180304 (58h58m42.51247889s) [noise=true , address=77.90.31.16:4751 , piece_id=LR2DDIJUVM2BXIAJKD3LJFKABHQTQ42Y3RNAQYRWKPKXQY3A4C4A, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.322195626 (58h58m42.512494215s) [noise=true , address=64.49.13.170:4718 , piece_id=ETDQXE55JNTEUNXSS3CJK7DUSRYPGUCFI2F6H4A3UZURBKEZ5NXQ, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.322587864 (58h58m42.512886453s) [noise=true , address=79.127.205.245:4251 , piece_id=HXJ4SYW37W6NLZP2MQLO22JCK6TEGTRIPFYNNUFXVGESZFBWYMUA, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.322814415 (58h58m42.513113001s) [noise=true , address=156.146.43.225:4449 , piece_id=R346CJ2RFMJZLYYD7TGJUYZG4F4QBOCAN3VK62JS3VY6XV4R3KFA, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.322920678 (58h58m42.513219267s) [noise=true , address=77.90.31.19:4867 , piece_id=25JAFIHTYBGPUT5JFBA2WTXOQYB2RP5DMTXKEYKJV3WZ7OQXBSNQ, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.323255183 (58h58m42.513553765s) [noise=true , address=77.90.31.18:4828 , piece_id=LTMMF2PK7F5VJOX4FGY5SA4HTN3OBSEPDXP43YRDIXIVWBN2ZGSA, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.324172222 (58h58m42.514470811s) [noise=true , address=79.127.219.38:4006 , piece_id=P5CGL67CZK4I5XZHYV3MLMCSSB3A3OE7MRN5Y2BOQZBQAJ6QOO3A, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.324541666 (58h58m42.514840249s) [noise=true , address=79.127.220.97:4361 , piece_id=TPIJYKAADUSKUPBGVPXSBIAISV7CZYXTJTFE2XMAVFAWOTEJM2RA, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.324857179 (58h58m42.515155771s) [noise=true , address=64.49.13.138:4482 , piece_id=M7PX5EAZLT5R6YUPADHQHPHTS7JTDLA2Q24YT3DGWDJB44E4CIJQ, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.325289211 (58h58m42.515587802s) [noise=true , address=79.127.205.246:4286 , piece_id=EU4B36TTO4LNKPMPBJE3BSY4VIAZD6U7T7GJA7PU7JNPIDGDUP6Q, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.325665458 (58h58m42.515964043s) [noise=true , address=79.127.205.234:4603 , piece_id=FZ3SCGNCEGBJ63EYLLZ2ZCIVCEO2GTRAG437HPCW6WHZWWLZTI5Q, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.326231548 (58h58m42.51653015s) [noise=true , address=45.91.116.11:4146 , piece_id=ONZDCEH3EL4U32I7ZLX7WNIGDS6LBJKMQ2BM5HPGM73PI4KW5SHQ, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.326620079 (58h58m42.516918661s) [noise=true , address=77.90.31.11:4560 , piece_id=XNKEYPHFRAG7576QHO6DLFKLX7H6ELTPMP6NLWMPAL2WILGBVUJA, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.326976175 (58h58m42.517274764s) [noise=true , address=79.127.201.215:4347 , piece_id=KEQBM35UNQ4MX7P75XTWPIVL23J6RUYGRTNNXWK4TP4YBD3XA7RA, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.327299882 (58h58m42.517598464s) [noise=true , address=64.49.13.140:4514 , piece_id=UMBGV3GMSPBFDANS7JTETCZI3LRSONWRFUUOIJUFB65UPSHFN5IA, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.327607309 (58h58m42.517905891s) [noise=true , address=79.127.219.51:4212 , piece_id=A7PRZXFXLDQJ27GAY5RBDCYPJPXUPINHHAJVL4A3XCKVPBT2ZC7Q, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.327959291 (58h58m42.51825787s) [noise=true , address=79.127.219.47:4069 , piece_id=G55S75BXMTYQ5RWIPWL2BT64ODX2P4E4OKNJ2KI6IMKU4NP4ZY7Q, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.328438540 (58h58m42.518737122s) [noise=true , address=104.36.82.34:4565 , piece_id=LFCJWIDOVUZLJMXTTQ7XKWL4E6ANKEKRGJYSA3QZJF2UIR67LDFA, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.778832969 (58h58m42.969131559s) [noise=true , address=79.127.219.47:4069 , piece_id=G55S75BXMTYQ5RWIPWL2BT64ODX2P4E4OKNJ2KI6IMKU4NP4ZY7Q, seg_pos={0 0}, upload=56565]: Done uploading piece. err: ecclient: upload failed (node:18dK6RxbMxU4a46pW4YPiQW87xgpeoTb1ouqE9fzTdzTAFAa3W, address:79.127.219.47:4069): protocol: storage node overloaded, request limit: 2000 Jul 26 22:34:11.870845338 (58h58m43.061143929s) [noise=true , address=79.127.220.97:4361 , piece_id=TPIJYKAADUSKUPBGVPXSBIAISV7CZYXTJTFE2XMAVFAWOTEJM2RA, seg_pos={0 0}, upload=56565]: Done uploading piece. err: ecclient: upload failed (node:1ojv5zf5QshKyMqGypaEaYmGvzt96UM9kZgyGeFVPMFk8Twzyn, address:79.127.220.97:4361): protocol: storage node overloaded, request limit: 2000 Jul 26 22:34:11.911516886 (58h58m43.101815467s) [upload=56565]: Next returned nil segment Jul 26 22:34:11.911535331 (58h58m43.101833922s) [upload=56565]: Waiting for error group managing segments... Jul 26 22:34:12.023817302 (58h58m43.214115883s) [noise=true , address=64.49.13.138:4482 , piece_id=M7PX5EAZLT5R6YUPADHQHPHTS7JTDLA2Q24YT3DGWDJB44E4CIJQ, seg_pos={0 0}, upload=56565]: Done uploading piece. err: ecclient: upload failed (node:12joW2m4Uc87FSH65Cep1UKivUB1HG1119CQj14eWEKPcwaPsD5, address:64.49.13.138:4482): protocol: storage node overloaded, request limit: 2000 Jul 26 22:34:12.064253960 (58h58m43.25455264s) [noise=true , address=77.90.31.11:4560 , piece_id=XNKEYPHFRAG7576QHO6DLFKLX7H6ELTPMP6NLWMPAL2WILGBVUJA, seg_pos={0 0}, upload=56565]: Done uploading piece. err: <nil> Jul 26 22:34:12.085636218 (58h58m43.275934793s) [noise=true , address=79.127.219.38:4006 , piece_id=P5CGL67CZK4I5XZHYV3MLMCSSB3A3OE7MRN5Y2BOQZBQAJ6QOO3A, seg_pos={0 0}, upload=56565]: Done uploading piece. err: <nil> Jul 26 22:34:12.547021607 (58h58m43.737320188s) [noise=true , address=45.91.116.11:4146 , piece_id=ONZDCEH3EL4U32I7ZLX7WNIGDS6LBJKMQ2BM5HPGM73PI4KW5SHQ, seg_pos={0 0}, upload=56565]: Done uploading piece. err: <nil> Jul 26 22:34:12.608354679 (58h58m43.798653269s) [noise=true , address=64.49.13.140:4514 , piece_id=UMBGV3GMSPBFDANS7JTETCZI3LRSONWRFUUOIJUFB65UPSHFN5IA, seg_pos={0 0}, upload=56565]: Done uploading piece. err: <nil> Jul 26 22:34:12.628670275 (58h58m43.818968858s) [noise=true , address=79.127.205.246:4286 , piece_id=EU4B36TTO4LNKPMPBJE3BSY4VIAZD6U7T7GJA7PU7JNPIDGDUP6Q, seg_pos={0 0}, upload=56565]: Done uploading piece. err: <nil> Jul 26 22:34:12.653038575 (58h58m43.843337142s) [noise=true , address=79.127.205.245:4251 , piece_id=HXJ4SYW37W6NLZP2MQLO22JCK6TEGTRIPFYNNUFXVGESZFBWYMUA, seg_pos={0 0}, upload=56565]: Done uploading piece. err: <nil> Jul 26 22:34:12.680005647 (58h58m43.870304236s) [noise=true , address=64.49.13.170:4718 , piece_id=ETDQXE55JNTEUNXSS3CJK7DUSRYPGUCFI2F6H4A3UZURBKEZ5NXQ, seg_pos={0 0}, upload=56565]: Done uploading piece. err: <nil> Jul 26 22:34:12.987584251 (58h58m44.177882822s) [noise=true , address=79.127.205.234:4603 , piece_id=FZ3SCGNCEGBJ63EYLLZ2ZCIVCEO2GTRAG437HPCW6WHZWWLZTI5Q, seg_pos={0 0}, upload=56565]: Done uploading piece. err: <nil> Jul 26 22:34:13.171498895 (58h58m44.361797468s) [noise=true , address=156.146.43.225:4449 , piece_id=R346CJ2RFMJZLYYD7TGJUYZG4F4QBOCAN3VK62JS3VY6XV4R3KFA, seg_pos={0 0}, upload=56565]: Done uploading piece. err: <nil> Jul 26 22:34:14.205159349 (58h58m45.395457939s) [noise=true , address=79.127.219.51:4212 , piece_id=A7PRZXFXLDQJ27GAY5RBDCYPJPXUPINHHAJVL4A3XCKVPBT2ZC7Q, seg_pos={0 0}, upload=56565]: Done uploading piece. err: <nil> Jul 26 22:34:21.265563919 (58h58m52.455862501s) [noise=true , address=79.127.201.215:4347 , piece_id=KEQBM35UNQ4MX7P75XTWPIVL23J6RUYGRTNNXWK4TP4YBD3XA7RA, seg_pos={0 0}, upload=56565]: Done uploading piece. err: <nil> Jul 26 22:34:29.546831777 (58h59m0.737130356s) [noise=true , address=104.36.82.34:4565 , piece_id=LFCJWIDOVUZLJMXTTQ7XKWL4E6ANKEKRGJYSA3QZJF2UIR67LDFA, seg_pos={0 0}, upload=56565]: Done uploading piece. err: <nil> Jul 26 22:35:10.854110455 (58h59m42.044409045s) [noise=true , address=77.90.31.16:4751 , piece_id=LR2DDIJUVM2BXIAJKD3LJFKABHQTQ42Y3RNAQYRWKPKXQY3A4C4A, seg_pos={0 0}, upload=56565]: Done uploading piece. err: ecclient: upload failed (node:12Sj7nPQiL1EmwvnVCpRv1HEeDYFnknTXLNZTvhAUmHDcP2Wpuf, address:77.90.31.16:4751): protocol: rpc: dial tcp 77.90.31.16:4751: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 22:35:10.854232997 (58h59m42.044531588s) [noise=true , address=77.90.31.19:4867 , piece_id=25JAFIHTYBGPUT5JFBA2WTXOQYB2RP5DMTXKEYKJV3WZ7OQXBSNQ, seg_pos={0 0}, upload=56565]: Done uploading piece. err: ecclient: upload failed (node:1gczGyJMCtqUfaac5oy7o4CTwypVF8AZjrAAjoJweU3kBFbuXw, address:77.90.31.19:4867): protocol: rpc: dial tcp 77.90.31.19:4867: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 22:35:10.854402880 (58h59m42.044701459s) [noise=true , address=77.90.31.18:4828 , piece_id=LTMMF2PK7F5VJOX4FGY5SA4HTN3OBSEPDXP43YRDIXIVWBN2ZGSA, seg_pos={0 0}, upload=56565]: Done uploading piece. err: ecclient: upload failed (node:1jwTDXPShXfNdfMx3UXi22gQphBpFre1MhRsHyjq6xRz4wwTr9, address:77.90.31.18:4828): protocol: write tcp 139.59.144.22:39748->77.90.31.18:4828: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Jul 26 22:35:10.854431573 (58h59m42.044730153s) [seg_pos={0 0}, upload=56565]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Jul 26 22:35:10.969025541 (58h59m42.159324121s) [upload=56565]: Done uploading segments... Jul 26 22:35:10.969054726 (58h59m42.159353292s) [upload=56565]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 24.96258465s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................X.X.........................X.............] duration: 2.251355692s FAIL (2025-07-26 22:35:26 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.251378251s core uplink us1 vivint
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 7 out of 60 is failed (threshold=3) [....................................X.......X..X...X.XX...X.] duration: 4.301570663s FAIL (2025-07-26 22:35:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.301597088s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 7 out of 60 is failed (threshold=3) [.....................................X.......X..X...X.XX...X] duration: 1m0.000532297s FAIL (2025-07-26 22:35:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.00055555s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 17 out of 60 is failed (threshold=3) [.........................X.............XX...XXXXX.XXXX.XXXXX] duration: 1m0.117227893s FAIL (2025-07-26 22:35:10 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 600 (58h58m42.315297198s) [upload=56565]: Uploading segments... Jul 26 22:34:11.125202148 (58h58m42.315500741s) [upload=56565]: Got next segment. Inline: false Jul 26 22:34:11.125253419 (58h58m42.31555201s) [upload=56565]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:34:11.125260194 (58h58m42.315558785s) [upload=56565]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:34:11.322095960 (58h58m42.51239454s) [seg_pos={0 0}, upload=56565]: Begin upload segment... Jul 26 22:34:11.322152447 (58h58m42.512451017s) [seg_pos={0 0}, upload=56565]: Done begin upload segment. Jul 26 22:34:11.322180304 (58h58m42.51247889s) [noise=true , address=77.90.31.16:4751 , piece_id=LR2DDIJUVM2BXIAJKD3LJFKABHQTQ42Y3RNAQYRWKPKXQY3A4C4A, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.322195626 (58h58m42.512494215s) [noise=true , address=64.49.13.170:4718 , piece_id=ETDQXE55JNTEUNXSS3CJK7DUSRYPGUCFI2F6H4A3UZURBKEZ5NXQ, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.322587864 (58h58m42.512886453s) [noise=true , address=79.127.205.245:4251 , piece_id=HXJ4SYW37W6NLZP2MQLO22JCK6TEGTRIPFYNNUFXVGESZFBWYMUA, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.322814415 (58h58m42.513113001s) [noise=true , address=156.146.43.225:4449 , piece_id=R346CJ2RFMJZLYYD7TGJUYZG4F4QBOCAN3VK62JS3VY6XV4R3KFA, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.322920678 (58h58m42.513219267s) [noise=true , address=77.90.31.19:4867 , piece_id=25JAFIHTYBGPUT5JFBA2WTXOQYB2RP5DMTXKEYKJV3WZ7OQXBSNQ, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.323255183 (58h58m42.513553765s) [noise=true , address=77.90.31.18:4828 , piece_id=LTMMF2PK7F5VJOX4FGY5SA4HTN3OBSEPDXP43YRDIXIVWBN2ZGSA, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.324172222 (58h58m42.514470811s) [noise=true , address=79.127.219.38:4006 , piece_id=P5CGL67CZK4I5XZHYV3MLMCSSB3A3OE7MRN5Y2BOQZBQAJ6QOO3A, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.324541666 (58h58m42.514840249s) [noise=true , address=79.127.220.97:4361 , piece_id=TPIJYKAADUSKUPBGVPXSBIAISV7CZYXTJTFE2XMAVFAWOTEJM2RA, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.324857179 (58h58m42.515155771s) [noise=true , address=64.49.13.138:4482 , piece_id=M7PX5EAZLT5R6YUPADHQHPHTS7JTDLA2Q24YT3DGWDJB44E4CIJQ, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.325289211 (58h58m42.515587802s) [noise=true , address=79.127.205.246:4286 , piece_id=EU4B36TTO4LNKPMPBJE3BSY4VIAZD6U7T7GJA7PU7JNPIDGDUP6Q, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.325665458 (58h58m42.515964043s) [noise=true , address=79.127.205.234:4603 , piece_id=FZ3SCGNCEGBJ63EYLLZ2ZCIVCEO2GTRAG437HPCW6WHZWWLZTI5Q, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.326231548 (58h58m42.51653015s) [noise=true , address=45.91.116.11:4146 , piece_id=ONZDCEH3EL4U32I7ZLX7WNIGDS6LBJKMQ2BM5HPGM73PI4KW5SHQ, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.326620079 (58h58m42.516918661s) [noise=true , address=77.90.31.11:4560 , piece_id=XNKEYPHFRAG7576QHO6DLFKLX7H6ELTPMP6NLWMPAL2WILGBVUJA, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.326976175 (58h58m42.517274764s) [noise=true , address=79.127.201.215:4347 , piece_id=KEQBM35UNQ4MX7P75XTWPIVL23J6RUYGRTNNXWK4TP4YBD3XA7RA, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.327299882 (58h58m42.517598464s) [noise=true , address=64.49.13.140:4514 , piece_id=UMBGV3GMSPBFDANS7JTETCZI3LRSONWRFUUOIJUFB65UPSHFN5IA, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.327607309 (58h58m42.517905891s) [noise=true , address=79.127.219.51:4212 , piece_id=A7PRZXFXLDQJ27GAY5RBDCYPJPXUPINHHAJVL4A3XCKVPBT2ZC7Q, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.327959291 (58h58m42.51825787s) [noise=true , address=79.127.219.47:4069 , piece_id=G55S75BXMTYQ5RWIPWL2BT64ODX2P4E4OKNJ2KI6IMKU4NP4ZY7Q, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.328438540 (58h58m42.518737122s) [noise=true , address=104.36.82.34:4565 , piece_id=LFCJWIDOVUZLJMXTTQ7XKWL4E6ANKEKRGJYSA3QZJF2UIR67LDFA, seg_pos={0 0}, upload=56565]: Uploading piece... Jul 26 22:34:11.778832969 (58h58m42.969131559s) [noise=true , address=79.127.219.47:4069 , piece_id=G55S75BXMTYQ5RWIPWL2BT64ODX2P4E4OKNJ2KI6IMKU4NP4ZY7Q, seg_pos={0 0}, upload=56565]: Done uploading piece. err: ecclient: upload failed (node:18dK6RxbMxU4a46pW4YPiQW87xgpeoTb1ouqE9fzTdzTAFAa3W, address:79.127.219.47:4069): protocol: storage node overloaded, request limit: 2000 Jul 26 22:34:11.870845338 (58h58m43.061143929s) [noise=true , address=79.127.220.97:4361 , piece_id=TPIJYKAADUSKUPBGVPXSBIAISV7CZYXTJTFE2XMAVFAWOTEJM2RA, seg_pos={0 0}, upload=56565]: Done uploading piece. err: ecclient: upload failed (node:1ojv5zf5QshKyMqGypaEaYmGvzt96UM9kZgyGeFVPMFk8Twzyn, address:79.127.220.97:4361): protocol: storage node overloaded, request limit: 2000 Jul 26 22:34:11.911516886 (58h58m43.101815467s) [upload=56565]: Next returned nil segment Jul 26 22:34:11.911535331 (58h58m43.101833922s) [upload=56565]: Waiting for error group managing segments... Jul 26 22:34:12.023817302 (58h58m43.214115883s) [noise=true , address=64.49.13.138:4482 , piece_id=M7PX5EAZLT5R6YUPADHQHPHTS7JTDLA2Q24YT3DGWDJB44E4CIJQ, seg_pos={0 0}, upload=56565]: Done uploading piece. err: ecclient: upload failed (node:12joW2m4Uc87FSH65Cep1UKivUB1HG1119CQj14eWEKPcwaPsD5, address:64.49.13.138:4482): protocol: storage node overloaded, request limit: 2000 Jul 26 22:34:12.064253960 (58h58m43.25455264s) [noise=true , address=77.90.31.11:4560 , piece_id=XNKEYPHFRAG7576QHO6DLFKLX7H6ELTPMP6NLWMPAL2WILGBVUJA, seg_pos={0 0}, upload=56565]: Done uploading piece. err: <nil> Jul 26 22:34:12.085636218 (58h58m43.275934793s) [noise=true , address=79.127.219.38:4006 , piece_id=P5CGL67CZK4I5XZHYV3MLMCSSB3A3OE7MRN5Y2BOQZBQAJ6QOO3A, seg_pos={0 0}, upload=56565]: Done uploading piece. err: <nil> Jul 26 22:34:12.547021607 (58h58m43.737320188s) [noise=true , address=45.91.116.11:4146 , piece_id=ONZDCEH3EL4U32I7ZLX7WNIGDS6LBJKMQ2BM5HPGM73PI4KW5SHQ, seg_pos={0 0}, upload=56565]: Done uploading piece. err: <nil> Jul 26 22:34:12.608354679 (58h58m43.798653269s) [noise=true , address=64.49.13.140:4514 , piece_id=UMBGV3GMSPBFDANS7JTETCZI3LRSONWRFUUOIJUFB65UPSHFN5IA, seg_pos={0 0}, upload=56565]: Done uploading piece. err: <nil> Jul 26 22:34:12.628670275 (58h58m43.818968858s) [noise=true , address=79.127.205.246:4286 , piece_id=EU4B36TTO4LNKPMPBJE3BSY4VIAZD6U7T7GJA7PU7JNPIDGDUP6Q, seg_pos={0 0}, upload=56565]: Done uploading piece. err: <nil> Jul 26 22:34:12.653038575 (58h58m43.843337142s) [noise=true , address=79.127.205.245:4251 , piece_id=HXJ4SYW37W6NLZP2MQLO22JCK6TEGTRIPFYNNUFXVGESZFBWYMUA, seg_pos={0 0}, upload=56565]: Done uploading piece. err: <nil> Jul 26 22:34:12.680005647 (58h58m43.870304236s) [noise=true , address=64.49.13.170:4718 , piece_id=ETDQXE55JNTEUNXSS3CJK7DUSRYPGUCFI2F6H4A3UZURBKEZ5NXQ, seg_pos={0 0}, upload=56565]: Done uploading piece. err: <nil> Jul 26 22:34:12.987584251 (58h58m44.177882822s) [noise=true , address=79.127.205.234:4603 , piece_id=FZ3SCGNCEGBJ63EYLLZ2ZCIVCEO2GTRAG437HPCW6WHZWWLZTI5Q, seg_pos={0 0}, upload=56565]: Done uploading piece. err: <nil> Jul 26 22:34:13.171498895 (58h58m44.361797468s) [noise=true , address=156.146.43.225:4449 , piece_id=R346CJ2RFMJZLYYD7TGJUYZG4F4QBOCAN3VK62JS3VY6XV4R3KFA, seg_pos={0 0}, upload=56565]: Done uploading piece. err: <nil> Jul 26 22:34:14.205159349 (58h58m45.395457939s) [noise=true , address=79.127.219.51:4212 , piece_id=A7PRZXFXLDQJ27GAY5RBDCYPJPXUPINHHAJVL4A3XCKVPBT2ZC7Q, seg_pos={0 0}, upload=56565]: Done uploading piece. err: <nil> Jul 26 22:34:21.265563919 (58h58m52.455862501s) [noise=true , address=79.127.201.215:4347 , piece_id=KEQBM35UNQ4MX7P75XTWPIVL23J6RUYGRTNNXWK4TP4YBD3XA7RA, seg_pos={0 0}, upload=56565]: Done uploading piece. err: <nil> Jul 26 22:34:29.546831777 (58h59m0.737130356s) [noise=true , address=104.36.82.34:4565 , piece_id=LFCJWIDOVUZLJMXTTQ7XKWL4E6ANKEKRGJYSA3QZJF2UIR67LDFA, seg_pos={0 0}, upload=56565]: Done uploading piece. err: <nil> Jul 26 22:35:10.854110455 (58h59m42.044409045s) [noise=true , address=77.90.31.16:4751 , piece_id=LR2DDIJUVM2BXIAJKD3LJFKABHQTQ42Y3RNAQYRWKPKXQY3A4C4A, seg_pos={0 0}, upload=56565]: Done uploading piece. err: ecclient: upload failed (node:12Sj7nPQiL1EmwvnVCpRv1HEeDYFnknTXLNZTvhAUmHDcP2Wpuf, address:77.90.31.16:4751): protocol: rpc: dial tcp 77.90.31.16:4751: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 22:35:10.854232997 (58h59m42.044531588s) [noise=true , address=77.90.31.19:4867 , piece_id=25JAFIHTYBGPUT5JFBA2WTXOQYB2RP5DMTXKEYKJV3WZ7OQXBSNQ, seg_pos={0 0}, upload=56565]: Done uploading piece. err: ecclient: upload failed (node:1gczGyJMCtqUfaac5oy7o4CTwypVF8AZjrAAjoJweU3kBFbuXw, address:77.90.31.19:4867): protocol: rpc: dial tcp 77.90.31.19:4867: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 22:35:10.854402880 (58h59m42.044701459s) [noise=true , address=77.90.31.18:4828 , piece_id=LTMMF2PK7F5VJOX4FGY5SA4HTN3OBSEPDXP43YRDIXIVWBN2ZGSA, seg_pos={0 0}, upload=56565]: Done uploading piece. err: ecclient: upload failed (node:1jwTDXPShXfNdfMx3UXi22gQphBpFre1MhRsHyjq6xRz4wwTr9, address:77.90.31.18:4828): protocol: write tcp 139.59.144.22:39748->77.90.31.18:4828: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Jul 26 22:35:10.854431573 (58h59m42.044730153s) [seg_pos={0 0}, upload=56565]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Jul 26 22:35:10.969025541 (58h59m42.159324121s) [upload=56565]: Done uploading segments... Jul 26 22:35:10.969054726 (58h59m42.159353292s) [upload=56565]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 1m0.117262776s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [..X.X..X..XXX....X.X.X...............X......................] duration: 2.480053217s FAIL (2025-07-26 22:35:04 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 988 (58h42m45.061082575s) [upload=56308]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:18:14.018433272 (58h42m45.208731856s) [upload=56308]: Done uploading segments... Jul 26 22:18:14.018484438 (58h42m45.208783019s) [upload=56308]: Upload finished. err: <nil> Jul 26 22:18:57.520834389 (58h43m28.711132968s) [upload=56321]: Starting upload Jul 26 22:18:57.520934224 (58h43m28.711232794s) [upload=56321]: Done starting upload Jul 26 22:18:57.521309042 (58h43m28.711607621s) [upload=56321]: Uploading segments... Jul 26 22:18:57.521389036 (58h43m28.711687618s) [upload=56321]: Got next segment. Inline: false Jul 26 22:18:57.521430569 (58h43m28.711729148s) [upload=56321]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:18:57.521435990 (58h43m28.711734568s) [upload=56321]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:18:57.634354903 (58h43m28.824653493s) [seg_pos={0 0}, upload=56321]: Begin upload segment... Jul 26 22:18:57.634433002 (58h43m28.824731593s) [seg_pos={0 0}, upload=56321]: Done begin upload segment. Jul 26 22:18:57.634466006 (58h43m28.824764596s) [noise=true , address=104.36.82.12:4150 , piece_id=4PVRPBKTUNFKUWDJDTXYCIXDKONWG5OPQVB2M6KPKK66SMCYYHGA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.634507350 (58h43m28.82480593s) [noise=true , address=45.91.116.6:4973 , piece_id=S5BZKUNC2KNOVQMFRHSXOZNZZG6BUBK57ECM6W3S4YKN55KPOSFQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635011032 (58h43m28.825309612s) [noise=true , address=45.91.116.4:4938 , piece_id=INUA3CJT7KOAKKRYSBLTNXLX3MKWYYAFZ45BMUOZ6Y6I3B2EHHCQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635154508 (58h43m28.825453097s) [noise=true , address=45.91.116.8:4047 , piece_id=YTLRWKOOJ4XIGA3SKHFT2K66R57LBTW7OOVZSVJMCCV3YISPGWLA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635425419 (58h43m28.825724018s) [noise=true , address=45.91.116.8:4048 , piece_id=BF2LQK3BFGLPNCTDWB2M6HLRBSYBCXLR2AO7CMPERHT4DIBNKC4Q, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635823171 (58h43m28.826121762s) [noise=true , address=77.90.31.12:4590 , piece_id=PZSNEJOE623OVLNPZSRVVTTAGNHGRD7JCBUXCB3FXOMUE4CQNLFA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635848858 (58h43m28.826147438s) [noise=true , address=45.91.116.12:4227 , piece_id=FICCSE6IJQKUGYKTS3JHH4G65CPBOGDSPVIYXT25NR2GKRZQLOXA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.636411603 (58h43m28.826710186s) [noise=true , address=104.36.82.7:4909 , piece_id=HOFM67WVGW5QXUA65MEPQLJF7EV25AXBG4GNMREFYF2RNRGH2XWQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.636540327 (58h43m28.826838918s) [noise=true , address=77.90.31.11:4553 , piece_id=BP6FPURAP7D6RT34G72KZ32UY76QKXDG464WKNDPNUNOD6WS3KFQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.636837624 (58h43m28.827136209s) [noise=true , address=45.91.116.10:4081 , piece_id=2R44TDAMR2O3CNYFRZXOCSMG2ISUWMIQPVDWN3V2ECZWW4JUHNMA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.637107959 (58h43m28.827406542s) [noise=true , address=45.91.116.13:4154 , piece_id=PNBIV7A27TASCKR2CEQT7NJ5ZTG3NUQ2PLXJH7R47BFQQ53FLKUQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.637209450 (58h43m28.827508033s) [noise=true , address=77.90.31.21:4877 , piece_id=RCTFN3VAXG26GTA6GFQDQDIBAHJPOSI3QUSLLKO5APYYA57BSBRQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.637657116 (58h43m28.827955704s) [noise=true , address=45.91.116.7:4010 , piece_id=DZSLTEUQELXNHSH57IUZ6PRMOS56GR5TCWNAEEY7Y6YJ7KNDTRJA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.638167018 (58h43m28.828465602s) [noise=true , address=104.36.82.3:4767 , piece_id=IWRMBBOMGH7ZV3JA326TJ4SWHLEGA2T6CDLUDKJ4RFZBUZMTA3MQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.638722599 (58h43m28.829021185s) [noise=true , address=45.91.116.7:4009 , piece_id=2JHIJGUSNOSVSGOSGHHIYA2RNVLRS2P7MIKYHLSX4PTQOWNZKKGQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.639240448 (58h43m28.82953902s) [noise=true , address=104.36.82.11:4114 , piece_id=ZUBONJLMV4QRTG7DLIHWZICNNQ5OOF3HAPN26Q5AMK4TY4OYTXNA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:58.247247885 (58h43m29.437546466s) [upload=56321]: Next returned nil segment Jul 26 22:18:58.247273951 (58h43m29.437572537s) [upload=56321]: Waiting for error group managing segments... Jul 26 22:18:58.383663350 (58h43m29.573961931s) [noise=true , address=45.91.116.13:4154 , piece_id=PNBIV7A27TASCKR2CEQT7NJ5ZTG3NUQ2PLXJH7R47BFQQ53FLKUQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.384857324 (58h43m29.575155903s) [noise=true , address=45.91.116.8:4047 , piece_id=YTLRWKOOJ4XIGA3SKHFT2K66R57LBTW7OOVZSVJMCCV3YISPGWLA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.384981909 (58h43m29.575280495s) [noise=true , address=45.91.116.6:4973 , piece_id=S5BZKUNC2KNOVQMFRHSXOZNZZG6BUBK57ECM6W3S4YKN55KPOSFQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.387318149 (58h43m29.577616722s) [noise=true , address=45.91.116.7:4009 , piece_id=2JHIJGUSNOSVSGOSGHHIYA2RNVLRS2P7MIKYHLSX4PTQOWNZKKGQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.395512660 (58h43m29.585811243s) [noise=true , address=45.91.116.12:4227 , piece_id=FICCSE6IJQKUGYKTS3JHH4G65CPBOGDSPVIYXT25NR2GKRZQLOXA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.401895917 (58h43m29.592194499s) [noise=true , address=45.91.116.10:4081 , piece_id=2R44TDAMR2O3CNYFRZXOCSMG2ISUWMIQPVDWN3V2ECZWW4JUHNMA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.402135621 (58h43m29.59243419s) [noise=true , address=77.90.31.11:4553 , piece_id=BP6FPURAP7D6RT34G72KZ32UY76QKXDG464WKNDPNUNOD6WS3KFQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.403028120 (58h43m29.593326693s) [noise=true , address=45.91.116.8:4048 , piece_id=BF2LQK3BFGLPNCTDWB2M6HLRBSYBCXLR2AO7CMPERHT4DIBNKC4Q, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.404431307 (58h43m29.594729891s) [noise=true , address=45.91.116.4:4938 , piece_id=INUA3CJT7KOAKKRYSBLTNXLX3MKWYYAFZ45BMUOZ6Y6I3B2EHHCQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.409536042 (58h43m29.599834622s) [noise=true , address=45.91.116.7:4010 , piece_id=DZSLTEUQELXNHSH57IUZ6PRMOS56GR5TCWNAEEY7Y6YJ7KNDTRJA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:59.425708084 (58h43m30.616006681s) [noise=true , address=77.90.31.21:4877 , piece_id=RCTFN3VAXG26GTA6GFQDQDIBAHJPOSI3QUSLLKO5APYYA57BSBRQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:19:00.437139003 (58h43m31.627437582s) [noise=true , address=77.90.31.12:4590 , piece_id=PZSNEJOE623OVLNPZSRVVTTAGNHGRD7JCBUXCB3FXOMUE4CQNLFA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:19:27.246703337 (58h43m58.437001916s) [noise=true , address=104.36.82.7:4909 , piece_id=HOFM67WVGW5QXUA65MEPQLJF7EV25AXBG4GNMREFYF2RNRGH2XWQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: write tcp 139.59.144.22:57728->104.36.82.7:4909: use of closed network connection; write tcp 139.59.144.22:57728->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57728->104.36.82.7:4909: use of closed network connection Jul 26 22:19:27.246791447 (58h43m58.437090027s) [noise=true , address=104.36.82.11:4114 , piece_id=ZUBONJLMV4QRTG7DLIHWZICNNQ5OOF3HAPN26Q5AMK4TY4OYTXNA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 139.59.144.22:44152->104.36.82.11:4114: use of closed network connection; write tcp 139.59.144.22:44152->104.36.82.11:4114: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44152->104.36.82.11:4114: use of closed network connection Jul 26 22:19:27.246848250 (58h43m58.437146829s) [noise=true , address=104.36.82.3:4767 , piece_id=IWRMBBOMGH7ZV3JA326TJ4SWHLEGA2T6CDLUDKJ4RFZBUZMTA3MQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:52976->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:52976->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52976->104.36.82.3:4767: use of closed network connection Jul 26 22:19:27.246885606 (58h43m58.4371843s) [noise=true , address=104.36.82.12:4150 , piece_id=4PVRPBKTUNFKUWDJDTXYCIXDKONWG5OPQVB2M6KPKK66SMCYYHGA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:104.36.82.12:4150): protocol: write tcp 139.59.144.22:43452->104.36.82.12:4150: use of closed network connection; write tcp 139.59.144.22:43452->104.36.82.12:4150: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43452->104.36.82.12:4150: use of closed network connection Jul 26 22:19:27.246926689 (58h43m58.437225269s) [seg_pos={0 0}, upload=56321]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 22:19:27.354450344 (58h43m58.544748918s) [upload=56321]: Done uploading segments... Jul 26 22:19:27.354479247 (58h43m58.544777827s) [upload=56321]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 2.480161982s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 1.023159501s FAIL (2025-07-26 22:34:59 UTC) orbiter warning: digest mismatch: after b0275bfe409329e90caa68828ca9cf99d935619c40adf20794b6b4a165f10dac != before 689768e222479cddedfe8605c20bdb7052e22d65868c907a97a7972f7764c1dd Duration: 1.023185543s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [X..................X.X.........................X............] duration: 2.29119703s FAIL (2025-07-26 22:34:42 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.291230965s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [...X.X..X..XXX....X.X.X...............X.....................] duration: 2.287923201s FAIL (2025-07-26 22:34:40 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 988 (58h42m45.061082575s) [upload=56308]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:18:14.018433272 (58h42m45.208731856s) [upload=56308]: Done uploading segments... Jul 26 22:18:14.018484438 (58h42m45.208783019s) [upload=56308]: Upload finished. err: <nil> Jul 26 22:18:57.520834389 (58h43m28.711132968s) [upload=56321]: Starting upload Jul 26 22:18:57.520934224 (58h43m28.711232794s) [upload=56321]: Done starting upload Jul 26 22:18:57.521309042 (58h43m28.711607621s) [upload=56321]: Uploading segments... Jul 26 22:18:57.521389036 (58h43m28.711687618s) [upload=56321]: Got next segment. Inline: false Jul 26 22:18:57.521430569 (58h43m28.711729148s) [upload=56321]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:18:57.521435990 (58h43m28.711734568s) [upload=56321]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:18:57.634354903 (58h43m28.824653493s) [seg_pos={0 0}, upload=56321]: Begin upload segment... Jul 26 22:18:57.634433002 (58h43m28.824731593s) [seg_pos={0 0}, upload=56321]: Done begin upload segment. Jul 26 22:18:57.634466006 (58h43m28.824764596s) [noise=true , address=104.36.82.12:4150 , piece_id=4PVRPBKTUNFKUWDJDTXYCIXDKONWG5OPQVB2M6KPKK66SMCYYHGA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.634507350 (58h43m28.82480593s) [noise=true , address=45.91.116.6:4973 , piece_id=S5BZKUNC2KNOVQMFRHSXOZNZZG6BUBK57ECM6W3S4YKN55KPOSFQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635011032 (58h43m28.825309612s) [noise=true , address=45.91.116.4:4938 , piece_id=INUA3CJT7KOAKKRYSBLTNXLX3MKWYYAFZ45BMUOZ6Y6I3B2EHHCQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635154508 (58h43m28.825453097s) [noise=true , address=45.91.116.8:4047 , piece_id=YTLRWKOOJ4XIGA3SKHFT2K66R57LBTW7OOVZSVJMCCV3YISPGWLA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635425419 (58h43m28.825724018s) [noise=true , address=45.91.116.8:4048 , piece_id=BF2LQK3BFGLPNCTDWB2M6HLRBSYBCXLR2AO7CMPERHT4DIBNKC4Q, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635823171 (58h43m28.826121762s) [noise=true , address=77.90.31.12:4590 , piece_id=PZSNEJOE623OVLNPZSRVVTTAGNHGRD7JCBUXCB3FXOMUE4CQNLFA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635848858 (58h43m28.826147438s) [noise=true , address=45.91.116.12:4227 , piece_id=FICCSE6IJQKUGYKTS3JHH4G65CPBOGDSPVIYXT25NR2GKRZQLOXA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.636411603 (58h43m28.826710186s) [noise=true , address=104.36.82.7:4909 , piece_id=HOFM67WVGW5QXUA65MEPQLJF7EV25AXBG4GNMREFYF2RNRGH2XWQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.636540327 (58h43m28.826838918s) [noise=true , address=77.90.31.11:4553 , piece_id=BP6FPURAP7D6RT34G72KZ32UY76QKXDG464WKNDPNUNOD6WS3KFQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.636837624 (58h43m28.827136209s) [noise=true , address=45.91.116.10:4081 , piece_id=2R44TDAMR2O3CNYFRZXOCSMG2ISUWMIQPVDWN3V2ECZWW4JUHNMA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.637107959 (58h43m28.827406542s) [noise=true , address=45.91.116.13:4154 , piece_id=PNBIV7A27TASCKR2CEQT7NJ5ZTG3NUQ2PLXJH7R47BFQQ53FLKUQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.637209450 (58h43m28.827508033s) [noise=true , address=77.90.31.21:4877 , piece_id=RCTFN3VAXG26GTA6GFQDQDIBAHJPOSI3QUSLLKO5APYYA57BSBRQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.637657116 (58h43m28.827955704s) [noise=true , address=45.91.116.7:4010 , piece_id=DZSLTEUQELXNHSH57IUZ6PRMOS56GR5TCWNAEEY7Y6YJ7KNDTRJA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.638167018 (58h43m28.828465602s) [noise=true , address=104.36.82.3:4767 , piece_id=IWRMBBOMGH7ZV3JA326TJ4SWHLEGA2T6CDLUDKJ4RFZBUZMTA3MQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.638722599 (58h43m28.829021185s) [noise=true , address=45.91.116.7:4009 , piece_id=2JHIJGUSNOSVSGOSGHHIYA2RNVLRS2P7MIKYHLSX4PTQOWNZKKGQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.639240448 (58h43m28.82953902s) [noise=true , address=104.36.82.11:4114 , piece_id=ZUBONJLMV4QRTG7DLIHWZICNNQ5OOF3HAPN26Q5AMK4TY4OYTXNA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:58.247247885 (58h43m29.437546466s) [upload=56321]: Next returned nil segment Jul 26 22:18:58.247273951 (58h43m29.437572537s) [upload=56321]: Waiting for error group managing segments... Jul 26 22:18:58.383663350 (58h43m29.573961931s) [noise=true , address=45.91.116.13:4154 , piece_id=PNBIV7A27TASCKR2CEQT7NJ5ZTG3NUQ2PLXJH7R47BFQQ53FLKUQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.384857324 (58h43m29.575155903s) [noise=true , address=45.91.116.8:4047 , piece_id=YTLRWKOOJ4XIGA3SKHFT2K66R57LBTW7OOVZSVJMCCV3YISPGWLA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.384981909 (58h43m29.575280495s) [noise=true , address=45.91.116.6:4973 , piece_id=S5BZKUNC2KNOVQMFRHSXOZNZZG6BUBK57ECM6W3S4YKN55KPOSFQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.387318149 (58h43m29.577616722s) [noise=true , address=45.91.116.7:4009 , piece_id=2JHIJGUSNOSVSGOSGHHIYA2RNVLRS2P7MIKYHLSX4PTQOWNZKKGQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.395512660 (58h43m29.585811243s) [noise=true , address=45.91.116.12:4227 , piece_id=FICCSE6IJQKUGYKTS3JHH4G65CPBOGDSPVIYXT25NR2GKRZQLOXA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.401895917 (58h43m29.592194499s) [noise=true , address=45.91.116.10:4081 , piece_id=2R44TDAMR2O3CNYFRZXOCSMG2ISUWMIQPVDWN3V2ECZWW4JUHNMA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.402135621 (58h43m29.59243419s) [noise=true , address=77.90.31.11:4553 , piece_id=BP6FPURAP7D6RT34G72KZ32UY76QKXDG464WKNDPNUNOD6WS3KFQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.403028120 (58h43m29.593326693s) [noise=true , address=45.91.116.8:4048 , piece_id=BF2LQK3BFGLPNCTDWB2M6HLRBSYBCXLR2AO7CMPERHT4DIBNKC4Q, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.404431307 (58h43m29.594729891s) [noise=true , address=45.91.116.4:4938 , piece_id=INUA3CJT7KOAKKRYSBLTNXLX3MKWYYAFZ45BMUOZ6Y6I3B2EHHCQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.409536042 (58h43m29.599834622s) [noise=true , address=45.91.116.7:4010 , piece_id=DZSLTEUQELXNHSH57IUZ6PRMOS56GR5TCWNAEEY7Y6YJ7KNDTRJA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:59.425708084 (58h43m30.616006681s) [noise=true , address=77.90.31.21:4877 , piece_id=RCTFN3VAXG26GTA6GFQDQDIBAHJPOSI3QUSLLKO5APYYA57BSBRQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:19:00.437139003 (58h43m31.627437582s) [noise=true , address=77.90.31.12:4590 , piece_id=PZSNEJOE623OVLNPZSRVVTTAGNHGRD7JCBUXCB3FXOMUE4CQNLFA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:19:27.246703337 (58h43m58.437001916s) [noise=true , address=104.36.82.7:4909 , piece_id=HOFM67WVGW5QXUA65MEPQLJF7EV25AXBG4GNMREFYF2RNRGH2XWQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: write tcp 139.59.144.22:57728->104.36.82.7:4909: use of closed network connection; write tcp 139.59.144.22:57728->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57728->104.36.82.7:4909: use of closed network connection Jul 26 22:19:27.246791447 (58h43m58.437090027s) [noise=true , address=104.36.82.11:4114 , piece_id=ZUBONJLMV4QRTG7DLIHWZICNNQ5OOF3HAPN26Q5AMK4TY4OYTXNA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 139.59.144.22:44152->104.36.82.11:4114: use of closed network connection; write tcp 139.59.144.22:44152->104.36.82.11:4114: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44152->104.36.82.11:4114: use of closed network connection Jul 26 22:19:27.246848250 (58h43m58.437146829s) [noise=true , address=104.36.82.3:4767 , piece_id=IWRMBBOMGH7ZV3JA326TJ4SWHLEGA2T6CDLUDKJ4RFZBUZMTA3MQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:52976->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:52976->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52976->104.36.82.3:4767: use of closed network connection Jul 26 22:19:27.246885606 (58h43m58.4371843s) [noise=true , address=104.36.82.12:4150 , piece_id=4PVRPBKTUNFKUWDJDTXYCIXDKONWG5OPQVB2M6KPKK66SMCYYHGA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:104.36.82.12:4150): protocol: write tcp 139.59.144.22:43452->104.36.82.12:4150: use of closed network connection; write tcp 139.59.144.22:43452->104.36.82.12:4150: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43452->104.36.82.12:4150: use of closed network connection Jul 26 22:19:27.246926689 (58h43m58.437225269s) [seg_pos={0 0}, upload=56321]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 22:19:27.354450344 (58h43m58.544748918s) [upload=56321]: Done uploading segments... Jul 26 22:19:27.354479247 (58h43m58.544777827s) [upload=56321]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 2.287958381s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.X..................X.X.........................X...........] duration: 2.275733009s FAIL (2025-07-26 22:34:12 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.275762789s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 848.243674ms FAIL (2025-07-26 22:34:06 UTC) orbiter warning: digest mismatch: after b0275bfe409329e90caa68828ca9cf99d935619c40adf20794b6b4a165f10dac != before 689768e222479cddedfe8605c20bdb7052e22d65868c907a97a7972f7764c1dd Duration: 848.273604ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 16 out of 60 is failed (threshold=3) [..........................X.............XX...XXXXX.XXXX.XXXX] duration: 1m0.119719833s FAIL (2025-07-26 22:34:01 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 45]: Starting upload Jul 26 22:33:02.090916963 (58h57m33.281215548s) [upload=56545]: Done starting upload Jul 26 22:33:02.091086873 (58h57m33.281385456s) [upload=56545]: Uploading segments... Jul 26 22:33:02.091301464 (58h57m33.281600046s) [upload=56545]: Got next segment. Inline: false Jul 26 22:33:02.091383516 (58h57m33.281682099s) [upload=56545]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:33:02.091390367 (58h57m33.281688951s) [upload=56545]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:33:02.286705331 (58h57m33.477003903s) [seg_pos={0 0}, upload=56545]: Begin upload segment... Jul 26 22:33:02.286763651 (58h57m33.477062225s) [seg_pos={0 0}, upload=56545]: Done begin upload segment. Jul 26 22:33:02.286828154 (58h57m33.47712675s) [noise=true , address=45.91.116.11:4146 , piece_id=Z5FLZZWYO23RTHCVDWCFMUDJICSPBOFYF4UM6E3KBYK3LDE7WBMA, seg_pos={0 0}, upload=56545]: Uploading piece... Jul 26 22:33:02.286842472 (58h57m33.477141053s) [noise=true , address=79.127.220.101:4072 , piece_id=3G6GB6N3HYWCHV7O37S4HKIGI55I5Y3MPIKTRNIH4WB2MX343AXQ, seg_pos={0 0}, upload=56545]: Uploading piece... Jul 26 22:33:02.287334554 (58h57m33.477633141s) [noise=true , address=77.90.31.13:4644 , piece_id=OJ6TQGM2T232UUQKN3KA3VYZDE5RFKDSXSTPVCHYYSIIA573HOYA, seg_pos={0 0}, upload=56545]: Uploading piece... Jul 26 22:33:02.287642395 (58h57m33.477941025s) [noise=true , address=77.90.31.12:4593 , piece_id=XHXS6ZCCBF2ZHWQO2MWBGVBHDXV7TBOQLO4WXSUDMWOTVAM6KW4A, seg_pos={0 0}, upload=56545]: Uploading piece... Jul 26 22:33:02.287793693 (58h57m33.478092276s) [noise=true , address=45.91.116.14:4206 , piece_id=IDDOGBGGEKNTBUW76XUX2WLFCUXWWS7R4BXZM734UYDXCWUMLRGQ, seg_pos={0 0}, upload=56545]: Uploading piece... Jul 26 22:33:02.288236011 (58h57m33.478534596s) [noise=true , address=79.127.205.246:4260 , piece_id=YYSI3OV366YTEJR2IO26G462CBPTQFTUM5K2SSPXTJA44UCJTODQ, seg_pos={0 0}, upload=56545]: Uploading piece... Jul 26 22:33:02.288594170 (58h57m33.478892758s) [noise=true , address=79.127.205.235:4690 , piece_id=4ZINZ6F7XV2RLHW3CHASANU46BNZSWZZ5T5UFRIGLBQ57DDIAXRA, seg_pos={0 0}, upload=56545]: Uploading piece... Jul 26 22:33:02.288950175 (58h57m33.479248767s) [noise=true , address=79.127.205.245:4246 , piece_id=V2LAL6E6LYWCZNQGVLP5GWSGMTJ2XKVOIKPNVPQ35OIY5U4YFXTA, seg_pos={0 0}, upload=56545]: Uploading piece... Jul 26 22:33:02.289242107 (58h57m33.479540697s) [noise=true , address=45.91.116.5:4310 , piece_id=IXD4CUBDSN7V2JRPEZMB5SULOWMEZC643QAYMZMEUTNXAKMZFM4Q, seg_pos={0 0}, upload=56545]: Uploading piece... Jul 26 22:33:02.289442014 (58h57m33.479740605s) [noise=true , address=45.91.116.9:4269 , piece_id=MDU6T534DEIUN2P3AA5RHUKJALWLOPQJQPVLHQZPKCTQ737F4UJQ, seg_pos={0 0}, upload=56545]: Uploading piece... Jul 26 22:33:02.289960810 (58h57m33.480259381s) [noise=true , address=79.127.219.51:4201 , piece_id=OE574WUBWMLPYALNV2XXUBZI33DIW67FXMVVALD2P5UTWZYMHMYQ, seg_pos={0 0}, upload=56545]: Uploading piece... Jul 26 22:33:02.290132158 (58h57m33.480430887s) [noise=true , address=156.146.43.225:4449 , piece_id=LO4RFHPROMK4Q4FXVHPMVITXM2EAYEPNVJHHTAPLXIJGRITLVRFA, seg_pos={0 0}, upload=56545]: Uploading piece... Jul 26 22:33:02.291890895 (58h57m33.482189486s) [noise=true , address=79.127.219.47:4061 , piece_id=AIO3QEO4XQFJSDYYZNQH3QFABVB5UUD4YRYJDKFZMU6RX76C7IGA, seg_pos={0 0}, upload=56545]: Uploading piece... Jul 26 22:33:02.293271764 (58h57m33.483570357s) [noise=true , address=79.127.220.97:4361 , piece_id=WSPZRDBTVYL5GI2VJ5QZ2YBQQZ4V6MWJRTJYTRGHFC5H5TDH7EGQ, seg_pos={0 0}, upload=56545]: Uploading piece... Jul 26 22:33:02.295672521 (58h57m33.485971111s) [noise=true , address=77.90.31.15:4712 , piece_id=ZZLU4ICN5IW5SMJOFWZVVFGZGZDSNLPFXEZCP3SATDNOPVCEDPJA, seg_pos={0 0}, upload=56545]: Uploading piece... Jul 26 22:33:02.296739351 (58h57m33.487037935s) [noise=true , address=64.49.13.170:4718 , piece_id=DOOHRF3JQO5YRRZVVDFFNROYJIUAPCX47RMKDOMZESA3XWCPZP6Q, seg_pos={0 0}, upload=56545]: Uploading piece... Jul 26 22:33:02.299270707 (58h57m33.489569299s) [noise=true , address=104.36.82.13:4193 , piece_id=PV3X54LWLDE6V7JWWHPLUV6TEJUO7XIBFYDXDIA25KTUUIAHMXKQ, seg_pos={0 0}, upload=56545]: Uploading piece... Jul 26 22:33:02.302410233 (58h57m33.492708824s) [noise=true , address=79.127.219.38:4003 , piece_id=P5AJFHC4TFVBUCDKLU4QULYHNJIGHDKMPA7IG2DWL446U5QCYURA, seg_pos={0 0}, upload=56545]: Uploading piece... Jul 26 22:33:02.733318867 (58h57m33.923617453s) [noise=true , address=79.127.219.47:4061 , piece_id=AIO3QEO4XQFJSDYYZNQH3QFABVB5UUD4YRYJDKFZMU6RX76C7IGA, seg_pos={0 0}, upload=56545]: Done uploading piece. err: ecclient: upload failed (node:12f4AfVs2T4a8Yhd456WKdse7TZSjHBDULMR8myvo1ua9jiUrNu, address:79.127.219.47:4061): protocol: storage node overloaded, request limit: 2000 Jul 26 22:33:02.784836912 (58h57m33.975135492s) [noise=true , address=79.127.220.101:4072 , piece_id=3G6GB6N3HYWCHV7O37S4HKIGI55I5Y3MPIKTRNIH4WB2MX343AXQ, seg_pos={0 0}, upload=56545]: Done uploading piece. err: ecclient: upload failed (node:12bQXjK4bv5583RV4BtjhC8WTDJgz255Ptvs6f5kqvJu4SA99mc, address:79.127.220.101:4072): protocol: storage node overloaded, request limit: 2000 Jul 26 22:33:02.828776014 (58h57m34.019074594s) [noise=true , address=45.91.116.14:4206 , piece_id=IDDOGBGGEKNTBUW76XUX2WLFCUXWWS7R4BXZM734UYDXCWUMLRGQ, seg_pos={0 0}, upload=56545]: Done uploading piece. err: ecclient: upload failed (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK, address:45.91.116.14:4206): protocol: storage node overloaded, request limit: 2000 Jul 26 22:33:02.852310891 (58h57m34.04260948s) [upload=56545]: Next returned nil segment Jul 26 22:33:02.852332790 (58h57m34.042631379s) [upload=56545]: Waiting for error group managing segments... Jul 26 22:33:03.056172482 (58h57m34.246471067s) [noise=true , address=77.90.31.12:4593 , piece_id=XHXS6ZCCBF2ZHWQO2MWBGVBHDXV7TBOQLO4WXSUDMWOTVAM6KW4A, seg_pos={0 0}, upload=56545]: Done uploading piece. err: <nil> Jul 26 22:33:03.067106191 (58h57m34.257404762s) [noise=true , address=79.127.219.38:4003 , piece_id=P5AJFHC4TFVBUCDKLU4QULYHNJIGHDKMPA7IG2DWL446U5QCYURA, seg_pos={0 0}, upload=56545]: Done uploading piece. err: <nil> Jul 26 22:33:03.134445924 (58h57m34.324744517s) [noise=true , address=104.36.82.13:4193 , piece_id=PV3X54LWLDE6V7JWWHPLUV6TEJUO7XIBFYDXDIA25KTUUIAHMXKQ, seg_pos={0 0}, upload=56545]: Done uploading piece. err: <nil> Jul 26 22:33:03.514264769 (58h57m34.704563348s) [noise=true , address=79.127.205.235:4690 , piece_id=4ZINZ6F7XV2RLHW3CHASANU46BNZSWZZ5T5UFRIGLBQ57DDIAXRA, seg_pos={0 0}, upload=56545]: Done uploading piece. err: <nil> Jul 26 22:33:03.584087097 (58h57m34.774385662s) [noise=true , address=79.127.205.246:4260 , piece_id=YYSI3OV366YTEJR2IO26G462CBPTQFTUM5K2SSPXTJA44UCJTODQ, seg_pos={0 0}, upload=56545]: Done uploading piece. err: <nil> Jul 26 22:33:03.671267159 (58h57m34.861565731s) [noise=true , address=79.127.205.245:4246 , piece_id=V2LAL6E6LYWCZNQGVLP5GWSGMTJ2XKVOIKPNVPQ35OIY5U4YFXTA, seg_pos={0 0}, upload=56545]: Done uploading piece. err: <nil> Jul 26 22:33:03.780380119 (58h57m34.970678718s) [noise=true , address=64.49.13.170:4718 , piece_id=DOOHRF3JQO5YRRZVVDFFNROYJIUAPCX47RMKDOMZESA3XWCPZP6Q, seg_pos={0 0}, upload=56545]: Done uploading piece. err: ecclient: upload failed (node:12AcLsSmVz9sRtuDLs47WoWYC66YSbr2aXBjzBdjayg22daQ2Hp, address:64.49.13.170:4718): protocol: storage node overloaded, request limit: 2000 Jul 26 22:33:04.396255478 (58h57m35.586554047s) [noise=true , address=79.127.219.51:4201 , piece_id=OE574WUBWMLPYALNV2XXUBZI33DIW67FXMVVALD2P5UTWZYMHMYQ, seg_pos={0 0}, upload=56545]: Done uploading piece. err: <nil> Jul 26 22:33:04.653590468 (58h57m35.843889048s) [noise=true , address=79.127.220.97:4361 , piece_id=WSPZRDBTVYL5GI2VJ5QZ2YBQQZ4V6MWJRTJYTRGHFC5H5TDH7EGQ, seg_pos={0 0}, upload=56545]: Done uploading piece. err: <nil> Jul 26 22:33:05.501698441 (58h57m36.691997014s) [noise=true , address=45.91.116.9:4269 , piece_id=MDU6T534DEIUN2P3AA5RHUKJALWLOPQJQPVLHQZPKCTQ737F4UJQ, seg_pos={0 0}, upload=56545]: Done uploading piece. err: <nil> Jul 26 22:33:07.255746375 (58h57m38.446044944s) [noise=true , address=156.146.43.225:4449 , piece_id=LO4RFHPROMK4Q4FXVHPMVITXM2EAYEPNVJHHTAPLXIJGRITLVRFA, seg_pos={0 0}, upload=56545]: Done uploading piece. err: <nil> Jul 26 22:33:08.519141391 (58h57m39.70943997s) [noise=true , address=45.91.116.5:4310 , piece_id=IXD4CUBDSN7V2JRPEZMB5SULOWMEZC643QAYMZMEUTNXAKMZFM4Q, seg_pos={0 0}, upload=56545]: Done uploading piece. err: ecclient: upload failed (node:127RMUc62xRq9RU5z3XfkZsq5sehmv3kP8o53tr9BxVcPE3GQCY, address:45.91.116.5:4310): protocol: storage node overloaded, request limit: 2000 Jul 26 22:33:09.174073901 (58h57m40.364372474s) [noise=true , address=45.91.116.11:4146 , piece_id=Z5FLZZWYO23RTHCVDWCFMUDJICSPBOFYF4UM6E3KBYK3LDE7WBMA, seg_pos={0 0}, upload=56545]: Done uploading piece. err: <nil> Jul 26 22:33:34.522061412 (58h58m5.712359991s) [noise=true , address=77.90.31.13:4644 , piece_id=OJ6TQGM2T232UUQKN3KA3VYZDE5RFKDSXSTPVCHYYSIIA573HOYA, seg_pos={0 0}, upload=56545]: Done uploading piece. err: <nil> Jul 26 22:34:01.816861976 (58h58m33.007160569s) [noise=true , address=77.90.31.15:4712 , piece_id=ZZLU4ICN5IW5SMJOFWZVVFGZGZDSNLPFXEZCP3SATDNOPVCEDPJA, seg_pos={0 0}, upload=56545]: Done uploading piece. err: ecclient: upload failed (node:1bfFfSGARVArNEgchuNzxf7j1QzNbq84fuac8TRfzzsuGGCeT8, address:77.90.31.15:4712): protocol: write tcp 139.59.144.22:54174->77.90.31.15:4712: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 22:34:01.816903663 (58h58m33.007202256s) [seg_pos={0 0}, upload=56545]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Jul 26 22:34:01.932102776 (58h58m33.122401388s) [upload=56545]: Done uploading segments... Jul 26 22:34:01.932168624 (58h58m33.122467218s) [upload=56545]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 1m0.119775498s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [......................................X.......X..X...X.XX...] duration: 9.296423021s FAIL (2025-07-26 22:33:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 9.296512495s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [....X.X..X..XXX....X.X.X...............X....................] duration: 2.280840659s FAIL (2025-07-26 22:33:56 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 988 (58h42m45.061082575s) [upload=56308]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:18:14.018433272 (58h42m45.208731856s) [upload=56308]: Done uploading segments... Jul 26 22:18:14.018484438 (58h42m45.208783019s) [upload=56308]: Upload finished. err: <nil> Jul 26 22:18:57.520834389 (58h43m28.711132968s) [upload=56321]: Starting upload Jul 26 22:18:57.520934224 (58h43m28.711232794s) [upload=56321]: Done starting upload Jul 26 22:18:57.521309042 (58h43m28.711607621s) [upload=56321]: Uploading segments... Jul 26 22:18:57.521389036 (58h43m28.711687618s) [upload=56321]: Got next segment. Inline: false Jul 26 22:18:57.521430569 (58h43m28.711729148s) [upload=56321]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:18:57.521435990 (58h43m28.711734568s) [upload=56321]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:18:57.634354903 (58h43m28.824653493s) [seg_pos={0 0}, upload=56321]: Begin upload segment... Jul 26 22:18:57.634433002 (58h43m28.824731593s) [seg_pos={0 0}, upload=56321]: Done begin upload segment. Jul 26 22:18:57.634466006 (58h43m28.824764596s) [noise=true , address=104.36.82.12:4150 , piece_id=4PVRPBKTUNFKUWDJDTXYCIXDKONWG5OPQVB2M6KPKK66SMCYYHGA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.634507350 (58h43m28.82480593s) [noise=true , address=45.91.116.6:4973 , piece_id=S5BZKUNC2KNOVQMFRHSXOZNZZG6BUBK57ECM6W3S4YKN55KPOSFQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635011032 (58h43m28.825309612s) [noise=true , address=45.91.116.4:4938 , piece_id=INUA3CJT7KOAKKRYSBLTNXLX3MKWYYAFZ45BMUOZ6Y6I3B2EHHCQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635154508 (58h43m28.825453097s) [noise=true , address=45.91.116.8:4047 , piece_id=YTLRWKOOJ4XIGA3SKHFT2K66R57LBTW7OOVZSVJMCCV3YISPGWLA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635425419 (58h43m28.825724018s) [noise=true , address=45.91.116.8:4048 , piece_id=BF2LQK3BFGLPNCTDWB2M6HLRBSYBCXLR2AO7CMPERHT4DIBNKC4Q, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635823171 (58h43m28.826121762s) [noise=true , address=77.90.31.12:4590 , piece_id=PZSNEJOE623OVLNPZSRVVTTAGNHGRD7JCBUXCB3FXOMUE4CQNLFA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.635848858 (58h43m28.826147438s) [noise=true , address=45.91.116.12:4227 , piece_id=FICCSE6IJQKUGYKTS3JHH4G65CPBOGDSPVIYXT25NR2GKRZQLOXA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.636411603 (58h43m28.826710186s) [noise=true , address=104.36.82.7:4909 , piece_id=HOFM67WVGW5QXUA65MEPQLJF7EV25AXBG4GNMREFYF2RNRGH2XWQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.636540327 (58h43m28.826838918s) [noise=true , address=77.90.31.11:4553 , piece_id=BP6FPURAP7D6RT34G72KZ32UY76QKXDG464WKNDPNUNOD6WS3KFQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.636837624 (58h43m28.827136209s) [noise=true , address=45.91.116.10:4081 , piece_id=2R44TDAMR2O3CNYFRZXOCSMG2ISUWMIQPVDWN3V2ECZWW4JUHNMA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.637107959 (58h43m28.827406542s) [noise=true , address=45.91.116.13:4154 , piece_id=PNBIV7A27TASCKR2CEQT7NJ5ZTG3NUQ2PLXJH7R47BFQQ53FLKUQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.637209450 (58h43m28.827508033s) [noise=true , address=77.90.31.21:4877 , piece_id=RCTFN3VAXG26GTA6GFQDQDIBAHJPOSI3QUSLLKO5APYYA57BSBRQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.637657116 (58h43m28.827955704s) [noise=true , address=45.91.116.7:4010 , piece_id=DZSLTEUQELXNHSH57IUZ6PRMOS56GR5TCWNAEEY7Y6YJ7KNDTRJA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.638167018 (58h43m28.828465602s) [noise=true , address=104.36.82.3:4767 , piece_id=IWRMBBOMGH7ZV3JA326TJ4SWHLEGA2T6CDLUDKJ4RFZBUZMTA3MQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.638722599 (58h43m28.829021185s) [noise=true , address=45.91.116.7:4009 , piece_id=2JHIJGUSNOSVSGOSGHHIYA2RNVLRS2P7MIKYHLSX4PTQOWNZKKGQ, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:57.639240448 (58h43m28.82953902s) [noise=true , address=104.36.82.11:4114 , piece_id=ZUBONJLMV4QRTG7DLIHWZICNNQ5OOF3HAPN26Q5AMK4TY4OYTXNA, seg_pos={0 0}, upload=56321]: Uploading piece... Jul 26 22:18:58.247247885 (58h43m29.437546466s) [upload=56321]: Next returned nil segment Jul 26 22:18:58.247273951 (58h43m29.437572537s) [upload=56321]: Waiting for error group managing segments... Jul 26 22:18:58.383663350 (58h43m29.573961931s) [noise=true , address=45.91.116.13:4154 , piece_id=PNBIV7A27TASCKR2CEQT7NJ5ZTG3NUQ2PLXJH7R47BFQQ53FLKUQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.384857324 (58h43m29.575155903s) [noise=true , address=45.91.116.8:4047 , piece_id=YTLRWKOOJ4XIGA3SKHFT2K66R57LBTW7OOVZSVJMCCV3YISPGWLA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.384981909 (58h43m29.575280495s) [noise=true , address=45.91.116.6:4973 , piece_id=S5BZKUNC2KNOVQMFRHSXOZNZZG6BUBK57ECM6W3S4YKN55KPOSFQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.387318149 (58h43m29.577616722s) [noise=true , address=45.91.116.7:4009 , piece_id=2JHIJGUSNOSVSGOSGHHIYA2RNVLRS2P7MIKYHLSX4PTQOWNZKKGQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.395512660 (58h43m29.585811243s) [noise=true , address=45.91.116.12:4227 , piece_id=FICCSE6IJQKUGYKTS3JHH4G65CPBOGDSPVIYXT25NR2GKRZQLOXA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.401895917 (58h43m29.592194499s) [noise=true , address=45.91.116.10:4081 , piece_id=2R44TDAMR2O3CNYFRZXOCSMG2ISUWMIQPVDWN3V2ECZWW4JUHNMA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.402135621 (58h43m29.59243419s) [noise=true , address=77.90.31.11:4553 , piece_id=BP6FPURAP7D6RT34G72KZ32UY76QKXDG464WKNDPNUNOD6WS3KFQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.403028120 (58h43m29.593326693s) [noise=true , address=45.91.116.8:4048 , piece_id=BF2LQK3BFGLPNCTDWB2M6HLRBSYBCXLR2AO7CMPERHT4DIBNKC4Q, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.404431307 (58h43m29.594729891s) [noise=true , address=45.91.116.4:4938 , piece_id=INUA3CJT7KOAKKRYSBLTNXLX3MKWYYAFZ45BMUOZ6Y6I3B2EHHCQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:58.409536042 (58h43m29.599834622s) [noise=true , address=45.91.116.7:4010 , piece_id=DZSLTEUQELXNHSH57IUZ6PRMOS56GR5TCWNAEEY7Y6YJ7KNDTRJA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:18:59.425708084 (58h43m30.616006681s) [noise=true , address=77.90.31.21:4877 , piece_id=RCTFN3VAXG26GTA6GFQDQDIBAHJPOSI3QUSLLKO5APYYA57BSBRQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:19:00.437139003 (58h43m31.627437582s) [noise=true , address=77.90.31.12:4590 , piece_id=PZSNEJOE623OVLNPZSRVVTTAGNHGRD7JCBUXCB3FXOMUE4CQNLFA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: <nil> Jul 26 22:19:27.246703337 (58h43m58.437001916s) [noise=true , address=104.36.82.7:4909 , piece_id=HOFM67WVGW5QXUA65MEPQLJF7EV25AXBG4GNMREFYF2RNRGH2XWQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: write tcp 139.59.144.22:57728->104.36.82.7:4909: use of closed network connection; write tcp 139.59.144.22:57728->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57728->104.36.82.7:4909: use of closed network connection Jul 26 22:19:27.246791447 (58h43m58.437090027s) [noise=true , address=104.36.82.11:4114 , piece_id=ZUBONJLMV4QRTG7DLIHWZICNNQ5OOF3HAPN26Q5AMK4TY4OYTXNA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 139.59.144.22:44152->104.36.82.11:4114: use of closed network connection; write tcp 139.59.144.22:44152->104.36.82.11:4114: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44152->104.36.82.11:4114: use of closed network connection Jul 26 22:19:27.246848250 (58h43m58.437146829s) [noise=true , address=104.36.82.3:4767 , piece_id=IWRMBBOMGH7ZV3JA326TJ4SWHLEGA2T6CDLUDKJ4RFZBUZMTA3MQ, seg_pos={0 0}, upload=56321]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:52976->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:52976->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52976->104.36.82.3:4767: use of closed network connection Jul 26 22:19:27.246885606 (58h43m58.4371843s) [noise=true , address=104.36.82.12:4150 , piece_id=4PVRPBKTUNFKUWDJDTXYCIXDKONWG5OPQVB2M6KPKK66SMCYYHGA, seg_pos={0 0}, upload=56321]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:104.36.82.12:4150): protocol: write tcp 139.59.144.22:43452->104.36.82.12:4150: use of closed network connection; write tcp 139.59.144.22:43452->104.36.82.12:4150: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43452->104.36.82.12:4150: use of closed network connection Jul 26 22:19:27.246926689 (58h43m58.437225269s) [seg_pos={0 0}, upload=56321]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 22:19:27.354450344 (58h43m58.544748918s) [upload=56321]: Done uploading segments... Jul 26 22:19:27.354479247 (58h43m58.544777827s) [upload=56321]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 2.280939651s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [X.X..................X.X.........................X..........] duration: 2.242654794s FAIL (2025-07-26 22:33:32 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.242696144s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 919.813253ms FAIL (2025-07-26 22:33:27 UTC) orbiter warning: digest mismatch: after b0275bfe409329e90caa68828ca9cf99d935619c40adf20794b6b4a165f10dac != before 689768e222479cddedfe8605c20bdb7052e22d65868c907a97a7972f7764c1dd Duration: 919.842839ms core uplink us1 vivint
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [.......................................X.......X..X...X.XX..] duration: 6.661118272s FAIL (2025-07-26 22:33:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.661145523s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)