Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...........................................X........X.......] duration: 979.631681ms FAIL (2025-08-08 00:54:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 263481 (15h4m20.280320332s) [upload=15511]: Done uploading segments... Aug 8 00:43:58.536281876 (15h4m20.28033873s) [upload=15511]: Upload finished. err: <nil> Aug 8 00:45:01.015818127 (15h5m22.759874977s) [upload=15529]: Starting upload Aug 8 00:45:01.015874109 (15h5m22.759930972s) [upload=15529]: Done starting upload Aug 8 00:45:01.015956182 (15h5m22.76001303s) [upload=15529]: Uploading segments... Aug 8 00:45:01.016043908 (15h5m22.760100776s) [upload=15529]: Got next segment. Inline: true Aug 8 00:45:01.016049919 (15h5m22.760106782s) [upload=15529]: Waiting for error group managing segments... Aug 8 00:45:01.016115031 (15h5m22.760171895s) [upload=15529]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:45:01.016122390 (15h5m22.760179257s) [upload=15529]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:45:01.016125863 (15h5m22.760182727s) [upload=15529]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:45:01.173239782 (15h5m22.91729665s) [upload=15529]: Done uploading segments... Aug 8 00:45:01.173279478 (15h5m22.917336328s) [upload=15529]: Upload finished. err: <nil> Aug 8 00:46:00.429072612 (15h6m22.173129463s) [upload=15546]: Starting upload Aug 8 00:46:00.429156391 (15h6m22.173213278s) [upload=15546]: Done starting upload Aug 8 00:46:00.429267021 (15h6m22.173323882s) [upload=15546]: Uploading segments... Aug 8 00:46:00.429320811 (15h6m22.173377677s) [upload=15546]: Got next segment. Inline: true Aug 8 00:46:00.429326919 (15h6m22.17338377s) [upload=15546]: Waiting for error group managing segments... Aug 8 00:46:00.429377405 (15h6m22.173434256s) [upload=15546]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:46:00.429383461 (15h6m22.173440311s) [upload=15546]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:46:00.429387017 (15h6m22.173443877s) [upload=15546]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:46:00.684470407 (15h6m22.428527258s) [upload=15546]: Done uploading segments... Aug 8 00:46:00.684501074 (15h6m22.42855792s) [upload=15546]: Upload finished. err: <nil> Aug 8 00:47:14.991362846 (15h7m36.735419697s) [upload=15569]: Starting upload Aug 8 00:47:14.991436891 (15h7m36.735493754s) [upload=15569]: Done starting upload Aug 8 00:47:14.991521124 (15h7m36.735577974s) [upload=15569]: Uploading segments... Aug 8 00:47:14.991582184 (15h7m36.735639054s) [upload=15569]: Got next segment. Inline: true Aug 8 00:47:14.991588085 (15h7m36.735644945s) [upload=15569]: Waiting for error group managing segments... Aug 8 00:47:14.991641941 (15h7m36.735698801s) [upload=15569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:47:14.991647402 (15h7m36.735704266s) [upload=15569]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:47:14.991651037 (15h7m36.735707897s) [upload=15569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:47:15.149817056 (15h7m36.893873921s) [upload=15569]: Done uploading segments... Aug 8 00:47:15.149854826 (15h7m36.893911673s) [upload=15569]: Upload finished. err: <nil> Aug 8 00:48:05.184694323 (15h8m26.92875119s) [upload=15581]: Starting upload Aug 8 00:48:05.184775560 (15h8m26.928832424s) [upload=15581]: Done starting upload Aug 8 00:48:05.184860193 (15h8m26.928917046s) [upload=15581]: Uploading segments... Aug 8 00:48:05.184919879 (15h8m26.92897675s) [upload=15581]: Got next segment. Inline: true Aug 8 00:48:05.184927810 (15h8m26.928984686s) [upload=15581]: Waiting for error group managing segments... Aug 8 00:48:05.184989718 (15h8m26.929046568s) [upload=15581]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:48:05.184997233 (15h8m26.929054096s) [upload=15581]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:48:05.185002973 (15h8m26.929059837s) [upload=15581]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:48:05.333512154 (15h8m27.077569026s) [upload=15581]: Done uploading segments... Aug 8 00:48:05.333535225 (15h8m27.077592084s) [upload=15581]: Upload finished. err: <nil> Aug 8 00:48:25.959034104 (15h8m47.703090965s) [upload=15588]: Starting upload Aug 8 00:48:25.959129631 (15h8m47.703186501s) [upload=15588]: Done starting upload Aug 8 00:48:25.959263036 (15h8m47.703319897s) [upload=15588]: Uploading segments... Aug 8 00:48:25.959321571 (15h8m47.703378451s) [upload=15588]: Got next segment. Inline: true Aug 8 00:48:25.959328075 (15h8m47.70338494s) [upload=15588]: Waiting for error group managing segments... Aug 8 00:48:25.959379365 (15h8m47.70343623s) [upload=15588]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:48:25.959385760 (15h8m47.703442625s) [upload=15588]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:48:25.959389437 (15h8m47.703446299s) [upload=15588]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:48:26.117426083 (15h8m47.861482953s) [upload=15588]: Done uploading segments... Aug 8 00:48:26.117449715 (15h8m47.861506585s) [upload=15588]: Upload finished. err: <nil> Aug 8 00:49:18.380712745 (15h9m40.124769598s) [upload=15601]: Starting upload Aug 8 00:49:18.380774191 (15h9m40.124831063s) [upload=15601]: Done starting upload Aug 8 00:49:18.380853700 (15h9m40.124910555s) [upload=15601]: Uploading segments... Aug 8 00:49:18.380935089 (15h9m40.124991961s) [upload=15601]: Got next segment. Inline: true Aug 8 00:49:18.380942228 (15h9m40.124999082s) [upload=15601]: Waiting for error group managing segments... Aug 8 00:49:18.380990837 (15h9m40.125047687s) [upload=15601]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:49:18.380996570 (15h9m40.125053423s) [upload=15601]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:49:18.381000556 (15h9m40.125057404s) [upload=15601]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:49:18.524510041 (15h9m40.26856689s) [upload=15601]: Done uploading segments... Aug 8 00:49:18.524532847 (15h9m40.268589697s) [upload=15601]: Upload finished. err: <nil> Aug 8 00:50:14.219213519 (15h10m35.96327037s) [upload=15618]: Starting upload Aug 8 00:50:14.219268923 (15h10m35.963325784s) [upload=15618]: Done starting upload Aug 8 00:50:14.219367379 (15h10m35.963424233s) [upload=15618]: Uploading segments... Aug 8 00:50:14.219578059 (15h10m35.963634929s) [upload=15618]: Got next segment. Inline: true Aug 8 00:50:14.219614810 (15h10m35.963671673s) [upload=15618]: Waiting for error group managing segments... Aug 8 00:50:14.219687157 (15h10m35.963744017s) [upload=15618]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:50:14.219693611 (15h10m35.963750471s) [upload=15618]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:50:14.219699847 (15h10m35.96375671s) [upload=15618]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:50:14.385631211 (15h10m36.129688064s) [upload=15618]: Done uploading segments... Aug 8 00:50:14.385650912 (15h10m36.129707762s) [upload=15618]: Upload finished. err: <nil> Aug 8 00:50:44.080806747 (15h11m5.824863598s) [upload=15628]: Starting upload Aug 8 00:50:44.080911523 (15h11m5.824968393s) [upload=15628]: Done starting upload Aug 8 00:50:44.081032828 (15h11m5.825089678s) [upload=15628]: Uploading segments... Aug 8 00:50:44.081102196 (15h11m5.825159064s) [upload=15628]: Got next segment. Inline: true Aug 8 00:50:44.081110880 (15h11m5.82516774s) [upload=15628]: Waiting for error group managing segments... Aug 8 00:50:44.081162262 (15h11m5.825219175s) [upload=15628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:50:44.081168840 (15h11m5.825225704s) [upload=15628]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:50:44.081172385 (15h11m5.825229249s) [upload=15628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:50:44.261318102 (15h11m6.005374965s) [upload=15628]: Done uploading segments... Aug 8 00:50:44.261339384 (15h11m6.005396236s) [upload=15628]: Upload finished. err: <nil> Aug 8 00:51:00.984479825 (15h11m22.728536675s) [upload=15634]: Starting upload Aug 8 00:51:00.984537080 (15h11m22.72859393s) [upload=15634]: Done starting upload Aug 8 00:51:00.984618934 (15h11m22.728675798s) [upload=15634]: Uploading segments... Aug 8 00:51:00.984689929 (15h11m22.728746817s) [upload=15634]: Got next segment. Inline: true Aug 8 00:51:00.984696447 (15h11m22.728753308s) [upload=15634]: Waiting for error group managing segments... Aug 8 00:51:00.984795333 (15h11m22.728852187s) [upload=15634]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:51:00.984801510 (15h11m22.72885837s) [upload=15634]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:51:00.984804953 (15h11m22.728861813s) [upload=15634]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:51:01.179830418 (15h11m22.923887268s) [upload=15634]: Done uploading segments... Aug 8 00:51:01.179852239 (15h11m22.923909093s) [upload=15634]: Upload finished. err: <nil> Aug 8 00:51:18.788113821 (15h11m40.532170668s) [upload=15636]: Starting upload Aug 8 00:51:18.788190945 (15h11m40.53224781s) [upload=15636]: Done starting upload Aug 8 00:51:18.788266338 (15h11m40.532323198s) [upload=15636]: Uploading segments... Aug 8 00:51:18.788334958 (15h11m40.532391824s) [upload=15636]: Got next segment. Inline: true Aug 8 00:51:18.788341211 (15h11m40.532398065s) [upload=15636]: Waiting for error group managing segments... Aug 8 00:51:18.788385583 (15h11m40.532442429s) [upload=15636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:51:18.788390526 (15h11m40.532447372s) [upload=15636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:51:18.788393111 (15h11m40.532449957s) [upload=15636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:51:18.876506251 (15h11m40.620563101s) [upload=15636]: Done uploading segments... Aug 8 00:51:18.876525344 (15h11m40.620582195s) [upload=15636]: Upload finished. err: metaclient: Too Many Requests Duration: 979.659658ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [....X...........................................X...........] duration: 2.692420133s FAIL (2025-08-08 00:54:25 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Done starting upload Aug 8 00:45:16.266121630 (15h5m38.01017848s) [upload=15533]: Uploading segments... Aug 8 00:45:16.266428624 (15h5m38.010485484s) [upload=15533]: Got next segment. Inline: false Aug 8 00:45:16.266497281 (15h5m38.010554191s) [upload=15533]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:45:16.266512043 (15h5m38.010568914s) [upload=15533]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:45:16.467813786 (15h5m38.211870647s) [seg_pos={0 0}, upload=15533]: Begin upload segment... Aug 8 00:45:16.467883230 (15h5m38.211940081s) [seg_pos={0 0}, upload=15533]: Done begin upload segment. Aug 8 00:45:16.467935054 (15h5m38.211991969s) [noise=true , address=104.36.82.60:4369 , piece_id=WVPSCPUKKSYS4T3JIEEVDTS7WSFYXNGVQA6QNLGO3BKC3EKEYJGQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.467945924 (15h5m38.212002789s) [noise=true , address=104.36.82.54:4192 , piece_id=FDJHGRVMHAFB5JZM4Z77MF3NHRU3T4FNQPUAOXGP5NRCM5Q7RVRA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468440400 (15h5m38.212497272s) [noise=true , address=104.36.82.34:4577 , piece_id=4PTCNUDROG6OXW5DBUEMBAGPEMEMVLIW5OXRP2LPC55J2PTK3PAA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468457544 (15h5m38.21251441s) [noise=true , address=104.36.82.44:4887 , piece_id=FC4PG4NZJHR35DZLGLFZ6EJZYA2MJWLIDPBYHZCNJKJI4OVX5G4Q, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468847577 (15h5m38.212904441s) [noise=true , address=104.36.82.62:4430 , piece_id=JGWQSS5KYYCBOITFMENDLSGK6XSDVOT7TMRGJBZ55KBXX54ENY6A, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468887696 (15h5m38.21294456s) [noise=true , address=79.127.205.233:4633 , piece_id=F4RDTCFPIWZS4U65QMU5YXWOZSPPDIDMLJ26G3DCN3YV2UMAEFWA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469228336 (15h5m38.2132852s) [noise=true , address=104.36.82.43:4846 , piece_id=73PPCVC3HGFJDIB63URF7XA6PC37VCYLBCOMTSSFNC5UYZR6M5YA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469268209 (15h5m38.213325082s) [noise=true , address=64.49.13.140:4515 , piece_id=AQTWEM35E7E6YUISEOQGTPTPKD6RDOMKXLK267UILGD77273OY4A, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469613163 (15h5m38.21367003s) [noise=true , address=104.36.82.16:4307 , piece_id=B7ZUFMTGVRPCNJSTBCXMQRZW2AGKPXM2LJRDLYZWPXQ4F5JJ75UQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469702757 (15h5m38.213759621s) [noise=true , address=104.36.82.9:4038 , piece_id=RVXFCLSREWSON62WSTYVISC52HN33VUN2BHJ7UR72MEBMTFKQV6A, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470036233 (15h5m38.214093098s) [noise=true , address=104.36.82.23:4464 , piece_id=4HNT67XCPCNCECLCE36RVAHTJ7VBAADPQ7VMC7IASAUPVVEKIVWQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470129321 (15h5m38.214186184s) [noise=true , address=45.91.116.6:4982 , piece_id=NV3WES7XAJBCNO5EQMWFK6SNUEYE63YKTQC5ZMRF7URP6SX3FXGA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470411978 (15h5m38.214468842s) [noise=true , address=104.36.82.23:4452 , piece_id=B4XGMOK4RCAIOV3VSJXFZ73O52E4WIRUGSPVTCCJ4XUGNJHUQWBA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470482224 (15h5m38.21453909s) [noise=true , address=104.36.82.14:4254 , piece_id=AIB65II65FK6M3EQHX6PVX27N4Q4BQKI5QGVP3PD6D34ZKXVCLZA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470792753 (15h5m38.214849603s) [noise=true , address=104.36.82.19:4405 , piece_id=SA2CPGDA75CYSUDYYZ6V4NKTG5AEIZC7SVYNL2OLBK3E5MSK5FQQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470877563 (15h5m38.214934486s) [noise=true , address=104.36.82.14:4243 , piece_id=HSR3LCZHIDLJSJ7WISDYOWGPLBFZ73MIXTR7MLFQ2TLUBYLRL4DQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.471502176 (15h5m38.215559147s) [noise=true , address=104.36.82.45:4910 , piece_id=OGS6W5AGDOACXIMZ4OH3GNQ43YUU6WOYJHKAHNMHJZBQPJUVVCRA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.471850592 (15h5m38.215907452s) [noise=true , address=79.127.201.215:4358 , piece_id=Z3PRSJIAQMKCL2ZU5TA7WZ2IBCVIXB2K5UJBMS3UCI5H6HDNTWBA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:17.050924903 (15h5m38.794981774s) [upload=15533]: Next returned nil segment Aug 8 00:45:17.051086092 (15h5m38.795142971s) [upload=15533]: Waiting for error group managing segments... Aug 8 00:45:17.225555422 (15h5m38.969612281s) [noise=true , address=104.36.82.62:4430 , piece_id=JGWQSS5KYYCBOITFMENDLSGK6XSDVOT7TMRGJBZ55KBXX54ENY6A, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.232285957 (15h5m38.97634281s) [noise=true , address=45.91.116.6:4982 , piece_id=NV3WES7XAJBCNO5EQMWFK6SNUEYE63YKTQC5ZMRF7URP6SX3FXGA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.239931943 (15h5m38.983988797s) [noise=true , address=104.36.82.45:4910 , piece_id=OGS6W5AGDOACXIMZ4OH3GNQ43YUU6WOYJHKAHNMHJZBQPJUVVCRA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.240442032 (15h5m38.984498886s) [noise=true , address=104.36.82.34:4577 , piece_id=4PTCNUDROG6OXW5DBUEMBAGPEMEMVLIW5OXRP2LPC55J2PTK3PAA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.280384119 (15h5m39.024440979s) [noise=true , address=104.36.82.19:4405 , piece_id=SA2CPGDA75CYSUDYYZ6V4NKTG5AEIZC7SVYNL2OLBK3E5MSK5FQQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.284571751 (15h5m39.028628621s) [noise=true , address=104.36.82.16:4307 , piece_id=B7ZUFMTGVRPCNJSTBCXMQRZW2AGKPXM2LJRDLYZWPXQ4F5JJ75UQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.295563563 (15h5m39.03962043s) [noise=true , address=104.36.82.14:4254 , piece_id=AIB65II65FK6M3EQHX6PVX27N4Q4BQKI5QGVP3PD6D34ZKXVCLZA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.367859627 (15h5m39.11191649s) [noise=true , address=104.36.82.44:4887 , piece_id=FC4PG4NZJHR35DZLGLFZ6EJZYA2MJWLIDPBYHZCNJKJI4OVX5G4Q, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.486413539 (15h5m39.2304704s) [noise=true , address=104.36.82.9:4038 , piece_id=RVXFCLSREWSON62WSTYVISC52HN33VUN2BHJ7UR72MEBMTFKQV6A, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.510192603 (15h5m39.254249451s) [noise=true , address=79.127.201.215:4358 , piece_id=Z3PRSJIAQMKCL2ZU5TA7WZ2IBCVIXB2K5UJBMS3UCI5H6HDNTWBA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.609092139 (15h5m39.353148988s) [noise=true , address=104.36.82.54:4192 , piece_id=FDJHGRVMHAFB5JZM4Z77MF3NHRU3T4FNQPUAOXGP5NRCM5Q7RVRA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.647464269 (15h5m39.391521122s) [noise=true , address=104.36.82.14:4243 , piece_id=HSR3LCZHIDLJSJ7WISDYOWGPLBFZ73MIXTR7MLFQ2TLUBYLRL4DQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.681479215 (15h5m39.425536078s) [noise=true , address=104.36.82.43:4846 , piece_id=73PPCVC3HGFJDIB63URF7XA6PC37VCYLBCOMTSSFNC5UYZR6M5YA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.697970794 (15h5m39.442027648s) [noise=true , address=104.36.82.60:4369 , piece_id=WVPSCPUKKSYS4T3JIEEVDTS7WSFYXNGVQA6QNLGO3BKC3EKEYJGQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.697997236 (15h5m39.442054151s) [seg_pos={0 0}, upload=15533]: Segment reached optimal threshold of 14 pieces. Aug 8 00:45:17.698203533 (15h5m39.442260383s) [noise=true , address=79.127.205.233:4633 , piece_id=F4RDTCFPIWZS4U65QMU5YXWOZSPPDIDMLJ26G3DCN3YV2UMAEFWA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EEYb4mCySvZ54qzmygCQ7hLyTGYpdNJMyrvPvHJjtfGos9usk): protocol: expected piece hash; context canceled; EOF Aug 8 00:45:17.698290533 (15h5m39.442347383s) [noise=true , address=64.49.13.140:4515 , piece_id=AQTWEM35E7E6YUISEOQGTPTPKD6RDOMKXLK267UILGD77273OY4A, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mY7sRFAeDLmKPqq8BgwnGtkdPTsDRR6soTBshsn4qhP7evARb): protocol: expected piece hash; context canceled; EOF Aug 8 00:45:17.698544485 (15h5m39.442601338s) [noise=true , address=104.36.82.23:4464 , piece_id=4HNT67XCPCNCECLCE36RVAHTJ7VBAADPQ7VMC7IASAUPVVEKIVWQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mLbGHntspYmsTVg8433HYSdAEMrXjGnDhmmvX1JmmgcxtdLgk): protocol: write tcp 139.59.144.22:39260->104.36.82.23:4464: use of closed network connection; write tcp 139.59.144.22:39260->104.36.82.23:4464: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39260->104.36.82.23:4464: use of closed network connection Aug 8 00:45:17.698701613 (15h5m39.442758474s) [noise=true , address=104.36.82.23:4452 , piece_id=B4XGMOK4RCAIOV3VSJXFZ73O52E4WIRUGSPVTCCJ4XUGNJHUQWBA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sAh1mLmzbtmc4o2FAZssW8Y7iDDNmNTCXdCS8xrH8PBVQqxG5): protocol: context canceled; context canceled Aug 8 00:45:17.698723379 (15h5m39.44278023s) [seg_pos={0 0}, upload=15533]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 00:45:17.698815435 (15h5m39.442872285s) [upload=15533]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:45:17.698820522 (15h5m39.442877383s) [upload=15533]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:45:17.860271466 (15h5m39.604328326s) [upload=15533]: Done uploading segments... Aug 8 00:45:17.860291088 (15h5m39.604347935s) [upload=15533]: Upload finished. err: <nil> Duration: 2.693286228s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [............................................X........X......] duration: 1.241590201s FAIL (2025-08-08 00:54:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 263481 (15h4m20.280320332s) [upload=15511]: Done uploading segments... Aug 8 00:43:58.536281876 (15h4m20.28033873s) [upload=15511]: Upload finished. err: <nil> Aug 8 00:45:01.015818127 (15h5m22.759874977s) [upload=15529]: Starting upload Aug 8 00:45:01.015874109 (15h5m22.759930972s) [upload=15529]: Done starting upload Aug 8 00:45:01.015956182 (15h5m22.76001303s) [upload=15529]: Uploading segments... Aug 8 00:45:01.016043908 (15h5m22.760100776s) [upload=15529]: Got next segment. Inline: true Aug 8 00:45:01.016049919 (15h5m22.760106782s) [upload=15529]: Waiting for error group managing segments... Aug 8 00:45:01.016115031 (15h5m22.760171895s) [upload=15529]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:45:01.016122390 (15h5m22.760179257s) [upload=15529]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:45:01.016125863 (15h5m22.760182727s) [upload=15529]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:45:01.173239782 (15h5m22.91729665s) [upload=15529]: Done uploading segments... Aug 8 00:45:01.173279478 (15h5m22.917336328s) [upload=15529]: Upload finished. err: <nil> Aug 8 00:46:00.429072612 (15h6m22.173129463s) [upload=15546]: Starting upload Aug 8 00:46:00.429156391 (15h6m22.173213278s) [upload=15546]: Done starting upload Aug 8 00:46:00.429267021 (15h6m22.173323882s) [upload=15546]: Uploading segments... Aug 8 00:46:00.429320811 (15h6m22.173377677s) [upload=15546]: Got next segment. Inline: true Aug 8 00:46:00.429326919 (15h6m22.17338377s) [upload=15546]: Waiting for error group managing segments... Aug 8 00:46:00.429377405 (15h6m22.173434256s) [upload=15546]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:46:00.429383461 (15h6m22.173440311s) [upload=15546]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:46:00.429387017 (15h6m22.173443877s) [upload=15546]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:46:00.684470407 (15h6m22.428527258s) [upload=15546]: Done uploading segments... Aug 8 00:46:00.684501074 (15h6m22.42855792s) [upload=15546]: Upload finished. err: <nil> Aug 8 00:47:14.991362846 (15h7m36.735419697s) [upload=15569]: Starting upload Aug 8 00:47:14.991436891 (15h7m36.735493754s) [upload=15569]: Done starting upload Aug 8 00:47:14.991521124 (15h7m36.735577974s) [upload=15569]: Uploading segments... Aug 8 00:47:14.991582184 (15h7m36.735639054s) [upload=15569]: Got next segment. Inline: true Aug 8 00:47:14.991588085 (15h7m36.735644945s) [upload=15569]: Waiting for error group managing segments... Aug 8 00:47:14.991641941 (15h7m36.735698801s) [upload=15569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:47:14.991647402 (15h7m36.735704266s) [upload=15569]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:47:14.991651037 (15h7m36.735707897s) [upload=15569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:47:15.149817056 (15h7m36.893873921s) [upload=15569]: Done uploading segments... Aug 8 00:47:15.149854826 (15h7m36.893911673s) [upload=15569]: Upload finished. err: <nil> Aug 8 00:48:05.184694323 (15h8m26.92875119s) [upload=15581]: Starting upload Aug 8 00:48:05.184775560 (15h8m26.928832424s) [upload=15581]: Done starting upload Aug 8 00:48:05.184860193 (15h8m26.928917046s) [upload=15581]: Uploading segments... Aug 8 00:48:05.184919879 (15h8m26.92897675s) [upload=15581]: Got next segment. Inline: true Aug 8 00:48:05.184927810 (15h8m26.928984686s) [upload=15581]: Waiting for error group managing segments... Aug 8 00:48:05.184989718 (15h8m26.929046568s) [upload=15581]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:48:05.184997233 (15h8m26.929054096s) [upload=15581]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:48:05.185002973 (15h8m26.929059837s) [upload=15581]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:48:05.333512154 (15h8m27.077569026s) [upload=15581]: Done uploading segments... Aug 8 00:48:05.333535225 (15h8m27.077592084s) [upload=15581]: Upload finished. err: <nil> Aug 8 00:48:25.959034104 (15h8m47.703090965s) [upload=15588]: Starting upload Aug 8 00:48:25.959129631 (15h8m47.703186501s) [upload=15588]: Done starting upload Aug 8 00:48:25.959263036 (15h8m47.703319897s) [upload=15588]: Uploading segments... Aug 8 00:48:25.959321571 (15h8m47.703378451s) [upload=15588]: Got next segment. Inline: true Aug 8 00:48:25.959328075 (15h8m47.70338494s) [upload=15588]: Waiting for error group managing segments... Aug 8 00:48:25.959379365 (15h8m47.70343623s) [upload=15588]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:48:25.959385760 (15h8m47.703442625s) [upload=15588]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:48:25.959389437 (15h8m47.703446299s) [upload=15588]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:48:26.117426083 (15h8m47.861482953s) [upload=15588]: Done uploading segments... Aug 8 00:48:26.117449715 (15h8m47.861506585s) [upload=15588]: Upload finished. err: <nil> Aug 8 00:49:18.380712745 (15h9m40.124769598s) [upload=15601]: Starting upload Aug 8 00:49:18.380774191 (15h9m40.124831063s) [upload=15601]: Done starting upload Aug 8 00:49:18.380853700 (15h9m40.124910555s) [upload=15601]: Uploading segments... Aug 8 00:49:18.380935089 (15h9m40.124991961s) [upload=15601]: Got next segment. Inline: true Aug 8 00:49:18.380942228 (15h9m40.124999082s) [upload=15601]: Waiting for error group managing segments... Aug 8 00:49:18.380990837 (15h9m40.125047687s) [upload=15601]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:49:18.380996570 (15h9m40.125053423s) [upload=15601]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:49:18.381000556 (15h9m40.125057404s) [upload=15601]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:49:18.524510041 (15h9m40.26856689s) [upload=15601]: Done uploading segments... Aug 8 00:49:18.524532847 (15h9m40.268589697s) [upload=15601]: Upload finished. err: <nil> Aug 8 00:50:14.219213519 (15h10m35.96327037s) [upload=15618]: Starting upload Aug 8 00:50:14.219268923 (15h10m35.963325784s) [upload=15618]: Done starting upload Aug 8 00:50:14.219367379 (15h10m35.963424233s) [upload=15618]: Uploading segments... Aug 8 00:50:14.219578059 (15h10m35.963634929s) [upload=15618]: Got next segment. Inline: true Aug 8 00:50:14.219614810 (15h10m35.963671673s) [upload=15618]: Waiting for error group managing segments... Aug 8 00:50:14.219687157 (15h10m35.963744017s) [upload=15618]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:50:14.219693611 (15h10m35.963750471s) [upload=15618]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:50:14.219699847 (15h10m35.96375671s) [upload=15618]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:50:14.385631211 (15h10m36.129688064s) [upload=15618]: Done uploading segments... Aug 8 00:50:14.385650912 (15h10m36.129707762s) [upload=15618]: Upload finished. err: <nil> Aug 8 00:50:44.080806747 (15h11m5.824863598s) [upload=15628]: Starting upload Aug 8 00:50:44.080911523 (15h11m5.824968393s) [upload=15628]: Done starting upload Aug 8 00:50:44.081032828 (15h11m5.825089678s) [upload=15628]: Uploading segments... Aug 8 00:50:44.081102196 (15h11m5.825159064s) [upload=15628]: Got next segment. Inline: true Aug 8 00:50:44.081110880 (15h11m5.82516774s) [upload=15628]: Waiting for error group managing segments... Aug 8 00:50:44.081162262 (15h11m5.825219175s) [upload=15628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:50:44.081168840 (15h11m5.825225704s) [upload=15628]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:50:44.081172385 (15h11m5.825229249s) [upload=15628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:50:44.261318102 (15h11m6.005374965s) [upload=15628]: Done uploading segments... Aug 8 00:50:44.261339384 (15h11m6.005396236s) [upload=15628]: Upload finished. err: <nil> Aug 8 00:51:00.984479825 (15h11m22.728536675s) [upload=15634]: Starting upload Aug 8 00:51:00.984537080 (15h11m22.72859393s) [upload=15634]: Done starting upload Aug 8 00:51:00.984618934 (15h11m22.728675798s) [upload=15634]: Uploading segments... Aug 8 00:51:00.984689929 (15h11m22.728746817s) [upload=15634]: Got next segment. Inline: true Aug 8 00:51:00.984696447 (15h11m22.728753308s) [upload=15634]: Waiting for error group managing segments... Aug 8 00:51:00.984795333 (15h11m22.728852187s) [upload=15634]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:51:00.984801510 (15h11m22.72885837s) [upload=15634]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:51:00.984804953 (15h11m22.728861813s) [upload=15634]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:51:01.179830418 (15h11m22.923887268s) [upload=15634]: Done uploading segments... Aug 8 00:51:01.179852239 (15h11m22.923909093s) [upload=15634]: Upload finished. err: <nil> Aug 8 00:51:18.788113821 (15h11m40.532170668s) [upload=15636]: Starting upload Aug 8 00:51:18.788190945 (15h11m40.53224781s) [upload=15636]: Done starting upload Aug 8 00:51:18.788266338 (15h11m40.532323198s) [upload=15636]: Uploading segments... Aug 8 00:51:18.788334958 (15h11m40.532391824s) [upload=15636]: Got next segment. Inline: true Aug 8 00:51:18.788341211 (15h11m40.532398065s) [upload=15636]: Waiting for error group managing segments... Aug 8 00:51:18.788385583 (15h11m40.532442429s) [upload=15636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:51:18.788390526 (15h11m40.532447372s) [upload=15636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:51:18.788393111 (15h11m40.532449957s) [upload=15636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:51:18.876506251 (15h11m40.620563101s) [upload=15636]: Done uploading segments... Aug 8 00:51:18.876525344 (15h11m40.620582195s) [upload=15636]: Upload finished. err: metaclient: Too Many Requests Duration: 1.241621521s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....X...........................................X..........] duration: 2.524560111s FAIL (2025-08-08 00:53:53 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Done starting upload Aug 8 00:45:16.266121630 (15h5m38.01017848s) [upload=15533]: Uploading segments... Aug 8 00:45:16.266428624 (15h5m38.010485484s) [upload=15533]: Got next segment. Inline: false Aug 8 00:45:16.266497281 (15h5m38.010554191s) [upload=15533]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:45:16.266512043 (15h5m38.010568914s) [upload=15533]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:45:16.467813786 (15h5m38.211870647s) [seg_pos={0 0}, upload=15533]: Begin upload segment... Aug 8 00:45:16.467883230 (15h5m38.211940081s) [seg_pos={0 0}, upload=15533]: Done begin upload segment. Aug 8 00:45:16.467935054 (15h5m38.211991969s) [noise=true , address=104.36.82.60:4369 , piece_id=WVPSCPUKKSYS4T3JIEEVDTS7WSFYXNGVQA6QNLGO3BKC3EKEYJGQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.467945924 (15h5m38.212002789s) [noise=true , address=104.36.82.54:4192 , piece_id=FDJHGRVMHAFB5JZM4Z77MF3NHRU3T4FNQPUAOXGP5NRCM5Q7RVRA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468440400 (15h5m38.212497272s) [noise=true , address=104.36.82.34:4577 , piece_id=4PTCNUDROG6OXW5DBUEMBAGPEMEMVLIW5OXRP2LPC55J2PTK3PAA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468457544 (15h5m38.21251441s) [noise=true , address=104.36.82.44:4887 , piece_id=FC4PG4NZJHR35DZLGLFZ6EJZYA2MJWLIDPBYHZCNJKJI4OVX5G4Q, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468847577 (15h5m38.212904441s) [noise=true , address=104.36.82.62:4430 , piece_id=JGWQSS5KYYCBOITFMENDLSGK6XSDVOT7TMRGJBZ55KBXX54ENY6A, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468887696 (15h5m38.21294456s) [noise=true , address=79.127.205.233:4633 , piece_id=F4RDTCFPIWZS4U65QMU5YXWOZSPPDIDMLJ26G3DCN3YV2UMAEFWA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469228336 (15h5m38.2132852s) [noise=true , address=104.36.82.43:4846 , piece_id=73PPCVC3HGFJDIB63URF7XA6PC37VCYLBCOMTSSFNC5UYZR6M5YA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469268209 (15h5m38.213325082s) [noise=true , address=64.49.13.140:4515 , piece_id=AQTWEM35E7E6YUISEOQGTPTPKD6RDOMKXLK267UILGD77273OY4A, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469613163 (15h5m38.21367003s) [noise=true , address=104.36.82.16:4307 , piece_id=B7ZUFMTGVRPCNJSTBCXMQRZW2AGKPXM2LJRDLYZWPXQ4F5JJ75UQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469702757 (15h5m38.213759621s) [noise=true , address=104.36.82.9:4038 , piece_id=RVXFCLSREWSON62WSTYVISC52HN33VUN2BHJ7UR72MEBMTFKQV6A, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470036233 (15h5m38.214093098s) [noise=true , address=104.36.82.23:4464 , piece_id=4HNT67XCPCNCECLCE36RVAHTJ7VBAADPQ7VMC7IASAUPVVEKIVWQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470129321 (15h5m38.214186184s) [noise=true , address=45.91.116.6:4982 , piece_id=NV3WES7XAJBCNO5EQMWFK6SNUEYE63YKTQC5ZMRF7URP6SX3FXGA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470411978 (15h5m38.214468842s) [noise=true , address=104.36.82.23:4452 , piece_id=B4XGMOK4RCAIOV3VSJXFZ73O52E4WIRUGSPVTCCJ4XUGNJHUQWBA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470482224 (15h5m38.21453909s) [noise=true , address=104.36.82.14:4254 , piece_id=AIB65II65FK6M3EQHX6PVX27N4Q4BQKI5QGVP3PD6D34ZKXVCLZA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470792753 (15h5m38.214849603s) [noise=true , address=104.36.82.19:4405 , piece_id=SA2CPGDA75CYSUDYYZ6V4NKTG5AEIZC7SVYNL2OLBK3E5MSK5FQQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470877563 (15h5m38.214934486s) [noise=true , address=104.36.82.14:4243 , piece_id=HSR3LCZHIDLJSJ7WISDYOWGPLBFZ73MIXTR7MLFQ2TLUBYLRL4DQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.471502176 (15h5m38.215559147s) [noise=true , address=104.36.82.45:4910 , piece_id=OGS6W5AGDOACXIMZ4OH3GNQ43YUU6WOYJHKAHNMHJZBQPJUVVCRA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.471850592 (15h5m38.215907452s) [noise=true , address=79.127.201.215:4358 , piece_id=Z3PRSJIAQMKCL2ZU5TA7WZ2IBCVIXB2K5UJBMS3UCI5H6HDNTWBA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:17.050924903 (15h5m38.794981774s) [upload=15533]: Next returned nil segment Aug 8 00:45:17.051086092 (15h5m38.795142971s) [upload=15533]: Waiting for error group managing segments... Aug 8 00:45:17.225555422 (15h5m38.969612281s) [noise=true , address=104.36.82.62:4430 , piece_id=JGWQSS5KYYCBOITFMENDLSGK6XSDVOT7TMRGJBZ55KBXX54ENY6A, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.232285957 (15h5m38.97634281s) [noise=true , address=45.91.116.6:4982 , piece_id=NV3WES7XAJBCNO5EQMWFK6SNUEYE63YKTQC5ZMRF7URP6SX3FXGA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.239931943 (15h5m38.983988797s) [noise=true , address=104.36.82.45:4910 , piece_id=OGS6W5AGDOACXIMZ4OH3GNQ43YUU6WOYJHKAHNMHJZBQPJUVVCRA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.240442032 (15h5m38.984498886s) [noise=true , address=104.36.82.34:4577 , piece_id=4PTCNUDROG6OXW5DBUEMBAGPEMEMVLIW5OXRP2LPC55J2PTK3PAA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.280384119 (15h5m39.024440979s) [noise=true , address=104.36.82.19:4405 , piece_id=SA2CPGDA75CYSUDYYZ6V4NKTG5AEIZC7SVYNL2OLBK3E5MSK5FQQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.284571751 (15h5m39.028628621s) [noise=true , address=104.36.82.16:4307 , piece_id=B7ZUFMTGVRPCNJSTBCXMQRZW2AGKPXM2LJRDLYZWPXQ4F5JJ75UQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.295563563 (15h5m39.03962043s) [noise=true , address=104.36.82.14:4254 , piece_id=AIB65II65FK6M3EQHX6PVX27N4Q4BQKI5QGVP3PD6D34ZKXVCLZA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.367859627 (15h5m39.11191649s) [noise=true , address=104.36.82.44:4887 , piece_id=FC4PG4NZJHR35DZLGLFZ6EJZYA2MJWLIDPBYHZCNJKJI4OVX5G4Q, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.486413539 (15h5m39.2304704s) [noise=true , address=104.36.82.9:4038 , piece_id=RVXFCLSREWSON62WSTYVISC52HN33VUN2BHJ7UR72MEBMTFKQV6A, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.510192603 (15h5m39.254249451s) [noise=true , address=79.127.201.215:4358 , piece_id=Z3PRSJIAQMKCL2ZU5TA7WZ2IBCVIXB2K5UJBMS3UCI5H6HDNTWBA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.609092139 (15h5m39.353148988s) [noise=true , address=104.36.82.54:4192 , piece_id=FDJHGRVMHAFB5JZM4Z77MF3NHRU3T4FNQPUAOXGP5NRCM5Q7RVRA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.647464269 (15h5m39.391521122s) [noise=true , address=104.36.82.14:4243 , piece_id=HSR3LCZHIDLJSJ7WISDYOWGPLBFZ73MIXTR7MLFQ2TLUBYLRL4DQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.681479215 (15h5m39.425536078s) [noise=true , address=104.36.82.43:4846 , piece_id=73PPCVC3HGFJDIB63URF7XA6PC37VCYLBCOMTSSFNC5UYZR6M5YA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.697970794 (15h5m39.442027648s) [noise=true , address=104.36.82.60:4369 , piece_id=WVPSCPUKKSYS4T3JIEEVDTS7WSFYXNGVQA6QNLGO3BKC3EKEYJGQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.697997236 (15h5m39.442054151s) [seg_pos={0 0}, upload=15533]: Segment reached optimal threshold of 14 pieces. Aug 8 00:45:17.698203533 (15h5m39.442260383s) [noise=true , address=79.127.205.233:4633 , piece_id=F4RDTCFPIWZS4U65QMU5YXWOZSPPDIDMLJ26G3DCN3YV2UMAEFWA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EEYb4mCySvZ54qzmygCQ7hLyTGYpdNJMyrvPvHJjtfGos9usk): protocol: expected piece hash; context canceled; EOF Aug 8 00:45:17.698290533 (15h5m39.442347383s) [noise=true , address=64.49.13.140:4515 , piece_id=AQTWEM35E7E6YUISEOQGTPTPKD6RDOMKXLK267UILGD77273OY4A, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mY7sRFAeDLmKPqq8BgwnGtkdPTsDRR6soTBshsn4qhP7evARb): protocol: expected piece hash; context canceled; EOF Aug 8 00:45:17.698544485 (15h5m39.442601338s) [noise=true , address=104.36.82.23:4464 , piece_id=4HNT67XCPCNCECLCE36RVAHTJ7VBAADPQ7VMC7IASAUPVVEKIVWQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mLbGHntspYmsTVg8433HYSdAEMrXjGnDhmmvX1JmmgcxtdLgk): protocol: write tcp 139.59.144.22:39260->104.36.82.23:4464: use of closed network connection; write tcp 139.59.144.22:39260->104.36.82.23:4464: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39260->104.36.82.23:4464: use of closed network connection Aug 8 00:45:17.698701613 (15h5m39.442758474s) [noise=true , address=104.36.82.23:4452 , piece_id=B4XGMOK4RCAIOV3VSJXFZ73O52E4WIRUGSPVTCCJ4XUGNJHUQWBA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sAh1mLmzbtmc4o2FAZssW8Y7iDDNmNTCXdCS8xrH8PBVQqxG5): protocol: context canceled; context canceled Aug 8 00:45:17.698723379 (15h5m39.44278023s) [seg_pos={0 0}, upload=15533]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 00:45:17.698815435 (15h5m39.442872285s) [upload=15533]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:45:17.698820522 (15h5m39.442877383s) [upload=15533]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:45:17.860271466 (15h5m39.604328326s) [upload=15533]: Done uploading segments... Aug 8 00:45:17.860291088 (15h5m39.604347935s) [upload=15533]: Upload finished. err: <nil> Duration: 2.524613312s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.............................................X........X.....] duration: 851.563057ms FAIL (2025-08-08 00:53:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 263481 (15h4m20.280320332s) [upload=15511]: Done uploading segments... Aug 8 00:43:58.536281876 (15h4m20.28033873s) [upload=15511]: Upload finished. err: <nil> Aug 8 00:45:01.015818127 (15h5m22.759874977s) [upload=15529]: Starting upload Aug 8 00:45:01.015874109 (15h5m22.759930972s) [upload=15529]: Done starting upload Aug 8 00:45:01.015956182 (15h5m22.76001303s) [upload=15529]: Uploading segments... Aug 8 00:45:01.016043908 (15h5m22.760100776s) [upload=15529]: Got next segment. Inline: true Aug 8 00:45:01.016049919 (15h5m22.760106782s) [upload=15529]: Waiting for error group managing segments... Aug 8 00:45:01.016115031 (15h5m22.760171895s) [upload=15529]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:45:01.016122390 (15h5m22.760179257s) [upload=15529]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:45:01.016125863 (15h5m22.760182727s) [upload=15529]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:45:01.173239782 (15h5m22.91729665s) [upload=15529]: Done uploading segments... Aug 8 00:45:01.173279478 (15h5m22.917336328s) [upload=15529]: Upload finished. err: <nil> Aug 8 00:46:00.429072612 (15h6m22.173129463s) [upload=15546]: Starting upload Aug 8 00:46:00.429156391 (15h6m22.173213278s) [upload=15546]: Done starting upload Aug 8 00:46:00.429267021 (15h6m22.173323882s) [upload=15546]: Uploading segments... Aug 8 00:46:00.429320811 (15h6m22.173377677s) [upload=15546]: Got next segment. Inline: true Aug 8 00:46:00.429326919 (15h6m22.17338377s) [upload=15546]: Waiting for error group managing segments... Aug 8 00:46:00.429377405 (15h6m22.173434256s) [upload=15546]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:46:00.429383461 (15h6m22.173440311s) [upload=15546]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:46:00.429387017 (15h6m22.173443877s) [upload=15546]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:46:00.684470407 (15h6m22.428527258s) [upload=15546]: Done uploading segments... Aug 8 00:46:00.684501074 (15h6m22.42855792s) [upload=15546]: Upload finished. err: <nil> Aug 8 00:47:14.991362846 (15h7m36.735419697s) [upload=15569]: Starting upload Aug 8 00:47:14.991436891 (15h7m36.735493754s) [upload=15569]: Done starting upload Aug 8 00:47:14.991521124 (15h7m36.735577974s) [upload=15569]: Uploading segments... Aug 8 00:47:14.991582184 (15h7m36.735639054s) [upload=15569]: Got next segment. Inline: true Aug 8 00:47:14.991588085 (15h7m36.735644945s) [upload=15569]: Waiting for error group managing segments... Aug 8 00:47:14.991641941 (15h7m36.735698801s) [upload=15569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:47:14.991647402 (15h7m36.735704266s) [upload=15569]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:47:14.991651037 (15h7m36.735707897s) [upload=15569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:47:15.149817056 (15h7m36.893873921s) [upload=15569]: Done uploading segments... Aug 8 00:47:15.149854826 (15h7m36.893911673s) [upload=15569]: Upload finished. err: <nil> Aug 8 00:48:05.184694323 (15h8m26.92875119s) [upload=15581]: Starting upload Aug 8 00:48:05.184775560 (15h8m26.928832424s) [upload=15581]: Done starting upload Aug 8 00:48:05.184860193 (15h8m26.928917046s) [upload=15581]: Uploading segments... Aug 8 00:48:05.184919879 (15h8m26.92897675s) [upload=15581]: Got next segment. Inline: true Aug 8 00:48:05.184927810 (15h8m26.928984686s) [upload=15581]: Waiting for error group managing segments... Aug 8 00:48:05.184989718 (15h8m26.929046568s) [upload=15581]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:48:05.184997233 (15h8m26.929054096s) [upload=15581]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:48:05.185002973 (15h8m26.929059837s) [upload=15581]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:48:05.333512154 (15h8m27.077569026s) [upload=15581]: Done uploading segments... Aug 8 00:48:05.333535225 (15h8m27.077592084s) [upload=15581]: Upload finished. err: <nil> Aug 8 00:48:25.959034104 (15h8m47.703090965s) [upload=15588]: Starting upload Aug 8 00:48:25.959129631 (15h8m47.703186501s) [upload=15588]: Done starting upload Aug 8 00:48:25.959263036 (15h8m47.703319897s) [upload=15588]: Uploading segments... Aug 8 00:48:25.959321571 (15h8m47.703378451s) [upload=15588]: Got next segment. Inline: true Aug 8 00:48:25.959328075 (15h8m47.70338494s) [upload=15588]: Waiting for error group managing segments... Aug 8 00:48:25.959379365 (15h8m47.70343623s) [upload=15588]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:48:25.959385760 (15h8m47.703442625s) [upload=15588]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:48:25.959389437 (15h8m47.703446299s) [upload=15588]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:48:26.117426083 (15h8m47.861482953s) [upload=15588]: Done uploading segments... Aug 8 00:48:26.117449715 (15h8m47.861506585s) [upload=15588]: Upload finished. err: <nil> Aug 8 00:49:18.380712745 (15h9m40.124769598s) [upload=15601]: Starting upload Aug 8 00:49:18.380774191 (15h9m40.124831063s) [upload=15601]: Done starting upload Aug 8 00:49:18.380853700 (15h9m40.124910555s) [upload=15601]: Uploading segments... Aug 8 00:49:18.380935089 (15h9m40.124991961s) [upload=15601]: Got next segment. Inline: true Aug 8 00:49:18.380942228 (15h9m40.124999082s) [upload=15601]: Waiting for error group managing segments... Aug 8 00:49:18.380990837 (15h9m40.125047687s) [upload=15601]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:49:18.380996570 (15h9m40.125053423s) [upload=15601]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:49:18.381000556 (15h9m40.125057404s) [upload=15601]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:49:18.524510041 (15h9m40.26856689s) [upload=15601]: Done uploading segments... Aug 8 00:49:18.524532847 (15h9m40.268589697s) [upload=15601]: Upload finished. err: <nil> Aug 8 00:50:14.219213519 (15h10m35.96327037s) [upload=15618]: Starting upload Aug 8 00:50:14.219268923 (15h10m35.963325784s) [upload=15618]: Done starting upload Aug 8 00:50:14.219367379 (15h10m35.963424233s) [upload=15618]: Uploading segments... Aug 8 00:50:14.219578059 (15h10m35.963634929s) [upload=15618]: Got next segment. Inline: true Aug 8 00:50:14.219614810 (15h10m35.963671673s) [upload=15618]: Waiting for error group managing segments... Aug 8 00:50:14.219687157 (15h10m35.963744017s) [upload=15618]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:50:14.219693611 (15h10m35.963750471s) [upload=15618]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:50:14.219699847 (15h10m35.96375671s) [upload=15618]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:50:14.385631211 (15h10m36.129688064s) [upload=15618]: Done uploading segments... Aug 8 00:50:14.385650912 (15h10m36.129707762s) [upload=15618]: Upload finished. err: <nil> Aug 8 00:50:44.080806747 (15h11m5.824863598s) [upload=15628]: Starting upload Aug 8 00:50:44.080911523 (15h11m5.824968393s) [upload=15628]: Done starting upload Aug 8 00:50:44.081032828 (15h11m5.825089678s) [upload=15628]: Uploading segments... Aug 8 00:50:44.081102196 (15h11m5.825159064s) [upload=15628]: Got next segment. Inline: true Aug 8 00:50:44.081110880 (15h11m5.82516774s) [upload=15628]: Waiting for error group managing segments... Aug 8 00:50:44.081162262 (15h11m5.825219175s) [upload=15628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:50:44.081168840 (15h11m5.825225704s) [upload=15628]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:50:44.081172385 (15h11m5.825229249s) [upload=15628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:50:44.261318102 (15h11m6.005374965s) [upload=15628]: Done uploading segments... Aug 8 00:50:44.261339384 (15h11m6.005396236s) [upload=15628]: Upload finished. err: <nil> Aug 8 00:51:00.984479825 (15h11m22.728536675s) [upload=15634]: Starting upload Aug 8 00:51:00.984537080 (15h11m22.72859393s) [upload=15634]: Done starting upload Aug 8 00:51:00.984618934 (15h11m22.728675798s) [upload=15634]: Uploading segments... Aug 8 00:51:00.984689929 (15h11m22.728746817s) [upload=15634]: Got next segment. Inline: true Aug 8 00:51:00.984696447 (15h11m22.728753308s) [upload=15634]: Waiting for error group managing segments... Aug 8 00:51:00.984795333 (15h11m22.728852187s) [upload=15634]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:51:00.984801510 (15h11m22.72885837s) [upload=15634]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:51:00.984804953 (15h11m22.728861813s) [upload=15634]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:51:01.179830418 (15h11m22.923887268s) [upload=15634]: Done uploading segments... Aug 8 00:51:01.179852239 (15h11m22.923909093s) [upload=15634]: Upload finished. err: <nil> Aug 8 00:51:18.788113821 (15h11m40.532170668s) [upload=15636]: Starting upload Aug 8 00:51:18.788190945 (15h11m40.53224781s) [upload=15636]: Done starting upload Aug 8 00:51:18.788266338 (15h11m40.532323198s) [upload=15636]: Uploading segments... Aug 8 00:51:18.788334958 (15h11m40.532391824s) [upload=15636]: Got next segment. Inline: true Aug 8 00:51:18.788341211 (15h11m40.532398065s) [upload=15636]: Waiting for error group managing segments... Aug 8 00:51:18.788385583 (15h11m40.532442429s) [upload=15636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:51:18.788390526 (15h11m40.532447372s) [upload=15636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:51:18.788393111 (15h11m40.532449957s) [upload=15636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:51:18.876506251 (15h11m40.620563101s) [upload=15636]: Done uploading segments... Aug 8 00:51:18.876525344 (15h11m40.620582195s) [upload=15636]: Upload finished. err: metaclient: Too Many Requests Duration: 851.595655ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..............................................X........X....] duration: 872.698792ms FAIL (2025-08-08 00:53:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 263481 (15h4m20.280320332s) [upload=15511]: Done uploading segments... Aug 8 00:43:58.536281876 (15h4m20.28033873s) [upload=15511]: Upload finished. err: <nil> Aug 8 00:45:01.015818127 (15h5m22.759874977s) [upload=15529]: Starting upload Aug 8 00:45:01.015874109 (15h5m22.759930972s) [upload=15529]: Done starting upload Aug 8 00:45:01.015956182 (15h5m22.76001303s) [upload=15529]: Uploading segments... Aug 8 00:45:01.016043908 (15h5m22.760100776s) [upload=15529]: Got next segment. Inline: true Aug 8 00:45:01.016049919 (15h5m22.760106782s) [upload=15529]: Waiting for error group managing segments... Aug 8 00:45:01.016115031 (15h5m22.760171895s) [upload=15529]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:45:01.016122390 (15h5m22.760179257s) [upload=15529]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:45:01.016125863 (15h5m22.760182727s) [upload=15529]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:45:01.173239782 (15h5m22.91729665s) [upload=15529]: Done uploading segments... Aug 8 00:45:01.173279478 (15h5m22.917336328s) [upload=15529]: Upload finished. err: <nil> Aug 8 00:46:00.429072612 (15h6m22.173129463s) [upload=15546]: Starting upload Aug 8 00:46:00.429156391 (15h6m22.173213278s) [upload=15546]: Done starting upload Aug 8 00:46:00.429267021 (15h6m22.173323882s) [upload=15546]: Uploading segments... Aug 8 00:46:00.429320811 (15h6m22.173377677s) [upload=15546]: Got next segment. Inline: true Aug 8 00:46:00.429326919 (15h6m22.17338377s) [upload=15546]: Waiting for error group managing segments... Aug 8 00:46:00.429377405 (15h6m22.173434256s) [upload=15546]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:46:00.429383461 (15h6m22.173440311s) [upload=15546]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:46:00.429387017 (15h6m22.173443877s) [upload=15546]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:46:00.684470407 (15h6m22.428527258s) [upload=15546]: Done uploading segments... Aug 8 00:46:00.684501074 (15h6m22.42855792s) [upload=15546]: Upload finished. err: <nil> Aug 8 00:47:14.991362846 (15h7m36.735419697s) [upload=15569]: Starting upload Aug 8 00:47:14.991436891 (15h7m36.735493754s) [upload=15569]: Done starting upload Aug 8 00:47:14.991521124 (15h7m36.735577974s) [upload=15569]: Uploading segments... Aug 8 00:47:14.991582184 (15h7m36.735639054s) [upload=15569]: Got next segment. Inline: true Aug 8 00:47:14.991588085 (15h7m36.735644945s) [upload=15569]: Waiting for error group managing segments... Aug 8 00:47:14.991641941 (15h7m36.735698801s) [upload=15569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:47:14.991647402 (15h7m36.735704266s) [upload=15569]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:47:14.991651037 (15h7m36.735707897s) [upload=15569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:47:15.149817056 (15h7m36.893873921s) [upload=15569]: Done uploading segments... Aug 8 00:47:15.149854826 (15h7m36.893911673s) [upload=15569]: Upload finished. err: <nil> Aug 8 00:48:05.184694323 (15h8m26.92875119s) [upload=15581]: Starting upload Aug 8 00:48:05.184775560 (15h8m26.928832424s) [upload=15581]: Done starting upload Aug 8 00:48:05.184860193 (15h8m26.928917046s) [upload=15581]: Uploading segments... Aug 8 00:48:05.184919879 (15h8m26.92897675s) [upload=15581]: Got next segment. Inline: true Aug 8 00:48:05.184927810 (15h8m26.928984686s) [upload=15581]: Waiting for error group managing segments... Aug 8 00:48:05.184989718 (15h8m26.929046568s) [upload=15581]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:48:05.184997233 (15h8m26.929054096s) [upload=15581]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:48:05.185002973 (15h8m26.929059837s) [upload=15581]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:48:05.333512154 (15h8m27.077569026s) [upload=15581]: Done uploading segments... Aug 8 00:48:05.333535225 (15h8m27.077592084s) [upload=15581]: Upload finished. err: <nil> Aug 8 00:48:25.959034104 (15h8m47.703090965s) [upload=15588]: Starting upload Aug 8 00:48:25.959129631 (15h8m47.703186501s) [upload=15588]: Done starting upload Aug 8 00:48:25.959263036 (15h8m47.703319897s) [upload=15588]: Uploading segments... Aug 8 00:48:25.959321571 (15h8m47.703378451s) [upload=15588]: Got next segment. Inline: true Aug 8 00:48:25.959328075 (15h8m47.70338494s) [upload=15588]: Waiting for error group managing segments... Aug 8 00:48:25.959379365 (15h8m47.70343623s) [upload=15588]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:48:25.959385760 (15h8m47.703442625s) [upload=15588]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:48:25.959389437 (15h8m47.703446299s) [upload=15588]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:48:26.117426083 (15h8m47.861482953s) [upload=15588]: Done uploading segments... Aug 8 00:48:26.117449715 (15h8m47.861506585s) [upload=15588]: Upload finished. err: <nil> Aug 8 00:49:18.380712745 (15h9m40.124769598s) [upload=15601]: Starting upload Aug 8 00:49:18.380774191 (15h9m40.124831063s) [upload=15601]: Done starting upload Aug 8 00:49:18.380853700 (15h9m40.124910555s) [upload=15601]: Uploading segments... Aug 8 00:49:18.380935089 (15h9m40.124991961s) [upload=15601]: Got next segment. Inline: true Aug 8 00:49:18.380942228 (15h9m40.124999082s) [upload=15601]: Waiting for error group managing segments... Aug 8 00:49:18.380990837 (15h9m40.125047687s) [upload=15601]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:49:18.380996570 (15h9m40.125053423s) [upload=15601]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:49:18.381000556 (15h9m40.125057404s) [upload=15601]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:49:18.524510041 (15h9m40.26856689s) [upload=15601]: Done uploading segments... Aug 8 00:49:18.524532847 (15h9m40.268589697s) [upload=15601]: Upload finished. err: <nil> Aug 8 00:50:14.219213519 (15h10m35.96327037s) [upload=15618]: Starting upload Aug 8 00:50:14.219268923 (15h10m35.963325784s) [upload=15618]: Done starting upload Aug 8 00:50:14.219367379 (15h10m35.963424233s) [upload=15618]: Uploading segments... Aug 8 00:50:14.219578059 (15h10m35.963634929s) [upload=15618]: Got next segment. Inline: true Aug 8 00:50:14.219614810 (15h10m35.963671673s) [upload=15618]: Waiting for error group managing segments... Aug 8 00:50:14.219687157 (15h10m35.963744017s) [upload=15618]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:50:14.219693611 (15h10m35.963750471s) [upload=15618]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:50:14.219699847 (15h10m35.96375671s) [upload=15618]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:50:14.385631211 (15h10m36.129688064s) [upload=15618]: Done uploading segments... Aug 8 00:50:14.385650912 (15h10m36.129707762s) [upload=15618]: Upload finished. err: <nil> Aug 8 00:50:44.080806747 (15h11m5.824863598s) [upload=15628]: Starting upload Aug 8 00:50:44.080911523 (15h11m5.824968393s) [upload=15628]: Done starting upload Aug 8 00:50:44.081032828 (15h11m5.825089678s) [upload=15628]: Uploading segments... Aug 8 00:50:44.081102196 (15h11m5.825159064s) [upload=15628]: Got next segment. Inline: true Aug 8 00:50:44.081110880 (15h11m5.82516774s) [upload=15628]: Waiting for error group managing segments... Aug 8 00:50:44.081162262 (15h11m5.825219175s) [upload=15628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:50:44.081168840 (15h11m5.825225704s) [upload=15628]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:50:44.081172385 (15h11m5.825229249s) [upload=15628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:50:44.261318102 (15h11m6.005374965s) [upload=15628]: Done uploading segments... Aug 8 00:50:44.261339384 (15h11m6.005396236s) [upload=15628]: Upload finished. err: <nil> Aug 8 00:51:00.984479825 (15h11m22.728536675s) [upload=15634]: Starting upload Aug 8 00:51:00.984537080 (15h11m22.72859393s) [upload=15634]: Done starting upload Aug 8 00:51:00.984618934 (15h11m22.728675798s) [upload=15634]: Uploading segments... Aug 8 00:51:00.984689929 (15h11m22.728746817s) [upload=15634]: Got next segment. Inline: true Aug 8 00:51:00.984696447 (15h11m22.728753308s) [upload=15634]: Waiting for error group managing segments... Aug 8 00:51:00.984795333 (15h11m22.728852187s) [upload=15634]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:51:00.984801510 (15h11m22.72885837s) [upload=15634]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:51:00.984804953 (15h11m22.728861813s) [upload=15634]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:51:01.179830418 (15h11m22.923887268s) [upload=15634]: Done uploading segments... Aug 8 00:51:01.179852239 (15h11m22.923909093s) [upload=15634]: Upload finished. err: <nil> Aug 8 00:51:18.788113821 (15h11m40.532170668s) [upload=15636]: Starting upload Aug 8 00:51:18.788190945 (15h11m40.53224781s) [upload=15636]: Done starting upload Aug 8 00:51:18.788266338 (15h11m40.532323198s) [upload=15636]: Uploading segments... Aug 8 00:51:18.788334958 (15h11m40.532391824s) [upload=15636]: Got next segment. Inline: true Aug 8 00:51:18.788341211 (15h11m40.532398065s) [upload=15636]: Waiting for error group managing segments... Aug 8 00:51:18.788385583 (15h11m40.532442429s) [upload=15636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:51:18.788390526 (15h11m40.532447372s) [upload=15636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:51:18.788393111 (15h11m40.532449957s) [upload=15636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:51:18.876506251 (15h11m40.620563101s) [upload=15636]: Done uploading segments... Aug 8 00:51:18.876525344 (15h11m40.620582195s) [upload=15636]: Upload finished. err: metaclient: Too Many Requests Duration: 872.732105ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [......X...........................................X.........] duration: 2.516990503s FAIL (2025-08-08 00:53:22 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Done starting upload Aug 8 00:45:16.266121630 (15h5m38.01017848s) [upload=15533]: Uploading segments... Aug 8 00:45:16.266428624 (15h5m38.010485484s) [upload=15533]: Got next segment. Inline: false Aug 8 00:45:16.266497281 (15h5m38.010554191s) [upload=15533]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:45:16.266512043 (15h5m38.010568914s) [upload=15533]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:45:16.467813786 (15h5m38.211870647s) [seg_pos={0 0}, upload=15533]: Begin upload segment... Aug 8 00:45:16.467883230 (15h5m38.211940081s) [seg_pos={0 0}, upload=15533]: Done begin upload segment. Aug 8 00:45:16.467935054 (15h5m38.211991969s) [noise=true , address=104.36.82.60:4369 , piece_id=WVPSCPUKKSYS4T3JIEEVDTS7WSFYXNGVQA6QNLGO3BKC3EKEYJGQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.467945924 (15h5m38.212002789s) [noise=true , address=104.36.82.54:4192 , piece_id=FDJHGRVMHAFB5JZM4Z77MF3NHRU3T4FNQPUAOXGP5NRCM5Q7RVRA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468440400 (15h5m38.212497272s) [noise=true , address=104.36.82.34:4577 , piece_id=4PTCNUDROG6OXW5DBUEMBAGPEMEMVLIW5OXRP2LPC55J2PTK3PAA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468457544 (15h5m38.21251441s) [noise=true , address=104.36.82.44:4887 , piece_id=FC4PG4NZJHR35DZLGLFZ6EJZYA2MJWLIDPBYHZCNJKJI4OVX5G4Q, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468847577 (15h5m38.212904441s) [noise=true , address=104.36.82.62:4430 , piece_id=JGWQSS5KYYCBOITFMENDLSGK6XSDVOT7TMRGJBZ55KBXX54ENY6A, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468887696 (15h5m38.21294456s) [noise=true , address=79.127.205.233:4633 , piece_id=F4RDTCFPIWZS4U65QMU5YXWOZSPPDIDMLJ26G3DCN3YV2UMAEFWA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469228336 (15h5m38.2132852s) [noise=true , address=104.36.82.43:4846 , piece_id=73PPCVC3HGFJDIB63URF7XA6PC37VCYLBCOMTSSFNC5UYZR6M5YA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469268209 (15h5m38.213325082s) [noise=true , address=64.49.13.140:4515 , piece_id=AQTWEM35E7E6YUISEOQGTPTPKD6RDOMKXLK267UILGD77273OY4A, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469613163 (15h5m38.21367003s) [noise=true , address=104.36.82.16:4307 , piece_id=B7ZUFMTGVRPCNJSTBCXMQRZW2AGKPXM2LJRDLYZWPXQ4F5JJ75UQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469702757 (15h5m38.213759621s) [noise=true , address=104.36.82.9:4038 , piece_id=RVXFCLSREWSON62WSTYVISC52HN33VUN2BHJ7UR72MEBMTFKQV6A, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470036233 (15h5m38.214093098s) [noise=true , address=104.36.82.23:4464 , piece_id=4HNT67XCPCNCECLCE36RVAHTJ7VBAADPQ7VMC7IASAUPVVEKIVWQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470129321 (15h5m38.214186184s) [noise=true , address=45.91.116.6:4982 , piece_id=NV3WES7XAJBCNO5EQMWFK6SNUEYE63YKTQC5ZMRF7URP6SX3FXGA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470411978 (15h5m38.214468842s) [noise=true , address=104.36.82.23:4452 , piece_id=B4XGMOK4RCAIOV3VSJXFZ73O52E4WIRUGSPVTCCJ4XUGNJHUQWBA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470482224 (15h5m38.21453909s) [noise=true , address=104.36.82.14:4254 , piece_id=AIB65II65FK6M3EQHX6PVX27N4Q4BQKI5QGVP3PD6D34ZKXVCLZA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470792753 (15h5m38.214849603s) [noise=true , address=104.36.82.19:4405 , piece_id=SA2CPGDA75CYSUDYYZ6V4NKTG5AEIZC7SVYNL2OLBK3E5MSK5FQQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470877563 (15h5m38.214934486s) [noise=true , address=104.36.82.14:4243 , piece_id=HSR3LCZHIDLJSJ7WISDYOWGPLBFZ73MIXTR7MLFQ2TLUBYLRL4DQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.471502176 (15h5m38.215559147s) [noise=true , address=104.36.82.45:4910 , piece_id=OGS6W5AGDOACXIMZ4OH3GNQ43YUU6WOYJHKAHNMHJZBQPJUVVCRA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.471850592 (15h5m38.215907452s) [noise=true , address=79.127.201.215:4358 , piece_id=Z3PRSJIAQMKCL2ZU5TA7WZ2IBCVIXB2K5UJBMS3UCI5H6HDNTWBA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:17.050924903 (15h5m38.794981774s) [upload=15533]: Next returned nil segment Aug 8 00:45:17.051086092 (15h5m38.795142971s) [upload=15533]: Waiting for error group managing segments... Aug 8 00:45:17.225555422 (15h5m38.969612281s) [noise=true , address=104.36.82.62:4430 , piece_id=JGWQSS5KYYCBOITFMENDLSGK6XSDVOT7TMRGJBZ55KBXX54ENY6A, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.232285957 (15h5m38.97634281s) [noise=true , address=45.91.116.6:4982 , piece_id=NV3WES7XAJBCNO5EQMWFK6SNUEYE63YKTQC5ZMRF7URP6SX3FXGA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.239931943 (15h5m38.983988797s) [noise=true , address=104.36.82.45:4910 , piece_id=OGS6W5AGDOACXIMZ4OH3GNQ43YUU6WOYJHKAHNMHJZBQPJUVVCRA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.240442032 (15h5m38.984498886s) [noise=true , address=104.36.82.34:4577 , piece_id=4PTCNUDROG6OXW5DBUEMBAGPEMEMVLIW5OXRP2LPC55J2PTK3PAA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.280384119 (15h5m39.024440979s) [noise=true , address=104.36.82.19:4405 , piece_id=SA2CPGDA75CYSUDYYZ6V4NKTG5AEIZC7SVYNL2OLBK3E5MSK5FQQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.284571751 (15h5m39.028628621s) [noise=true , address=104.36.82.16:4307 , piece_id=B7ZUFMTGVRPCNJSTBCXMQRZW2AGKPXM2LJRDLYZWPXQ4F5JJ75UQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.295563563 (15h5m39.03962043s) [noise=true , address=104.36.82.14:4254 , piece_id=AIB65II65FK6M3EQHX6PVX27N4Q4BQKI5QGVP3PD6D34ZKXVCLZA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.367859627 (15h5m39.11191649s) [noise=true , address=104.36.82.44:4887 , piece_id=FC4PG4NZJHR35DZLGLFZ6EJZYA2MJWLIDPBYHZCNJKJI4OVX5G4Q, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.486413539 (15h5m39.2304704s) [noise=true , address=104.36.82.9:4038 , piece_id=RVXFCLSREWSON62WSTYVISC52HN33VUN2BHJ7UR72MEBMTFKQV6A, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.510192603 (15h5m39.254249451s) [noise=true , address=79.127.201.215:4358 , piece_id=Z3PRSJIAQMKCL2ZU5TA7WZ2IBCVIXB2K5UJBMS3UCI5H6HDNTWBA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.609092139 (15h5m39.353148988s) [noise=true , address=104.36.82.54:4192 , piece_id=FDJHGRVMHAFB5JZM4Z77MF3NHRU3T4FNQPUAOXGP5NRCM5Q7RVRA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.647464269 (15h5m39.391521122s) [noise=true , address=104.36.82.14:4243 , piece_id=HSR3LCZHIDLJSJ7WISDYOWGPLBFZ73MIXTR7MLFQ2TLUBYLRL4DQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.681479215 (15h5m39.425536078s) [noise=true , address=104.36.82.43:4846 , piece_id=73PPCVC3HGFJDIB63URF7XA6PC37VCYLBCOMTSSFNC5UYZR6M5YA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.697970794 (15h5m39.442027648s) [noise=true , address=104.36.82.60:4369 , piece_id=WVPSCPUKKSYS4T3JIEEVDTS7WSFYXNGVQA6QNLGO3BKC3EKEYJGQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.697997236 (15h5m39.442054151s) [seg_pos={0 0}, upload=15533]: Segment reached optimal threshold of 14 pieces. Aug 8 00:45:17.698203533 (15h5m39.442260383s) [noise=true , address=79.127.205.233:4633 , piece_id=F4RDTCFPIWZS4U65QMU5YXWOZSPPDIDMLJ26G3DCN3YV2UMAEFWA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EEYb4mCySvZ54qzmygCQ7hLyTGYpdNJMyrvPvHJjtfGos9usk): protocol: expected piece hash; context canceled; EOF Aug 8 00:45:17.698290533 (15h5m39.442347383s) [noise=true , address=64.49.13.140:4515 , piece_id=AQTWEM35E7E6YUISEOQGTPTPKD6RDOMKXLK267UILGD77273OY4A, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mY7sRFAeDLmKPqq8BgwnGtkdPTsDRR6soTBshsn4qhP7evARb): protocol: expected piece hash; context canceled; EOF Aug 8 00:45:17.698544485 (15h5m39.442601338s) [noise=true , address=104.36.82.23:4464 , piece_id=4HNT67XCPCNCECLCE36RVAHTJ7VBAADPQ7VMC7IASAUPVVEKIVWQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mLbGHntspYmsTVg8433HYSdAEMrXjGnDhmmvX1JmmgcxtdLgk): protocol: write tcp 139.59.144.22:39260->104.36.82.23:4464: use of closed network connection; write tcp 139.59.144.22:39260->104.36.82.23:4464: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39260->104.36.82.23:4464: use of closed network connection Aug 8 00:45:17.698701613 (15h5m39.442758474s) [noise=true , address=104.36.82.23:4452 , piece_id=B4XGMOK4RCAIOV3VSJXFZ73O52E4WIRUGSPVTCCJ4XUGNJHUQWBA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sAh1mLmzbtmc4o2FAZssW8Y7iDDNmNTCXdCS8xrH8PBVQqxG5): protocol: context canceled; context canceled Aug 8 00:45:17.698723379 (15h5m39.44278023s) [seg_pos={0 0}, upload=15533]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 00:45:17.698815435 (15h5m39.442872285s) [upload=15533]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:45:17.698820522 (15h5m39.442877383s) [upload=15533]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:45:17.860271466 (15h5m39.604328326s) [upload=15533]: Done uploading segments... Aug 8 00:45:17.860291088 (15h5m39.604347935s) [upload=15533]: Upload finished. err: <nil> Duration: 2.517020753s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...............................................X........X...] duration: 840.327113ms FAIL (2025-08-08 00:52:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 263481 (15h4m20.280320332s) [upload=15511]: Done uploading segments... Aug 8 00:43:58.536281876 (15h4m20.28033873s) [upload=15511]: Upload finished. err: <nil> Aug 8 00:45:01.015818127 (15h5m22.759874977s) [upload=15529]: Starting upload Aug 8 00:45:01.015874109 (15h5m22.759930972s) [upload=15529]: Done starting upload Aug 8 00:45:01.015956182 (15h5m22.76001303s) [upload=15529]: Uploading segments... Aug 8 00:45:01.016043908 (15h5m22.760100776s) [upload=15529]: Got next segment. Inline: true Aug 8 00:45:01.016049919 (15h5m22.760106782s) [upload=15529]: Waiting for error group managing segments... Aug 8 00:45:01.016115031 (15h5m22.760171895s) [upload=15529]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:45:01.016122390 (15h5m22.760179257s) [upload=15529]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:45:01.016125863 (15h5m22.760182727s) [upload=15529]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:45:01.173239782 (15h5m22.91729665s) [upload=15529]: Done uploading segments... Aug 8 00:45:01.173279478 (15h5m22.917336328s) [upload=15529]: Upload finished. err: <nil> Aug 8 00:46:00.429072612 (15h6m22.173129463s) [upload=15546]: Starting upload Aug 8 00:46:00.429156391 (15h6m22.173213278s) [upload=15546]: Done starting upload Aug 8 00:46:00.429267021 (15h6m22.173323882s) [upload=15546]: Uploading segments... Aug 8 00:46:00.429320811 (15h6m22.173377677s) [upload=15546]: Got next segment. Inline: true Aug 8 00:46:00.429326919 (15h6m22.17338377s) [upload=15546]: Waiting for error group managing segments... Aug 8 00:46:00.429377405 (15h6m22.173434256s) [upload=15546]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:46:00.429383461 (15h6m22.173440311s) [upload=15546]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:46:00.429387017 (15h6m22.173443877s) [upload=15546]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:46:00.684470407 (15h6m22.428527258s) [upload=15546]: Done uploading segments... Aug 8 00:46:00.684501074 (15h6m22.42855792s) [upload=15546]: Upload finished. err: <nil> Aug 8 00:47:14.991362846 (15h7m36.735419697s) [upload=15569]: Starting upload Aug 8 00:47:14.991436891 (15h7m36.735493754s) [upload=15569]: Done starting upload Aug 8 00:47:14.991521124 (15h7m36.735577974s) [upload=15569]: Uploading segments... Aug 8 00:47:14.991582184 (15h7m36.735639054s) [upload=15569]: Got next segment. Inline: true Aug 8 00:47:14.991588085 (15h7m36.735644945s) [upload=15569]: Waiting for error group managing segments... Aug 8 00:47:14.991641941 (15h7m36.735698801s) [upload=15569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:47:14.991647402 (15h7m36.735704266s) [upload=15569]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:47:14.991651037 (15h7m36.735707897s) [upload=15569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:47:15.149817056 (15h7m36.893873921s) [upload=15569]: Done uploading segments... Aug 8 00:47:15.149854826 (15h7m36.893911673s) [upload=15569]: Upload finished. err: <nil> Aug 8 00:48:05.184694323 (15h8m26.92875119s) [upload=15581]: Starting upload Aug 8 00:48:05.184775560 (15h8m26.928832424s) [upload=15581]: Done starting upload Aug 8 00:48:05.184860193 (15h8m26.928917046s) [upload=15581]: Uploading segments... Aug 8 00:48:05.184919879 (15h8m26.92897675s) [upload=15581]: Got next segment. Inline: true Aug 8 00:48:05.184927810 (15h8m26.928984686s) [upload=15581]: Waiting for error group managing segments... Aug 8 00:48:05.184989718 (15h8m26.929046568s) [upload=15581]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:48:05.184997233 (15h8m26.929054096s) [upload=15581]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:48:05.185002973 (15h8m26.929059837s) [upload=15581]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:48:05.333512154 (15h8m27.077569026s) [upload=15581]: Done uploading segments... Aug 8 00:48:05.333535225 (15h8m27.077592084s) [upload=15581]: Upload finished. err: <nil> Aug 8 00:48:25.959034104 (15h8m47.703090965s) [upload=15588]: Starting upload Aug 8 00:48:25.959129631 (15h8m47.703186501s) [upload=15588]: Done starting upload Aug 8 00:48:25.959263036 (15h8m47.703319897s) [upload=15588]: Uploading segments... Aug 8 00:48:25.959321571 (15h8m47.703378451s) [upload=15588]: Got next segment. Inline: true Aug 8 00:48:25.959328075 (15h8m47.70338494s) [upload=15588]: Waiting for error group managing segments... Aug 8 00:48:25.959379365 (15h8m47.70343623s) [upload=15588]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:48:25.959385760 (15h8m47.703442625s) [upload=15588]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:48:25.959389437 (15h8m47.703446299s) [upload=15588]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:48:26.117426083 (15h8m47.861482953s) [upload=15588]: Done uploading segments... Aug 8 00:48:26.117449715 (15h8m47.861506585s) [upload=15588]: Upload finished. err: <nil> Aug 8 00:49:18.380712745 (15h9m40.124769598s) [upload=15601]: Starting upload Aug 8 00:49:18.380774191 (15h9m40.124831063s) [upload=15601]: Done starting upload Aug 8 00:49:18.380853700 (15h9m40.124910555s) [upload=15601]: Uploading segments... Aug 8 00:49:18.380935089 (15h9m40.124991961s) [upload=15601]: Got next segment. Inline: true Aug 8 00:49:18.380942228 (15h9m40.124999082s) [upload=15601]: Waiting for error group managing segments... Aug 8 00:49:18.380990837 (15h9m40.125047687s) [upload=15601]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:49:18.380996570 (15h9m40.125053423s) [upload=15601]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:49:18.381000556 (15h9m40.125057404s) [upload=15601]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:49:18.524510041 (15h9m40.26856689s) [upload=15601]: Done uploading segments... Aug 8 00:49:18.524532847 (15h9m40.268589697s) [upload=15601]: Upload finished. err: <nil> Aug 8 00:50:14.219213519 (15h10m35.96327037s) [upload=15618]: Starting upload Aug 8 00:50:14.219268923 (15h10m35.963325784s) [upload=15618]: Done starting upload Aug 8 00:50:14.219367379 (15h10m35.963424233s) [upload=15618]: Uploading segments... Aug 8 00:50:14.219578059 (15h10m35.963634929s) [upload=15618]: Got next segment. Inline: true Aug 8 00:50:14.219614810 (15h10m35.963671673s) [upload=15618]: Waiting for error group managing segments... Aug 8 00:50:14.219687157 (15h10m35.963744017s) [upload=15618]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:50:14.219693611 (15h10m35.963750471s) [upload=15618]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:50:14.219699847 (15h10m35.96375671s) [upload=15618]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:50:14.385631211 (15h10m36.129688064s) [upload=15618]: Done uploading segments... Aug 8 00:50:14.385650912 (15h10m36.129707762s) [upload=15618]: Upload finished. err: <nil> Aug 8 00:50:44.080806747 (15h11m5.824863598s) [upload=15628]: Starting upload Aug 8 00:50:44.080911523 (15h11m5.824968393s) [upload=15628]: Done starting upload Aug 8 00:50:44.081032828 (15h11m5.825089678s) [upload=15628]: Uploading segments... Aug 8 00:50:44.081102196 (15h11m5.825159064s) [upload=15628]: Got next segment. Inline: true Aug 8 00:50:44.081110880 (15h11m5.82516774s) [upload=15628]: Waiting for error group managing segments... Aug 8 00:50:44.081162262 (15h11m5.825219175s) [upload=15628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:50:44.081168840 (15h11m5.825225704s) [upload=15628]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:50:44.081172385 (15h11m5.825229249s) [upload=15628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:50:44.261318102 (15h11m6.005374965s) [upload=15628]: Done uploading segments... Aug 8 00:50:44.261339384 (15h11m6.005396236s) [upload=15628]: Upload finished. err: <nil> Aug 8 00:51:00.984479825 (15h11m22.728536675s) [upload=15634]: Starting upload Aug 8 00:51:00.984537080 (15h11m22.72859393s) [upload=15634]: Done starting upload Aug 8 00:51:00.984618934 (15h11m22.728675798s) [upload=15634]: Uploading segments... Aug 8 00:51:00.984689929 (15h11m22.728746817s) [upload=15634]: Got next segment. Inline: true Aug 8 00:51:00.984696447 (15h11m22.728753308s) [upload=15634]: Waiting for error group managing segments... Aug 8 00:51:00.984795333 (15h11m22.728852187s) [upload=15634]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:51:00.984801510 (15h11m22.72885837s) [upload=15634]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:51:00.984804953 (15h11m22.728861813s) [upload=15634]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:51:01.179830418 (15h11m22.923887268s) [upload=15634]: Done uploading segments... Aug 8 00:51:01.179852239 (15h11m22.923909093s) [upload=15634]: Upload finished. err: <nil> Aug 8 00:51:18.788113821 (15h11m40.532170668s) [upload=15636]: Starting upload Aug 8 00:51:18.788190945 (15h11m40.53224781s) [upload=15636]: Done starting upload Aug 8 00:51:18.788266338 (15h11m40.532323198s) [upload=15636]: Uploading segments... Aug 8 00:51:18.788334958 (15h11m40.532391824s) [upload=15636]: Got next segment. Inline: true Aug 8 00:51:18.788341211 (15h11m40.532398065s) [upload=15636]: Waiting for error group managing segments... Aug 8 00:51:18.788385583 (15h11m40.532442429s) [upload=15636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:51:18.788390526 (15h11m40.532447372s) [upload=15636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:51:18.788393111 (15h11m40.532449957s) [upload=15636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:51:18.876506251 (15h11m40.620563101s) [upload=15636]: Done uploading segments... Aug 8 00:51:18.876525344 (15h11m40.620582195s) [upload=15636]: Upload finished. err: metaclient: Too Many Requests Duration: 840.373927ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......X...........................................X........] duration: 2.632678778s FAIL (2025-08-08 00:52:24 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Done starting upload Aug 8 00:45:16.266121630 (15h5m38.01017848s) [upload=15533]: Uploading segments... Aug 8 00:45:16.266428624 (15h5m38.010485484s) [upload=15533]: Got next segment. Inline: false Aug 8 00:45:16.266497281 (15h5m38.010554191s) [upload=15533]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:45:16.266512043 (15h5m38.010568914s) [upload=15533]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:45:16.467813786 (15h5m38.211870647s) [seg_pos={0 0}, upload=15533]: Begin upload segment... Aug 8 00:45:16.467883230 (15h5m38.211940081s) [seg_pos={0 0}, upload=15533]: Done begin upload segment. Aug 8 00:45:16.467935054 (15h5m38.211991969s) [noise=true , address=104.36.82.60:4369 , piece_id=WVPSCPUKKSYS4T3JIEEVDTS7WSFYXNGVQA6QNLGO3BKC3EKEYJGQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.467945924 (15h5m38.212002789s) [noise=true , address=104.36.82.54:4192 , piece_id=FDJHGRVMHAFB5JZM4Z77MF3NHRU3T4FNQPUAOXGP5NRCM5Q7RVRA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468440400 (15h5m38.212497272s) [noise=true , address=104.36.82.34:4577 , piece_id=4PTCNUDROG6OXW5DBUEMBAGPEMEMVLIW5OXRP2LPC55J2PTK3PAA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468457544 (15h5m38.21251441s) [noise=true , address=104.36.82.44:4887 , piece_id=FC4PG4NZJHR35DZLGLFZ6EJZYA2MJWLIDPBYHZCNJKJI4OVX5G4Q, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468847577 (15h5m38.212904441s) [noise=true , address=104.36.82.62:4430 , piece_id=JGWQSS5KYYCBOITFMENDLSGK6XSDVOT7TMRGJBZ55KBXX54ENY6A, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468887696 (15h5m38.21294456s) [noise=true , address=79.127.205.233:4633 , piece_id=F4RDTCFPIWZS4U65QMU5YXWOZSPPDIDMLJ26G3DCN3YV2UMAEFWA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469228336 (15h5m38.2132852s) [noise=true , address=104.36.82.43:4846 , piece_id=73PPCVC3HGFJDIB63URF7XA6PC37VCYLBCOMTSSFNC5UYZR6M5YA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469268209 (15h5m38.213325082s) [noise=true , address=64.49.13.140:4515 , piece_id=AQTWEM35E7E6YUISEOQGTPTPKD6RDOMKXLK267UILGD77273OY4A, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469613163 (15h5m38.21367003s) [noise=true , address=104.36.82.16:4307 , piece_id=B7ZUFMTGVRPCNJSTBCXMQRZW2AGKPXM2LJRDLYZWPXQ4F5JJ75UQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469702757 (15h5m38.213759621s) [noise=true , address=104.36.82.9:4038 , piece_id=RVXFCLSREWSON62WSTYVISC52HN33VUN2BHJ7UR72MEBMTFKQV6A, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470036233 (15h5m38.214093098s) [noise=true , address=104.36.82.23:4464 , piece_id=4HNT67XCPCNCECLCE36RVAHTJ7VBAADPQ7VMC7IASAUPVVEKIVWQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470129321 (15h5m38.214186184s) [noise=true , address=45.91.116.6:4982 , piece_id=NV3WES7XAJBCNO5EQMWFK6SNUEYE63YKTQC5ZMRF7URP6SX3FXGA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470411978 (15h5m38.214468842s) [noise=true , address=104.36.82.23:4452 , piece_id=B4XGMOK4RCAIOV3VSJXFZ73O52E4WIRUGSPVTCCJ4XUGNJHUQWBA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470482224 (15h5m38.21453909s) [noise=true , address=104.36.82.14:4254 , piece_id=AIB65II65FK6M3EQHX6PVX27N4Q4BQKI5QGVP3PD6D34ZKXVCLZA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470792753 (15h5m38.214849603s) [noise=true , address=104.36.82.19:4405 , piece_id=SA2CPGDA75CYSUDYYZ6V4NKTG5AEIZC7SVYNL2OLBK3E5MSK5FQQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470877563 (15h5m38.214934486s) [noise=true , address=104.36.82.14:4243 , piece_id=HSR3LCZHIDLJSJ7WISDYOWGPLBFZ73MIXTR7MLFQ2TLUBYLRL4DQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.471502176 (15h5m38.215559147s) [noise=true , address=104.36.82.45:4910 , piece_id=OGS6W5AGDOACXIMZ4OH3GNQ43YUU6WOYJHKAHNMHJZBQPJUVVCRA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.471850592 (15h5m38.215907452s) [noise=true , address=79.127.201.215:4358 , piece_id=Z3PRSJIAQMKCL2ZU5TA7WZ2IBCVIXB2K5UJBMS3UCI5H6HDNTWBA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:17.050924903 (15h5m38.794981774s) [upload=15533]: Next returned nil segment Aug 8 00:45:17.051086092 (15h5m38.795142971s) [upload=15533]: Waiting for error group managing segments... Aug 8 00:45:17.225555422 (15h5m38.969612281s) [noise=true , address=104.36.82.62:4430 , piece_id=JGWQSS5KYYCBOITFMENDLSGK6XSDVOT7TMRGJBZ55KBXX54ENY6A, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.232285957 (15h5m38.97634281s) [noise=true , address=45.91.116.6:4982 , piece_id=NV3WES7XAJBCNO5EQMWFK6SNUEYE63YKTQC5ZMRF7URP6SX3FXGA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.239931943 (15h5m38.983988797s) [noise=true , address=104.36.82.45:4910 , piece_id=OGS6W5AGDOACXIMZ4OH3GNQ43YUU6WOYJHKAHNMHJZBQPJUVVCRA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.240442032 (15h5m38.984498886s) [noise=true , address=104.36.82.34:4577 , piece_id=4PTCNUDROG6OXW5DBUEMBAGPEMEMVLIW5OXRP2LPC55J2PTK3PAA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.280384119 (15h5m39.024440979s) [noise=true , address=104.36.82.19:4405 , piece_id=SA2CPGDA75CYSUDYYZ6V4NKTG5AEIZC7SVYNL2OLBK3E5MSK5FQQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.284571751 (15h5m39.028628621s) [noise=true , address=104.36.82.16:4307 , piece_id=B7ZUFMTGVRPCNJSTBCXMQRZW2AGKPXM2LJRDLYZWPXQ4F5JJ75UQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.295563563 (15h5m39.03962043s) [noise=true , address=104.36.82.14:4254 , piece_id=AIB65II65FK6M3EQHX6PVX27N4Q4BQKI5QGVP3PD6D34ZKXVCLZA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.367859627 (15h5m39.11191649s) [noise=true , address=104.36.82.44:4887 , piece_id=FC4PG4NZJHR35DZLGLFZ6EJZYA2MJWLIDPBYHZCNJKJI4OVX5G4Q, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.486413539 (15h5m39.2304704s) [noise=true , address=104.36.82.9:4038 , piece_id=RVXFCLSREWSON62WSTYVISC52HN33VUN2BHJ7UR72MEBMTFKQV6A, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.510192603 (15h5m39.254249451s) [noise=true , address=79.127.201.215:4358 , piece_id=Z3PRSJIAQMKCL2ZU5TA7WZ2IBCVIXB2K5UJBMS3UCI5H6HDNTWBA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.609092139 (15h5m39.353148988s) [noise=true , address=104.36.82.54:4192 , piece_id=FDJHGRVMHAFB5JZM4Z77MF3NHRU3T4FNQPUAOXGP5NRCM5Q7RVRA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.647464269 (15h5m39.391521122s) [noise=true , address=104.36.82.14:4243 , piece_id=HSR3LCZHIDLJSJ7WISDYOWGPLBFZ73MIXTR7MLFQ2TLUBYLRL4DQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.681479215 (15h5m39.425536078s) [noise=true , address=104.36.82.43:4846 , piece_id=73PPCVC3HGFJDIB63URF7XA6PC37VCYLBCOMTSSFNC5UYZR6M5YA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.697970794 (15h5m39.442027648s) [noise=true , address=104.36.82.60:4369 , piece_id=WVPSCPUKKSYS4T3JIEEVDTS7WSFYXNGVQA6QNLGO3BKC3EKEYJGQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.697997236 (15h5m39.442054151s) [seg_pos={0 0}, upload=15533]: Segment reached optimal threshold of 14 pieces. Aug 8 00:45:17.698203533 (15h5m39.442260383s) [noise=true , address=79.127.205.233:4633 , piece_id=F4RDTCFPIWZS4U65QMU5YXWOZSPPDIDMLJ26G3DCN3YV2UMAEFWA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EEYb4mCySvZ54qzmygCQ7hLyTGYpdNJMyrvPvHJjtfGos9usk): protocol: expected piece hash; context canceled; EOF Aug 8 00:45:17.698290533 (15h5m39.442347383s) [noise=true , address=64.49.13.140:4515 , piece_id=AQTWEM35E7E6YUISEOQGTPTPKD6RDOMKXLK267UILGD77273OY4A, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mY7sRFAeDLmKPqq8BgwnGtkdPTsDRR6soTBshsn4qhP7evARb): protocol: expected piece hash; context canceled; EOF Aug 8 00:45:17.698544485 (15h5m39.442601338s) [noise=true , address=104.36.82.23:4464 , piece_id=4HNT67XCPCNCECLCE36RVAHTJ7VBAADPQ7VMC7IASAUPVVEKIVWQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mLbGHntspYmsTVg8433HYSdAEMrXjGnDhmmvX1JmmgcxtdLgk): protocol: write tcp 139.59.144.22:39260->104.36.82.23:4464: use of closed network connection; write tcp 139.59.144.22:39260->104.36.82.23:4464: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39260->104.36.82.23:4464: use of closed network connection Aug 8 00:45:17.698701613 (15h5m39.442758474s) [noise=true , address=104.36.82.23:4452 , piece_id=B4XGMOK4RCAIOV3VSJXFZ73O52E4WIRUGSPVTCCJ4XUGNJHUQWBA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sAh1mLmzbtmc4o2FAZssW8Y7iDDNmNTCXdCS8xrH8PBVQqxG5): protocol: context canceled; context canceled Aug 8 00:45:17.698723379 (15h5m39.44278023s) [seg_pos={0 0}, upload=15533]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 00:45:17.698815435 (15h5m39.442872285s) [upload=15533]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:45:17.698820522 (15h5m39.442877383s) [upload=15533]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:45:17.860271466 (15h5m39.604328326s) [upload=15533]: Done uploading segments... Aug 8 00:45:17.860291088 (15h5m39.604347935s) [upload=15533]: Upload finished. err: <nil> Duration: 2.632749234s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [................................................X........X..] duration: 905.048925ms FAIL (2025-08-08 00:52:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 263481 (15h4m20.280320332s) [upload=15511]: Done uploading segments... Aug 8 00:43:58.536281876 (15h4m20.28033873s) [upload=15511]: Upload finished. err: <nil> Aug 8 00:45:01.015818127 (15h5m22.759874977s) [upload=15529]: Starting upload Aug 8 00:45:01.015874109 (15h5m22.759930972s) [upload=15529]: Done starting upload Aug 8 00:45:01.015956182 (15h5m22.76001303s) [upload=15529]: Uploading segments... Aug 8 00:45:01.016043908 (15h5m22.760100776s) [upload=15529]: Got next segment. Inline: true Aug 8 00:45:01.016049919 (15h5m22.760106782s) [upload=15529]: Waiting for error group managing segments... Aug 8 00:45:01.016115031 (15h5m22.760171895s) [upload=15529]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:45:01.016122390 (15h5m22.760179257s) [upload=15529]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:45:01.016125863 (15h5m22.760182727s) [upload=15529]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:45:01.173239782 (15h5m22.91729665s) [upload=15529]: Done uploading segments... Aug 8 00:45:01.173279478 (15h5m22.917336328s) [upload=15529]: Upload finished. err: <nil> Aug 8 00:46:00.429072612 (15h6m22.173129463s) [upload=15546]: Starting upload Aug 8 00:46:00.429156391 (15h6m22.173213278s) [upload=15546]: Done starting upload Aug 8 00:46:00.429267021 (15h6m22.173323882s) [upload=15546]: Uploading segments... Aug 8 00:46:00.429320811 (15h6m22.173377677s) [upload=15546]: Got next segment. Inline: true Aug 8 00:46:00.429326919 (15h6m22.17338377s) [upload=15546]: Waiting for error group managing segments... Aug 8 00:46:00.429377405 (15h6m22.173434256s) [upload=15546]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:46:00.429383461 (15h6m22.173440311s) [upload=15546]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:46:00.429387017 (15h6m22.173443877s) [upload=15546]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:46:00.684470407 (15h6m22.428527258s) [upload=15546]: Done uploading segments... Aug 8 00:46:00.684501074 (15h6m22.42855792s) [upload=15546]: Upload finished. err: <nil> Aug 8 00:47:14.991362846 (15h7m36.735419697s) [upload=15569]: Starting upload Aug 8 00:47:14.991436891 (15h7m36.735493754s) [upload=15569]: Done starting upload Aug 8 00:47:14.991521124 (15h7m36.735577974s) [upload=15569]: Uploading segments... Aug 8 00:47:14.991582184 (15h7m36.735639054s) [upload=15569]: Got next segment. Inline: true Aug 8 00:47:14.991588085 (15h7m36.735644945s) [upload=15569]: Waiting for error group managing segments... Aug 8 00:47:14.991641941 (15h7m36.735698801s) [upload=15569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:47:14.991647402 (15h7m36.735704266s) [upload=15569]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:47:14.991651037 (15h7m36.735707897s) [upload=15569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:47:15.149817056 (15h7m36.893873921s) [upload=15569]: Done uploading segments... Aug 8 00:47:15.149854826 (15h7m36.893911673s) [upload=15569]: Upload finished. err: <nil> Aug 8 00:48:05.184694323 (15h8m26.92875119s) [upload=15581]: Starting upload Aug 8 00:48:05.184775560 (15h8m26.928832424s) [upload=15581]: Done starting upload Aug 8 00:48:05.184860193 (15h8m26.928917046s) [upload=15581]: Uploading segments... Aug 8 00:48:05.184919879 (15h8m26.92897675s) [upload=15581]: Got next segment. Inline: true Aug 8 00:48:05.184927810 (15h8m26.928984686s) [upload=15581]: Waiting for error group managing segments... Aug 8 00:48:05.184989718 (15h8m26.929046568s) [upload=15581]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:48:05.184997233 (15h8m26.929054096s) [upload=15581]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:48:05.185002973 (15h8m26.929059837s) [upload=15581]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:48:05.333512154 (15h8m27.077569026s) [upload=15581]: Done uploading segments... Aug 8 00:48:05.333535225 (15h8m27.077592084s) [upload=15581]: Upload finished. err: <nil> Aug 8 00:48:25.959034104 (15h8m47.703090965s) [upload=15588]: Starting upload Aug 8 00:48:25.959129631 (15h8m47.703186501s) [upload=15588]: Done starting upload Aug 8 00:48:25.959263036 (15h8m47.703319897s) [upload=15588]: Uploading segments... Aug 8 00:48:25.959321571 (15h8m47.703378451s) [upload=15588]: Got next segment. Inline: true Aug 8 00:48:25.959328075 (15h8m47.70338494s) [upload=15588]: Waiting for error group managing segments... Aug 8 00:48:25.959379365 (15h8m47.70343623s) [upload=15588]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:48:25.959385760 (15h8m47.703442625s) [upload=15588]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:48:25.959389437 (15h8m47.703446299s) [upload=15588]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:48:26.117426083 (15h8m47.861482953s) [upload=15588]: Done uploading segments... Aug 8 00:48:26.117449715 (15h8m47.861506585s) [upload=15588]: Upload finished. err: <nil> Aug 8 00:49:18.380712745 (15h9m40.124769598s) [upload=15601]: Starting upload Aug 8 00:49:18.380774191 (15h9m40.124831063s) [upload=15601]: Done starting upload Aug 8 00:49:18.380853700 (15h9m40.124910555s) [upload=15601]: Uploading segments... Aug 8 00:49:18.380935089 (15h9m40.124991961s) [upload=15601]: Got next segment. Inline: true Aug 8 00:49:18.380942228 (15h9m40.124999082s) [upload=15601]: Waiting for error group managing segments... Aug 8 00:49:18.380990837 (15h9m40.125047687s) [upload=15601]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:49:18.380996570 (15h9m40.125053423s) [upload=15601]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:49:18.381000556 (15h9m40.125057404s) [upload=15601]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:49:18.524510041 (15h9m40.26856689s) [upload=15601]: Done uploading segments... Aug 8 00:49:18.524532847 (15h9m40.268589697s) [upload=15601]: Upload finished. err: <nil> Aug 8 00:50:14.219213519 (15h10m35.96327037s) [upload=15618]: Starting upload Aug 8 00:50:14.219268923 (15h10m35.963325784s) [upload=15618]: Done starting upload Aug 8 00:50:14.219367379 (15h10m35.963424233s) [upload=15618]: Uploading segments... Aug 8 00:50:14.219578059 (15h10m35.963634929s) [upload=15618]: Got next segment. Inline: true Aug 8 00:50:14.219614810 (15h10m35.963671673s) [upload=15618]: Waiting for error group managing segments... Aug 8 00:50:14.219687157 (15h10m35.963744017s) [upload=15618]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:50:14.219693611 (15h10m35.963750471s) [upload=15618]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:50:14.219699847 (15h10m35.96375671s) [upload=15618]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:50:14.385631211 (15h10m36.129688064s) [upload=15618]: Done uploading segments... Aug 8 00:50:14.385650912 (15h10m36.129707762s) [upload=15618]: Upload finished. err: <nil> Aug 8 00:50:44.080806747 (15h11m5.824863598s) [upload=15628]: Starting upload Aug 8 00:50:44.080911523 (15h11m5.824968393s) [upload=15628]: Done starting upload Aug 8 00:50:44.081032828 (15h11m5.825089678s) [upload=15628]: Uploading segments... Aug 8 00:50:44.081102196 (15h11m5.825159064s) [upload=15628]: Got next segment. Inline: true Aug 8 00:50:44.081110880 (15h11m5.82516774s) [upload=15628]: Waiting for error group managing segments... Aug 8 00:50:44.081162262 (15h11m5.825219175s) [upload=15628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:50:44.081168840 (15h11m5.825225704s) [upload=15628]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:50:44.081172385 (15h11m5.825229249s) [upload=15628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:50:44.261318102 (15h11m6.005374965s) [upload=15628]: Done uploading segments... Aug 8 00:50:44.261339384 (15h11m6.005396236s) [upload=15628]: Upload finished. err: <nil> Aug 8 00:51:00.984479825 (15h11m22.728536675s) [upload=15634]: Starting upload Aug 8 00:51:00.984537080 (15h11m22.72859393s) [upload=15634]: Done starting upload Aug 8 00:51:00.984618934 (15h11m22.728675798s) [upload=15634]: Uploading segments... Aug 8 00:51:00.984689929 (15h11m22.728746817s) [upload=15634]: Got next segment. Inline: true Aug 8 00:51:00.984696447 (15h11m22.728753308s) [upload=15634]: Waiting for error group managing segments... Aug 8 00:51:00.984795333 (15h11m22.728852187s) [upload=15634]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:51:00.984801510 (15h11m22.72885837s) [upload=15634]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:51:00.984804953 (15h11m22.728861813s) [upload=15634]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:51:01.179830418 (15h11m22.923887268s) [upload=15634]: Done uploading segments... Aug 8 00:51:01.179852239 (15h11m22.923909093s) [upload=15634]: Upload finished. err: <nil> Aug 8 00:51:18.788113821 (15h11m40.532170668s) [upload=15636]: Starting upload Aug 8 00:51:18.788190945 (15h11m40.53224781s) [upload=15636]: Done starting upload Aug 8 00:51:18.788266338 (15h11m40.532323198s) [upload=15636]: Uploading segments... Aug 8 00:51:18.788334958 (15h11m40.532391824s) [upload=15636]: Got next segment. Inline: true Aug 8 00:51:18.788341211 (15h11m40.532398065s) [upload=15636]: Waiting for error group managing segments... Aug 8 00:51:18.788385583 (15h11m40.532442429s) [upload=15636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:51:18.788390526 (15h11m40.532447372s) [upload=15636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:51:18.788393111 (15h11m40.532449957s) [upload=15636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:51:18.876506251 (15h11m40.620563101s) [upload=15636]: Done uploading segments... Aug 8 00:51:18.876525344 (15h11m40.620582195s) [upload=15636]: Upload finished. err: metaclient: Too Many Requests Duration: 905.081308ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.................................................X........X.] duration: 1.040368413s FAIL (2025-08-08 00:51:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 263481 (15h4m20.280320332s) [upload=15511]: Done uploading segments... Aug 8 00:43:58.536281876 (15h4m20.28033873s) [upload=15511]: Upload finished. err: <nil> Aug 8 00:45:01.015818127 (15h5m22.759874977s) [upload=15529]: Starting upload Aug 8 00:45:01.015874109 (15h5m22.759930972s) [upload=15529]: Done starting upload Aug 8 00:45:01.015956182 (15h5m22.76001303s) [upload=15529]: Uploading segments... Aug 8 00:45:01.016043908 (15h5m22.760100776s) [upload=15529]: Got next segment. Inline: true Aug 8 00:45:01.016049919 (15h5m22.760106782s) [upload=15529]: Waiting for error group managing segments... Aug 8 00:45:01.016115031 (15h5m22.760171895s) [upload=15529]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:45:01.016122390 (15h5m22.760179257s) [upload=15529]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:45:01.016125863 (15h5m22.760182727s) [upload=15529]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:45:01.173239782 (15h5m22.91729665s) [upload=15529]: Done uploading segments... Aug 8 00:45:01.173279478 (15h5m22.917336328s) [upload=15529]: Upload finished. err: <nil> Aug 8 00:46:00.429072612 (15h6m22.173129463s) [upload=15546]: Starting upload Aug 8 00:46:00.429156391 (15h6m22.173213278s) [upload=15546]: Done starting upload Aug 8 00:46:00.429267021 (15h6m22.173323882s) [upload=15546]: Uploading segments... Aug 8 00:46:00.429320811 (15h6m22.173377677s) [upload=15546]: Got next segment. Inline: true Aug 8 00:46:00.429326919 (15h6m22.17338377s) [upload=15546]: Waiting for error group managing segments... Aug 8 00:46:00.429377405 (15h6m22.173434256s) [upload=15546]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:46:00.429383461 (15h6m22.173440311s) [upload=15546]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:46:00.429387017 (15h6m22.173443877s) [upload=15546]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:46:00.684470407 (15h6m22.428527258s) [upload=15546]: Done uploading segments... Aug 8 00:46:00.684501074 (15h6m22.42855792s) [upload=15546]: Upload finished. err: <nil> Aug 8 00:47:14.991362846 (15h7m36.735419697s) [upload=15569]: Starting upload Aug 8 00:47:14.991436891 (15h7m36.735493754s) [upload=15569]: Done starting upload Aug 8 00:47:14.991521124 (15h7m36.735577974s) [upload=15569]: Uploading segments... Aug 8 00:47:14.991582184 (15h7m36.735639054s) [upload=15569]: Got next segment. Inline: true Aug 8 00:47:14.991588085 (15h7m36.735644945s) [upload=15569]: Waiting for error group managing segments... Aug 8 00:47:14.991641941 (15h7m36.735698801s) [upload=15569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:47:14.991647402 (15h7m36.735704266s) [upload=15569]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:47:14.991651037 (15h7m36.735707897s) [upload=15569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:47:15.149817056 (15h7m36.893873921s) [upload=15569]: Done uploading segments... Aug 8 00:47:15.149854826 (15h7m36.893911673s) [upload=15569]: Upload finished. err: <nil> Aug 8 00:48:05.184694323 (15h8m26.92875119s) [upload=15581]: Starting upload Aug 8 00:48:05.184775560 (15h8m26.928832424s) [upload=15581]: Done starting upload Aug 8 00:48:05.184860193 (15h8m26.928917046s) [upload=15581]: Uploading segments... Aug 8 00:48:05.184919879 (15h8m26.92897675s) [upload=15581]: Got next segment. Inline: true Aug 8 00:48:05.184927810 (15h8m26.928984686s) [upload=15581]: Waiting for error group managing segments... Aug 8 00:48:05.184989718 (15h8m26.929046568s) [upload=15581]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:48:05.184997233 (15h8m26.929054096s) [upload=15581]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:48:05.185002973 (15h8m26.929059837s) [upload=15581]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:48:05.333512154 (15h8m27.077569026s) [upload=15581]: Done uploading segments... Aug 8 00:48:05.333535225 (15h8m27.077592084s) [upload=15581]: Upload finished. err: <nil> Aug 8 00:48:25.959034104 (15h8m47.703090965s) [upload=15588]: Starting upload Aug 8 00:48:25.959129631 (15h8m47.703186501s) [upload=15588]: Done starting upload Aug 8 00:48:25.959263036 (15h8m47.703319897s) [upload=15588]: Uploading segments... Aug 8 00:48:25.959321571 (15h8m47.703378451s) [upload=15588]: Got next segment. Inline: true Aug 8 00:48:25.959328075 (15h8m47.70338494s) [upload=15588]: Waiting for error group managing segments... Aug 8 00:48:25.959379365 (15h8m47.70343623s) [upload=15588]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:48:25.959385760 (15h8m47.703442625s) [upload=15588]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:48:25.959389437 (15h8m47.703446299s) [upload=15588]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:48:26.117426083 (15h8m47.861482953s) [upload=15588]: Done uploading segments... Aug 8 00:48:26.117449715 (15h8m47.861506585s) [upload=15588]: Upload finished. err: <nil> Aug 8 00:49:18.380712745 (15h9m40.124769598s) [upload=15601]: Starting upload Aug 8 00:49:18.380774191 (15h9m40.124831063s) [upload=15601]: Done starting upload Aug 8 00:49:18.380853700 (15h9m40.124910555s) [upload=15601]: Uploading segments... Aug 8 00:49:18.380935089 (15h9m40.124991961s) [upload=15601]: Got next segment. Inline: true Aug 8 00:49:18.380942228 (15h9m40.124999082s) [upload=15601]: Waiting for error group managing segments... Aug 8 00:49:18.380990837 (15h9m40.125047687s) [upload=15601]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:49:18.380996570 (15h9m40.125053423s) [upload=15601]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:49:18.381000556 (15h9m40.125057404s) [upload=15601]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:49:18.524510041 (15h9m40.26856689s) [upload=15601]: Done uploading segments... Aug 8 00:49:18.524532847 (15h9m40.268589697s) [upload=15601]: Upload finished. err: <nil> Aug 8 00:50:14.219213519 (15h10m35.96327037s) [upload=15618]: Starting upload Aug 8 00:50:14.219268923 (15h10m35.963325784s) [upload=15618]: Done starting upload Aug 8 00:50:14.219367379 (15h10m35.963424233s) [upload=15618]: Uploading segments... Aug 8 00:50:14.219578059 (15h10m35.963634929s) [upload=15618]: Got next segment. Inline: true Aug 8 00:50:14.219614810 (15h10m35.963671673s) [upload=15618]: Waiting for error group managing segments... Aug 8 00:50:14.219687157 (15h10m35.963744017s) [upload=15618]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:50:14.219693611 (15h10m35.963750471s) [upload=15618]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:50:14.219699847 (15h10m35.96375671s) [upload=15618]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:50:14.385631211 (15h10m36.129688064s) [upload=15618]: Done uploading segments... Aug 8 00:50:14.385650912 (15h10m36.129707762s) [upload=15618]: Upload finished. err: <nil> Aug 8 00:50:44.080806747 (15h11m5.824863598s) [upload=15628]: Starting upload Aug 8 00:50:44.080911523 (15h11m5.824968393s) [upload=15628]: Done starting upload Aug 8 00:50:44.081032828 (15h11m5.825089678s) [upload=15628]: Uploading segments... Aug 8 00:50:44.081102196 (15h11m5.825159064s) [upload=15628]: Got next segment. Inline: true Aug 8 00:50:44.081110880 (15h11m5.82516774s) [upload=15628]: Waiting for error group managing segments... Aug 8 00:50:44.081162262 (15h11m5.825219175s) [upload=15628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:50:44.081168840 (15h11m5.825225704s) [upload=15628]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:50:44.081172385 (15h11m5.825229249s) [upload=15628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:50:44.261318102 (15h11m6.005374965s) [upload=15628]: Done uploading segments... Aug 8 00:50:44.261339384 (15h11m6.005396236s) [upload=15628]: Upload finished. err: <nil> Aug 8 00:51:00.984479825 (15h11m22.728536675s) [upload=15634]: Starting upload Aug 8 00:51:00.984537080 (15h11m22.72859393s) [upload=15634]: Done starting upload Aug 8 00:51:00.984618934 (15h11m22.728675798s) [upload=15634]: Uploading segments... Aug 8 00:51:00.984689929 (15h11m22.728746817s) [upload=15634]: Got next segment. Inline: true Aug 8 00:51:00.984696447 (15h11m22.728753308s) [upload=15634]: Waiting for error group managing segments... Aug 8 00:51:00.984795333 (15h11m22.728852187s) [upload=15634]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:51:00.984801510 (15h11m22.72885837s) [upload=15634]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:51:00.984804953 (15h11m22.728861813s) [upload=15634]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:51:01.179830418 (15h11m22.923887268s) [upload=15634]: Done uploading segments... Aug 8 00:51:01.179852239 (15h11m22.923909093s) [upload=15634]: Upload finished. err: <nil> Aug 8 00:51:18.788113821 (15h11m40.532170668s) [upload=15636]: Starting upload Aug 8 00:51:18.788190945 (15h11m40.53224781s) [upload=15636]: Done starting upload Aug 8 00:51:18.788266338 (15h11m40.532323198s) [upload=15636]: Uploading segments... Aug 8 00:51:18.788334958 (15h11m40.532391824s) [upload=15636]: Got next segment. Inline: true Aug 8 00:51:18.788341211 (15h11m40.532398065s) [upload=15636]: Waiting for error group managing segments... Aug 8 00:51:18.788385583 (15h11m40.532442429s) [upload=15636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:51:18.788390526 (15h11m40.532447372s) [upload=15636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:51:18.788393111 (15h11m40.532449957s) [upload=15636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:51:18.876506251 (15h11m40.620563101s) [upload=15636]: Done uploading segments... Aug 8 00:51:18.876525344 (15h11m40.620582195s) [upload=15636]: Upload finished. err: metaclient: Too Many Requests Duration: 1.040396046s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [........X...........................................X.......] duration: 3.302830969s FAIL (2025-08-08 00:51:41 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Done starting upload Aug 8 00:45:16.266121630 (15h5m38.01017848s) [upload=15533]: Uploading segments... Aug 8 00:45:16.266428624 (15h5m38.010485484s) [upload=15533]: Got next segment. Inline: false Aug 8 00:45:16.266497281 (15h5m38.010554191s) [upload=15533]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:45:16.266512043 (15h5m38.010568914s) [upload=15533]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:45:16.467813786 (15h5m38.211870647s) [seg_pos={0 0}, upload=15533]: Begin upload segment... Aug 8 00:45:16.467883230 (15h5m38.211940081s) [seg_pos={0 0}, upload=15533]: Done begin upload segment. Aug 8 00:45:16.467935054 (15h5m38.211991969s) [noise=true , address=104.36.82.60:4369 , piece_id=WVPSCPUKKSYS4T3JIEEVDTS7WSFYXNGVQA6QNLGO3BKC3EKEYJGQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.467945924 (15h5m38.212002789s) [noise=true , address=104.36.82.54:4192 , piece_id=FDJHGRVMHAFB5JZM4Z77MF3NHRU3T4FNQPUAOXGP5NRCM5Q7RVRA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468440400 (15h5m38.212497272s) [noise=true , address=104.36.82.34:4577 , piece_id=4PTCNUDROG6OXW5DBUEMBAGPEMEMVLIW5OXRP2LPC55J2PTK3PAA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468457544 (15h5m38.21251441s) [noise=true , address=104.36.82.44:4887 , piece_id=FC4PG4NZJHR35DZLGLFZ6EJZYA2MJWLIDPBYHZCNJKJI4OVX5G4Q, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468847577 (15h5m38.212904441s) [noise=true , address=104.36.82.62:4430 , piece_id=JGWQSS5KYYCBOITFMENDLSGK6XSDVOT7TMRGJBZ55KBXX54ENY6A, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468887696 (15h5m38.21294456s) [noise=true , address=79.127.205.233:4633 , piece_id=F4RDTCFPIWZS4U65QMU5YXWOZSPPDIDMLJ26G3DCN3YV2UMAEFWA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469228336 (15h5m38.2132852s) [noise=true , address=104.36.82.43:4846 , piece_id=73PPCVC3HGFJDIB63URF7XA6PC37VCYLBCOMTSSFNC5UYZR6M5YA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469268209 (15h5m38.213325082s) [noise=true , address=64.49.13.140:4515 , piece_id=AQTWEM35E7E6YUISEOQGTPTPKD6RDOMKXLK267UILGD77273OY4A, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469613163 (15h5m38.21367003s) [noise=true , address=104.36.82.16:4307 , piece_id=B7ZUFMTGVRPCNJSTBCXMQRZW2AGKPXM2LJRDLYZWPXQ4F5JJ75UQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469702757 (15h5m38.213759621s) [noise=true , address=104.36.82.9:4038 , piece_id=RVXFCLSREWSON62WSTYVISC52HN33VUN2BHJ7UR72MEBMTFKQV6A, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470036233 (15h5m38.214093098s) [noise=true , address=104.36.82.23:4464 , piece_id=4HNT67XCPCNCECLCE36RVAHTJ7VBAADPQ7VMC7IASAUPVVEKIVWQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470129321 (15h5m38.214186184s) [noise=true , address=45.91.116.6:4982 , piece_id=NV3WES7XAJBCNO5EQMWFK6SNUEYE63YKTQC5ZMRF7URP6SX3FXGA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470411978 (15h5m38.214468842s) [noise=true , address=104.36.82.23:4452 , piece_id=B4XGMOK4RCAIOV3VSJXFZ73O52E4WIRUGSPVTCCJ4XUGNJHUQWBA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470482224 (15h5m38.21453909s) [noise=true , address=104.36.82.14:4254 , piece_id=AIB65II65FK6M3EQHX6PVX27N4Q4BQKI5QGVP3PD6D34ZKXVCLZA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470792753 (15h5m38.214849603s) [noise=true , address=104.36.82.19:4405 , piece_id=SA2CPGDA75CYSUDYYZ6V4NKTG5AEIZC7SVYNL2OLBK3E5MSK5FQQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470877563 (15h5m38.214934486s) [noise=true , address=104.36.82.14:4243 , piece_id=HSR3LCZHIDLJSJ7WISDYOWGPLBFZ73MIXTR7MLFQ2TLUBYLRL4DQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.471502176 (15h5m38.215559147s) [noise=true , address=104.36.82.45:4910 , piece_id=OGS6W5AGDOACXIMZ4OH3GNQ43YUU6WOYJHKAHNMHJZBQPJUVVCRA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.471850592 (15h5m38.215907452s) [noise=true , address=79.127.201.215:4358 , piece_id=Z3PRSJIAQMKCL2ZU5TA7WZ2IBCVIXB2K5UJBMS3UCI5H6HDNTWBA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:17.050924903 (15h5m38.794981774s) [upload=15533]: Next returned nil segment Aug 8 00:45:17.051086092 (15h5m38.795142971s) [upload=15533]: Waiting for error group managing segments... Aug 8 00:45:17.225555422 (15h5m38.969612281s) [noise=true , address=104.36.82.62:4430 , piece_id=JGWQSS5KYYCBOITFMENDLSGK6XSDVOT7TMRGJBZ55KBXX54ENY6A, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.232285957 (15h5m38.97634281s) [noise=true , address=45.91.116.6:4982 , piece_id=NV3WES7XAJBCNO5EQMWFK6SNUEYE63YKTQC5ZMRF7URP6SX3FXGA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.239931943 (15h5m38.983988797s) [noise=true , address=104.36.82.45:4910 , piece_id=OGS6W5AGDOACXIMZ4OH3GNQ43YUU6WOYJHKAHNMHJZBQPJUVVCRA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.240442032 (15h5m38.984498886s) [noise=true , address=104.36.82.34:4577 , piece_id=4PTCNUDROG6OXW5DBUEMBAGPEMEMVLIW5OXRP2LPC55J2PTK3PAA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.280384119 (15h5m39.024440979s) [noise=true , address=104.36.82.19:4405 , piece_id=SA2CPGDA75CYSUDYYZ6V4NKTG5AEIZC7SVYNL2OLBK3E5MSK5FQQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.284571751 (15h5m39.028628621s) [noise=true , address=104.36.82.16:4307 , piece_id=B7ZUFMTGVRPCNJSTBCXMQRZW2AGKPXM2LJRDLYZWPXQ4F5JJ75UQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.295563563 (15h5m39.03962043s) [noise=true , address=104.36.82.14:4254 , piece_id=AIB65II65FK6M3EQHX6PVX27N4Q4BQKI5QGVP3PD6D34ZKXVCLZA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.367859627 (15h5m39.11191649s) [noise=true , address=104.36.82.44:4887 , piece_id=FC4PG4NZJHR35DZLGLFZ6EJZYA2MJWLIDPBYHZCNJKJI4OVX5G4Q, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.486413539 (15h5m39.2304704s) [noise=true , address=104.36.82.9:4038 , piece_id=RVXFCLSREWSON62WSTYVISC52HN33VUN2BHJ7UR72MEBMTFKQV6A, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.510192603 (15h5m39.254249451s) [noise=true , address=79.127.201.215:4358 , piece_id=Z3PRSJIAQMKCL2ZU5TA7WZ2IBCVIXB2K5UJBMS3UCI5H6HDNTWBA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.609092139 (15h5m39.353148988s) [noise=true , address=104.36.82.54:4192 , piece_id=FDJHGRVMHAFB5JZM4Z77MF3NHRU3T4FNQPUAOXGP5NRCM5Q7RVRA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.647464269 (15h5m39.391521122s) [noise=true , address=104.36.82.14:4243 , piece_id=HSR3LCZHIDLJSJ7WISDYOWGPLBFZ73MIXTR7MLFQ2TLUBYLRL4DQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.681479215 (15h5m39.425536078s) [noise=true , address=104.36.82.43:4846 , piece_id=73PPCVC3HGFJDIB63URF7XA6PC37VCYLBCOMTSSFNC5UYZR6M5YA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.697970794 (15h5m39.442027648s) [noise=true , address=104.36.82.60:4369 , piece_id=WVPSCPUKKSYS4T3JIEEVDTS7WSFYXNGVQA6QNLGO3BKC3EKEYJGQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.697997236 (15h5m39.442054151s) [seg_pos={0 0}, upload=15533]: Segment reached optimal threshold of 14 pieces. Aug 8 00:45:17.698203533 (15h5m39.442260383s) [noise=true , address=79.127.205.233:4633 , piece_id=F4RDTCFPIWZS4U65QMU5YXWOZSPPDIDMLJ26G3DCN3YV2UMAEFWA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EEYb4mCySvZ54qzmygCQ7hLyTGYpdNJMyrvPvHJjtfGos9usk): protocol: expected piece hash; context canceled; EOF Aug 8 00:45:17.698290533 (15h5m39.442347383s) [noise=true , address=64.49.13.140:4515 , piece_id=AQTWEM35E7E6YUISEOQGTPTPKD6RDOMKXLK267UILGD77273OY4A, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mY7sRFAeDLmKPqq8BgwnGtkdPTsDRR6soTBshsn4qhP7evARb): protocol: expected piece hash; context canceled; EOF Aug 8 00:45:17.698544485 (15h5m39.442601338s) [noise=true , address=104.36.82.23:4464 , piece_id=4HNT67XCPCNCECLCE36RVAHTJ7VBAADPQ7VMC7IASAUPVVEKIVWQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mLbGHntspYmsTVg8433HYSdAEMrXjGnDhmmvX1JmmgcxtdLgk): protocol: write tcp 139.59.144.22:39260->104.36.82.23:4464: use of closed network connection; write tcp 139.59.144.22:39260->104.36.82.23:4464: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39260->104.36.82.23:4464: use of closed network connection Aug 8 00:45:17.698701613 (15h5m39.442758474s) [noise=true , address=104.36.82.23:4452 , piece_id=B4XGMOK4RCAIOV3VSJXFZ73O52E4WIRUGSPVTCCJ4XUGNJHUQWBA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sAh1mLmzbtmc4o2FAZssW8Y7iDDNmNTCXdCS8xrH8PBVQqxG5): protocol: context canceled; context canceled Aug 8 00:45:17.698723379 (15h5m39.44278023s) [seg_pos={0 0}, upload=15533]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 00:45:17.698815435 (15h5m39.442872285s) [upload=15533]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:45:17.698820522 (15h5m39.442877383s) [upload=15533]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:45:17.860271466 (15h5m39.604328326s) [upload=15533]: Done uploading segments... Aug 8 00:45:17.860291088 (15h5m39.604347935s) [upload=15533]: Upload finished. err: <nil> Duration: 3.302861875s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..................................................X........X] duration: 360.625462ms FAIL (2025-08-08 00:51:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 263481 (15h4m20.280320332s) [upload=15511]: Done uploading segments... Aug 8 00:43:58.536281876 (15h4m20.28033873s) [upload=15511]: Upload finished. err: <nil> Aug 8 00:45:01.015818127 (15h5m22.759874977s) [upload=15529]: Starting upload Aug 8 00:45:01.015874109 (15h5m22.759930972s) [upload=15529]: Done starting upload Aug 8 00:45:01.015956182 (15h5m22.76001303s) [upload=15529]: Uploading segments... Aug 8 00:45:01.016043908 (15h5m22.760100776s) [upload=15529]: Got next segment. Inline: true Aug 8 00:45:01.016049919 (15h5m22.760106782s) [upload=15529]: Waiting for error group managing segments... Aug 8 00:45:01.016115031 (15h5m22.760171895s) [upload=15529]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:45:01.016122390 (15h5m22.760179257s) [upload=15529]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:45:01.016125863 (15h5m22.760182727s) [upload=15529]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:45:01.173239782 (15h5m22.91729665s) [upload=15529]: Done uploading segments... Aug 8 00:45:01.173279478 (15h5m22.917336328s) [upload=15529]: Upload finished. err: <nil> Aug 8 00:46:00.429072612 (15h6m22.173129463s) [upload=15546]: Starting upload Aug 8 00:46:00.429156391 (15h6m22.173213278s) [upload=15546]: Done starting upload Aug 8 00:46:00.429267021 (15h6m22.173323882s) [upload=15546]: Uploading segments... Aug 8 00:46:00.429320811 (15h6m22.173377677s) [upload=15546]: Got next segment. Inline: true Aug 8 00:46:00.429326919 (15h6m22.17338377s) [upload=15546]: Waiting for error group managing segments... Aug 8 00:46:00.429377405 (15h6m22.173434256s) [upload=15546]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:46:00.429383461 (15h6m22.173440311s) [upload=15546]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:46:00.429387017 (15h6m22.173443877s) [upload=15546]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:46:00.684470407 (15h6m22.428527258s) [upload=15546]: Done uploading segments... Aug 8 00:46:00.684501074 (15h6m22.42855792s) [upload=15546]: Upload finished. err: <nil> Aug 8 00:47:14.991362846 (15h7m36.735419697s) [upload=15569]: Starting upload Aug 8 00:47:14.991436891 (15h7m36.735493754s) [upload=15569]: Done starting upload Aug 8 00:47:14.991521124 (15h7m36.735577974s) [upload=15569]: Uploading segments... Aug 8 00:47:14.991582184 (15h7m36.735639054s) [upload=15569]: Got next segment. Inline: true Aug 8 00:47:14.991588085 (15h7m36.735644945s) [upload=15569]: Waiting for error group managing segments... Aug 8 00:47:14.991641941 (15h7m36.735698801s) [upload=15569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:47:14.991647402 (15h7m36.735704266s) [upload=15569]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:47:14.991651037 (15h7m36.735707897s) [upload=15569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:47:15.149817056 (15h7m36.893873921s) [upload=15569]: Done uploading segments... Aug 8 00:47:15.149854826 (15h7m36.893911673s) [upload=15569]: Upload finished. err: <nil> Aug 8 00:48:05.184694323 (15h8m26.92875119s) [upload=15581]: Starting upload Aug 8 00:48:05.184775560 (15h8m26.928832424s) [upload=15581]: Done starting upload Aug 8 00:48:05.184860193 (15h8m26.928917046s) [upload=15581]: Uploading segments... Aug 8 00:48:05.184919879 (15h8m26.92897675s) [upload=15581]: Got next segment. Inline: true Aug 8 00:48:05.184927810 (15h8m26.928984686s) [upload=15581]: Waiting for error group managing segments... Aug 8 00:48:05.184989718 (15h8m26.929046568s) [upload=15581]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:48:05.184997233 (15h8m26.929054096s) [upload=15581]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:48:05.185002973 (15h8m26.929059837s) [upload=15581]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:48:05.333512154 (15h8m27.077569026s) [upload=15581]: Done uploading segments... Aug 8 00:48:05.333535225 (15h8m27.077592084s) [upload=15581]: Upload finished. err: <nil> Aug 8 00:48:25.959034104 (15h8m47.703090965s) [upload=15588]: Starting upload Aug 8 00:48:25.959129631 (15h8m47.703186501s) [upload=15588]: Done starting upload Aug 8 00:48:25.959263036 (15h8m47.703319897s) [upload=15588]: Uploading segments... Aug 8 00:48:25.959321571 (15h8m47.703378451s) [upload=15588]: Got next segment. Inline: true Aug 8 00:48:25.959328075 (15h8m47.70338494s) [upload=15588]: Waiting for error group managing segments... Aug 8 00:48:25.959379365 (15h8m47.70343623s) [upload=15588]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:48:25.959385760 (15h8m47.703442625s) [upload=15588]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:48:25.959389437 (15h8m47.703446299s) [upload=15588]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:48:26.117426083 (15h8m47.861482953s) [upload=15588]: Done uploading segments... Aug 8 00:48:26.117449715 (15h8m47.861506585s) [upload=15588]: Upload finished. err: <nil> Aug 8 00:49:18.380712745 (15h9m40.124769598s) [upload=15601]: Starting upload Aug 8 00:49:18.380774191 (15h9m40.124831063s) [upload=15601]: Done starting upload Aug 8 00:49:18.380853700 (15h9m40.124910555s) [upload=15601]: Uploading segments... Aug 8 00:49:18.380935089 (15h9m40.124991961s) [upload=15601]: Got next segment. Inline: true Aug 8 00:49:18.380942228 (15h9m40.124999082s) [upload=15601]: Waiting for error group managing segments... Aug 8 00:49:18.380990837 (15h9m40.125047687s) [upload=15601]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:49:18.380996570 (15h9m40.125053423s) [upload=15601]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:49:18.381000556 (15h9m40.125057404s) [upload=15601]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:49:18.524510041 (15h9m40.26856689s) [upload=15601]: Done uploading segments... Aug 8 00:49:18.524532847 (15h9m40.268589697s) [upload=15601]: Upload finished. err: <nil> Aug 8 00:50:14.219213519 (15h10m35.96327037s) [upload=15618]: Starting upload Aug 8 00:50:14.219268923 (15h10m35.963325784s) [upload=15618]: Done starting upload Aug 8 00:50:14.219367379 (15h10m35.963424233s) [upload=15618]: Uploading segments... Aug 8 00:50:14.219578059 (15h10m35.963634929s) [upload=15618]: Got next segment. Inline: true Aug 8 00:50:14.219614810 (15h10m35.963671673s) [upload=15618]: Waiting for error group managing segments... Aug 8 00:50:14.219687157 (15h10m35.963744017s) [upload=15618]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:50:14.219693611 (15h10m35.963750471s) [upload=15618]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:50:14.219699847 (15h10m35.96375671s) [upload=15618]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:50:14.385631211 (15h10m36.129688064s) [upload=15618]: Done uploading segments... Aug 8 00:50:14.385650912 (15h10m36.129707762s) [upload=15618]: Upload finished. err: <nil> Aug 8 00:50:44.080806747 (15h11m5.824863598s) [upload=15628]: Starting upload Aug 8 00:50:44.080911523 (15h11m5.824968393s) [upload=15628]: Done starting upload Aug 8 00:50:44.081032828 (15h11m5.825089678s) [upload=15628]: Uploading segments... Aug 8 00:50:44.081102196 (15h11m5.825159064s) [upload=15628]: Got next segment. Inline: true Aug 8 00:50:44.081110880 (15h11m5.82516774s) [upload=15628]: Waiting for error group managing segments... Aug 8 00:50:44.081162262 (15h11m5.825219175s) [upload=15628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:50:44.081168840 (15h11m5.825225704s) [upload=15628]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:50:44.081172385 (15h11m5.825229249s) [upload=15628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:50:44.261318102 (15h11m6.005374965s) [upload=15628]: Done uploading segments... Aug 8 00:50:44.261339384 (15h11m6.005396236s) [upload=15628]: Upload finished. err: <nil> Aug 8 00:51:00.984479825 (15h11m22.728536675s) [upload=15634]: Starting upload Aug 8 00:51:00.984537080 (15h11m22.72859393s) [upload=15634]: Done starting upload Aug 8 00:51:00.984618934 (15h11m22.728675798s) [upload=15634]: Uploading segments... Aug 8 00:51:00.984689929 (15h11m22.728746817s) [upload=15634]: Got next segment. Inline: true Aug 8 00:51:00.984696447 (15h11m22.728753308s) [upload=15634]: Waiting for error group managing segments... Aug 8 00:51:00.984795333 (15h11m22.728852187s) [upload=15634]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:51:00.984801510 (15h11m22.72885837s) [upload=15634]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:51:00.984804953 (15h11m22.728861813s) [upload=15634]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:51:01.179830418 (15h11m22.923887268s) [upload=15634]: Done uploading segments... Aug 8 00:51:01.179852239 (15h11m22.923909093s) [upload=15634]: Upload finished. err: <nil> Aug 8 00:51:18.788113821 (15h11m40.532170668s) [upload=15636]: Starting upload Aug 8 00:51:18.788190945 (15h11m40.53224781s) [upload=15636]: Done starting upload Aug 8 00:51:18.788266338 (15h11m40.532323198s) [upload=15636]: Uploading segments... Aug 8 00:51:18.788334958 (15h11m40.532391824s) [upload=15636]: Got next segment. Inline: true Aug 8 00:51:18.788341211 (15h11m40.532398065s) [upload=15636]: Waiting for error group managing segments... Aug 8 00:51:18.788385583 (15h11m40.532442429s) [upload=15636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:51:18.788390526 (15h11m40.532447372s) [upload=15636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:51:18.788393111 (15h11m40.532449957s) [upload=15636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:51:18.876506251 (15h11m40.620563101s) [upload=15636]: Done uploading segments... Aug 8 00:51:18.876525344 (15h11m40.620582195s) [upload=15636]: Upload finished. err: metaclient: Too Many Requests Duration: 360.66808ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 981.697094ms FAIL (2025-08-08 00:51:01 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== stItem_ObjectCommit Aug 8 00:37:35.928930883 (14h57m57.672987755s) [upload=15397]: Done uploading segments... Aug 8 00:37:35.928967703 (14h57m57.673024563s) [upload=15397]: Upload finished. err: <nil> Aug 8 00:37:56.695554589 (14h58m18.439611449s) [upload=15402]: Starting upload Aug 8 00:37:56.695609369 (14h58m18.43966623s) [upload=15402]: Done starting upload Aug 8 00:37:56.695671726 (14h58m18.439728574s) [upload=15402]: Uploading segments... Aug 8 00:37:56.695713147 (14h58m18.439770016s) [upload=15402]: Got next segment. Inline: true Aug 8 00:37:56.695718707 (14h58m18.439775558s) [upload=15402]: Waiting for error group managing segments... Aug 8 00:37:56.695760492 (14h58m18.439817342s) [upload=15402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:37:56.695765786 (14h58m18.439822646s) [upload=15402]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:37:56.695769297 (14h58m18.439826158s) [upload=15402]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:37:56.839484031 (14h58m18.583540885s) [upload=15402]: Done uploading segments... Aug 8 00:37:56.839508115 (14h58m18.583564984s) [upload=15402]: Upload finished. err: <nil> Aug 8 00:39:08.555730292 (14h59m30.299787146s) [upload=15423]: Starting upload Aug 8 00:39:08.555824312 (14h59m30.299881164s) [upload=15423]: Done starting upload Aug 8 00:39:08.555930281 (14h59m30.299987134s) [upload=15423]: Uploading segments... Aug 8 00:39:08.555989277 (14h59m30.300046148s) [upload=15423]: Got next segment. Inline: true Aug 8 00:39:08.555995450 (14h59m30.300052299s) [upload=15423]: Waiting for error group managing segments... Aug 8 00:39:08.556058440 (14h59m30.30011529s) [upload=15423]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:39:08.556064571 (14h59m30.300121422s) [upload=15423]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:39:08.556069271 (14h59m30.300126124s) [upload=15423]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:39:08.697938297 (14h59m30.441995159s) [upload=15423]: Done uploading segments... Aug 8 00:39:08.697959132 (14h59m30.442015982s) [upload=15423]: Upload finished. err: <nil> Aug 8 00:40:19.170886308 (15h0m40.914943159s) [upload=15444]: Starting upload Aug 8 00:40:19.170949280 (15h0m40.91500614s) [upload=15444]: Done starting upload Aug 8 00:40:19.171171058 (15h0m40.915227903s) [upload=15444]: Uploading segments... Aug 8 00:40:19.171300409 (15h0m40.915357281s) [upload=15444]: Got next segment. Inline: true Aug 8 00:40:19.171309541 (15h0m40.915366409s) [upload=15444]: Waiting for error group managing segments... Aug 8 00:40:19.171385631 (15h0m40.915442482s) [upload=15444]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:40:19.171391854 (15h0m40.915448714s) [upload=15444]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:40:19.171395665 (15h0m40.915452525s) [upload=15444]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:40:19.331296850 (15h0m41.075353715s) [upload=15444]: Done uploading segments... Aug 8 00:40:19.331319872 (15h0m41.075376736s) [upload=15444]: Upload finished. err: <nil> Aug 8 00:41:14.342295084 (15h1m36.086351944s) [upload=15459]: Starting upload Aug 8 00:41:14.342370744 (15h1m36.086427596s) [upload=15459]: Done starting upload Aug 8 00:41:14.342453152 (15h1m36.086510013s) [upload=15459]: Uploading segments... Aug 8 00:41:14.342497220 (15h1m36.086554158s) [upload=15459]: Got next segment. Inline: true Aug 8 00:41:14.342504023 (15h1m36.08656089s) [upload=15459]: Waiting for error group managing segments... Aug 8 00:41:14.342588479 (15h1m36.086645343s) [upload=15459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:41:14.342612029 (15h1m36.086668893s) [upload=15459]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:41:14.342616257 (15h1m36.08667312s) [upload=15459]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:41:14.552238378 (15h1m36.296295229s) [upload=15459]: Done uploading segments... Aug 8 00:41:14.552258831 (15h1m36.296315682s) [upload=15459]: Upload finished. err: <nil> Aug 8 00:41:33.520869236 (15h1m55.264926087s) [upload=15470]: Starting upload Aug 8 00:41:33.520992007 (15h1m55.265048867s) [upload=15470]: Done starting upload Aug 8 00:41:33.521087418 (15h1m55.265144282s) [upload=15470]: Uploading segments... Aug 8 00:41:33.521174747 (15h1m55.265231612s) [upload=15470]: Got next segment. Inline: true Aug 8 00:41:33.521182648 (15h1m55.265239509s) [upload=15470]: Waiting for error group managing segments... Aug 8 00:41:33.521241390 (15h1m55.265298239s) [upload=15470]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:41:33.521247030 (15h1m55.26530389s) [upload=15470]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:41:33.521250440 (15h1m55.265307301s) [upload=15470]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:41:33.674374631 (15h1m55.418431478s) [upload=15470]: Done uploading segments... Aug 8 00:41:33.674392958 (15h1m55.418449804s) [upload=15470]: Upload finished. err: <nil> Aug 8 00:42:13.521243964 (15h2m35.26530081s) [upload=15480]: Starting upload Aug 8 00:42:13.521322202 (15h2m35.265379069s) [upload=15480]: Done starting upload Aug 8 00:42:13.521399949 (15h2m35.265456796s) [upload=15480]: Uploading segments... Aug 8 00:42:13.521486038 (15h2m35.265542908s) [upload=15480]: Got next segment. Inline: true Aug 8 00:42:13.521492566 (15h2m35.265549425s) [upload=15480]: Waiting for error group managing segments... Aug 8 00:42:13.521546734 (15h2m35.26560358s) [upload=15480]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:42:13.521552405 (15h2m35.265609264s) [upload=15480]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:42:13.521555275 (15h2m35.265612134s) [upload=15480]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:42:13.690373889 (15h2m35.434430739s) [upload=15480]: Done uploading segments... Aug 8 00:42:13.690405230 (15h2m35.434462081s) [upload=15480]: Upload finished. err: <nil> Aug 8 00:42:43.968444265 (15h3m5.712501112s) [upload=15489]: Starting upload Aug 8 00:42:43.968495052 (15h3m5.712551899s) [upload=15489]: Done starting upload Aug 8 00:42:43.968563760 (15h3m5.712620606s) [upload=15489]: Uploading segments... Aug 8 00:42:43.968604135 (15h3m5.712661001s) [upload=15489]: Got next segment. Inline: true Aug 8 00:42:43.968608788 (15h3m5.712665649s) [upload=15489]: Waiting for error group managing segments... Aug 8 00:42:43.968647123 (15h3m5.712703969s) [upload=15489]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:42:43.968652138 (15h3m5.712708998s) [upload=15489]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:42:43.968654778 (15h3m5.712711638s) [upload=15489]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:42:44.310945173 (15h3m6.055002023s) [upload=15489]: Done uploading segments... Aug 8 00:42:44.311090602 (15h3m6.055147463s) [upload=15489]: Upload finished. err: <nil> Aug 8 00:43:08.872038018 (15h3m30.616094879s) [upload=15496]: Starting upload Aug 8 00:43:08.872131319 (15h3m30.616188187s) [upload=15496]: Done starting upload Aug 8 00:43:08.872241899 (15h3m30.616298752s) [upload=15496]: Uploading segments... Aug 8 00:43:08.872314522 (15h3m30.616371385s) [upload=15496]: Got next segment. Inline: true Aug 8 00:43:08.872320103 (15h3m30.616376964s) [upload=15496]: Waiting for error group managing segments... Aug 8 00:43:08.872387129 (15h3m30.616443979s) [upload=15496]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:43:08.872392943 (15h3m30.616449792s) [upload=15496]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:43:08.872396750 (15h3m30.61645361s) [upload=15496]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:43:09.042725445 (15h3m30.786782296s) [upload=15496]: Done uploading segments... Aug 8 00:43:09.042744523 (15h3m30.786801373s) [upload=15496]: Upload finished. err: <nil> Aug 8 00:43:58.382659382 (15h4m20.126716247s) [upload=15511]: Starting upload Aug 8 00:43:58.382746751 (15h4m20.126803611s) [upload=15511]: Done starting upload Aug 8 00:43:58.382850091 (15h4m20.126906951s) [upload=15511]: Uploading segments... Aug 8 00:43:58.382921355 (15h4m20.126978226s) [upload=15511]: Got next segment. Inline: true Aug 8 00:43:58.382928916 (15h4m20.12698578s) [upload=15511]: Waiting for error group managing segments... Aug 8 00:43:58.383111923 (15h4m20.127168791s) [upload=15511]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:43:58.383121251 (15h4m20.127178114s) [upload=15511]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:43:58.383125081 (15h4m20.127181945s) [upload=15511]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:43:58.536263481 (15h4m20.280320332s) [upload=15511]: Done uploading segments... Aug 8 00:43:58.536281876 (15h4m20.28033873s) [upload=15511]: Upload finished. err: <nil> Aug 8 00:45:01.015818127 (15h5m22.759874977s) [upload=15529]: Starting upload Aug 8 00:45:01.015874109 (15h5m22.759930972s) [upload=15529]: Done starting upload Aug 8 00:45:01.015956182 (15h5m22.76001303s) [upload=15529]: Uploading segments... Aug 8 00:45:01.016043908 (15h5m22.760100776s) [upload=15529]: Got next segment. Inline: true Aug 8 00:45:01.016049919 (15h5m22.760106782s) [upload=15529]: Waiting for error group managing segments... Aug 8 00:45:01.016115031 (15h5m22.760171895s) [upload=15529]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:45:01.016122390 (15h5m22.760179257s) [upload=15529]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:45:01.016125863 (15h5m22.760182727s) [upload=15529]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:45:01.173239782 (15h5m22.91729665s) [upload=15529]: Done uploading segments... Aug 8 00:45:01.173279478 (15h5m22.917336328s) [upload=15529]: Upload finished. err: <nil> Duration: 981.735676ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 944.420771ms FAIL (2025-08-08 00:50:44 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== stItem_ObjectCommit Aug 8 00:37:35.928930883 (14h57m57.672987755s) [upload=15397]: Done uploading segments... Aug 8 00:37:35.928967703 (14h57m57.673024563s) [upload=15397]: Upload finished. err: <nil> Aug 8 00:37:56.695554589 (14h58m18.439611449s) [upload=15402]: Starting upload Aug 8 00:37:56.695609369 (14h58m18.43966623s) [upload=15402]: Done starting upload Aug 8 00:37:56.695671726 (14h58m18.439728574s) [upload=15402]: Uploading segments... Aug 8 00:37:56.695713147 (14h58m18.439770016s) [upload=15402]: Got next segment. Inline: true Aug 8 00:37:56.695718707 (14h58m18.439775558s) [upload=15402]: Waiting for error group managing segments... Aug 8 00:37:56.695760492 (14h58m18.439817342s) [upload=15402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:37:56.695765786 (14h58m18.439822646s) [upload=15402]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:37:56.695769297 (14h58m18.439826158s) [upload=15402]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:37:56.839484031 (14h58m18.583540885s) [upload=15402]: Done uploading segments... Aug 8 00:37:56.839508115 (14h58m18.583564984s) [upload=15402]: Upload finished. err: <nil> Aug 8 00:39:08.555730292 (14h59m30.299787146s) [upload=15423]: Starting upload Aug 8 00:39:08.555824312 (14h59m30.299881164s) [upload=15423]: Done starting upload Aug 8 00:39:08.555930281 (14h59m30.299987134s) [upload=15423]: Uploading segments... Aug 8 00:39:08.555989277 (14h59m30.300046148s) [upload=15423]: Got next segment. Inline: true Aug 8 00:39:08.555995450 (14h59m30.300052299s) [upload=15423]: Waiting for error group managing segments... Aug 8 00:39:08.556058440 (14h59m30.30011529s) [upload=15423]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:39:08.556064571 (14h59m30.300121422s) [upload=15423]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:39:08.556069271 (14h59m30.300126124s) [upload=15423]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:39:08.697938297 (14h59m30.441995159s) [upload=15423]: Done uploading segments... Aug 8 00:39:08.697959132 (14h59m30.442015982s) [upload=15423]: Upload finished. err: <nil> Aug 8 00:40:19.170886308 (15h0m40.914943159s) [upload=15444]: Starting upload Aug 8 00:40:19.170949280 (15h0m40.91500614s) [upload=15444]: Done starting upload Aug 8 00:40:19.171171058 (15h0m40.915227903s) [upload=15444]: Uploading segments... Aug 8 00:40:19.171300409 (15h0m40.915357281s) [upload=15444]: Got next segment. Inline: true Aug 8 00:40:19.171309541 (15h0m40.915366409s) [upload=15444]: Waiting for error group managing segments... Aug 8 00:40:19.171385631 (15h0m40.915442482s) [upload=15444]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:40:19.171391854 (15h0m40.915448714s) [upload=15444]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:40:19.171395665 (15h0m40.915452525s) [upload=15444]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:40:19.331296850 (15h0m41.075353715s) [upload=15444]: Done uploading segments... Aug 8 00:40:19.331319872 (15h0m41.075376736s) [upload=15444]: Upload finished. err: <nil> Aug 8 00:41:14.342295084 (15h1m36.086351944s) [upload=15459]: Starting upload Aug 8 00:41:14.342370744 (15h1m36.086427596s) [upload=15459]: Done starting upload Aug 8 00:41:14.342453152 (15h1m36.086510013s) [upload=15459]: Uploading segments... Aug 8 00:41:14.342497220 (15h1m36.086554158s) [upload=15459]: Got next segment. Inline: true Aug 8 00:41:14.342504023 (15h1m36.08656089s) [upload=15459]: Waiting for error group managing segments... Aug 8 00:41:14.342588479 (15h1m36.086645343s) [upload=15459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:41:14.342612029 (15h1m36.086668893s) [upload=15459]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:41:14.342616257 (15h1m36.08667312s) [upload=15459]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:41:14.552238378 (15h1m36.296295229s) [upload=15459]: Done uploading segments... Aug 8 00:41:14.552258831 (15h1m36.296315682s) [upload=15459]: Upload finished. err: <nil> Aug 8 00:41:33.520869236 (15h1m55.264926087s) [upload=15470]: Starting upload Aug 8 00:41:33.520992007 (15h1m55.265048867s) [upload=15470]: Done starting upload Aug 8 00:41:33.521087418 (15h1m55.265144282s) [upload=15470]: Uploading segments... Aug 8 00:41:33.521174747 (15h1m55.265231612s) [upload=15470]: Got next segment. Inline: true Aug 8 00:41:33.521182648 (15h1m55.265239509s) [upload=15470]: Waiting for error group managing segments... Aug 8 00:41:33.521241390 (15h1m55.265298239s) [upload=15470]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:41:33.521247030 (15h1m55.26530389s) [upload=15470]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:41:33.521250440 (15h1m55.265307301s) [upload=15470]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:41:33.674374631 (15h1m55.418431478s) [upload=15470]: Done uploading segments... Aug 8 00:41:33.674392958 (15h1m55.418449804s) [upload=15470]: Upload finished. err: <nil> Aug 8 00:42:13.521243964 (15h2m35.26530081s) [upload=15480]: Starting upload Aug 8 00:42:13.521322202 (15h2m35.265379069s) [upload=15480]: Done starting upload Aug 8 00:42:13.521399949 (15h2m35.265456796s) [upload=15480]: Uploading segments... Aug 8 00:42:13.521486038 (15h2m35.265542908s) [upload=15480]: Got next segment. Inline: true Aug 8 00:42:13.521492566 (15h2m35.265549425s) [upload=15480]: Waiting for error group managing segments... Aug 8 00:42:13.521546734 (15h2m35.26560358s) [upload=15480]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:42:13.521552405 (15h2m35.265609264s) [upload=15480]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:42:13.521555275 (15h2m35.265612134s) [upload=15480]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:42:13.690373889 (15h2m35.434430739s) [upload=15480]: Done uploading segments... Aug 8 00:42:13.690405230 (15h2m35.434462081s) [upload=15480]: Upload finished. err: <nil> Aug 8 00:42:43.968444265 (15h3m5.712501112s) [upload=15489]: Starting upload Aug 8 00:42:43.968495052 (15h3m5.712551899s) [upload=15489]: Done starting upload Aug 8 00:42:43.968563760 (15h3m5.712620606s) [upload=15489]: Uploading segments... Aug 8 00:42:43.968604135 (15h3m5.712661001s) [upload=15489]: Got next segment. Inline: true Aug 8 00:42:43.968608788 (15h3m5.712665649s) [upload=15489]: Waiting for error group managing segments... Aug 8 00:42:43.968647123 (15h3m5.712703969s) [upload=15489]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:42:43.968652138 (15h3m5.712708998s) [upload=15489]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:42:43.968654778 (15h3m5.712711638s) [upload=15489]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:42:44.310945173 (15h3m6.055002023s) [upload=15489]: Done uploading segments... Aug 8 00:42:44.311090602 (15h3m6.055147463s) [upload=15489]: Upload finished. err: <nil> Aug 8 00:43:08.872038018 (15h3m30.616094879s) [upload=15496]: Starting upload Aug 8 00:43:08.872131319 (15h3m30.616188187s) [upload=15496]: Done starting upload Aug 8 00:43:08.872241899 (15h3m30.616298752s) [upload=15496]: Uploading segments... Aug 8 00:43:08.872314522 (15h3m30.616371385s) [upload=15496]: Got next segment. Inline: true Aug 8 00:43:08.872320103 (15h3m30.616376964s) [upload=15496]: Waiting for error group managing segments... Aug 8 00:43:08.872387129 (15h3m30.616443979s) [upload=15496]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:43:08.872392943 (15h3m30.616449792s) [upload=15496]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:43:08.872396750 (15h3m30.61645361s) [upload=15496]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:43:09.042725445 (15h3m30.786782296s) [upload=15496]: Done uploading segments... Aug 8 00:43:09.042744523 (15h3m30.786801373s) [upload=15496]: Upload finished. err: <nil> Aug 8 00:43:58.382659382 (15h4m20.126716247s) [upload=15511]: Starting upload Aug 8 00:43:58.382746751 (15h4m20.126803611s) [upload=15511]: Done starting upload Aug 8 00:43:58.382850091 (15h4m20.126906951s) [upload=15511]: Uploading segments... Aug 8 00:43:58.382921355 (15h4m20.126978226s) [upload=15511]: Got next segment. Inline: true Aug 8 00:43:58.382928916 (15h4m20.12698578s) [upload=15511]: Waiting for error group managing segments... Aug 8 00:43:58.383111923 (15h4m20.127168791s) [upload=15511]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:43:58.383121251 (15h4m20.127178114s) [upload=15511]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:43:58.383125081 (15h4m20.127181945s) [upload=15511]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:43:58.536263481 (15h4m20.280320332s) [upload=15511]: Done uploading segments... Aug 8 00:43:58.536281876 (15h4m20.28033873s) [upload=15511]: Upload finished. err: <nil> Aug 8 00:45:01.015818127 (15h5m22.759874977s) [upload=15529]: Starting upload Aug 8 00:45:01.015874109 (15h5m22.759930972s) [upload=15529]: Done starting upload Aug 8 00:45:01.015956182 (15h5m22.76001303s) [upload=15529]: Uploading segments... Aug 8 00:45:01.016043908 (15h5m22.760100776s) [upload=15529]: Got next segment. Inline: true Aug 8 00:45:01.016049919 (15h5m22.760106782s) [upload=15529]: Waiting for error group managing segments... Aug 8 00:45:01.016115031 (15h5m22.760171895s) [upload=15529]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:45:01.016122390 (15h5m22.760179257s) [upload=15529]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:45:01.016125863 (15h5m22.760182727s) [upload=15529]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:45:01.173239782 (15h5m22.91729665s) [upload=15529]: Done uploading segments... Aug 8 00:45:01.173279478 (15h5m22.917336328s) [upload=15529]: Upload finished. err: <nil> Duration: 944.47985ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........X...........................................X......] duration: 2.439898742s FAIL (2025-08-08 00:50:33 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Done starting upload Aug 8 00:45:16.266121630 (15h5m38.01017848s) [upload=15533]: Uploading segments... Aug 8 00:45:16.266428624 (15h5m38.010485484s) [upload=15533]: Got next segment. Inline: false Aug 8 00:45:16.266497281 (15h5m38.010554191s) [upload=15533]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:45:16.266512043 (15h5m38.010568914s) [upload=15533]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:45:16.467813786 (15h5m38.211870647s) [seg_pos={0 0}, upload=15533]: Begin upload segment... Aug 8 00:45:16.467883230 (15h5m38.211940081s) [seg_pos={0 0}, upload=15533]: Done begin upload segment. Aug 8 00:45:16.467935054 (15h5m38.211991969s) [noise=true , address=104.36.82.60:4369 , piece_id=WVPSCPUKKSYS4T3JIEEVDTS7WSFYXNGVQA6QNLGO3BKC3EKEYJGQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.467945924 (15h5m38.212002789s) [noise=true , address=104.36.82.54:4192 , piece_id=FDJHGRVMHAFB5JZM4Z77MF3NHRU3T4FNQPUAOXGP5NRCM5Q7RVRA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468440400 (15h5m38.212497272s) [noise=true , address=104.36.82.34:4577 , piece_id=4PTCNUDROG6OXW5DBUEMBAGPEMEMVLIW5OXRP2LPC55J2PTK3PAA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468457544 (15h5m38.21251441s) [noise=true , address=104.36.82.44:4887 , piece_id=FC4PG4NZJHR35DZLGLFZ6EJZYA2MJWLIDPBYHZCNJKJI4OVX5G4Q, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468847577 (15h5m38.212904441s) [noise=true , address=104.36.82.62:4430 , piece_id=JGWQSS5KYYCBOITFMENDLSGK6XSDVOT7TMRGJBZ55KBXX54ENY6A, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468887696 (15h5m38.21294456s) [noise=true , address=79.127.205.233:4633 , piece_id=F4RDTCFPIWZS4U65QMU5YXWOZSPPDIDMLJ26G3DCN3YV2UMAEFWA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469228336 (15h5m38.2132852s) [noise=true , address=104.36.82.43:4846 , piece_id=73PPCVC3HGFJDIB63URF7XA6PC37VCYLBCOMTSSFNC5UYZR6M5YA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469268209 (15h5m38.213325082s) [noise=true , address=64.49.13.140:4515 , piece_id=AQTWEM35E7E6YUISEOQGTPTPKD6RDOMKXLK267UILGD77273OY4A, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469613163 (15h5m38.21367003s) [noise=true , address=104.36.82.16:4307 , piece_id=B7ZUFMTGVRPCNJSTBCXMQRZW2AGKPXM2LJRDLYZWPXQ4F5JJ75UQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469702757 (15h5m38.213759621s) [noise=true , address=104.36.82.9:4038 , piece_id=RVXFCLSREWSON62WSTYVISC52HN33VUN2BHJ7UR72MEBMTFKQV6A, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470036233 (15h5m38.214093098s) [noise=true , address=104.36.82.23:4464 , piece_id=4HNT67XCPCNCECLCE36RVAHTJ7VBAADPQ7VMC7IASAUPVVEKIVWQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470129321 (15h5m38.214186184s) [noise=true , address=45.91.116.6:4982 , piece_id=NV3WES7XAJBCNO5EQMWFK6SNUEYE63YKTQC5ZMRF7URP6SX3FXGA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470411978 (15h5m38.214468842s) [noise=true , address=104.36.82.23:4452 , piece_id=B4XGMOK4RCAIOV3VSJXFZ73O52E4WIRUGSPVTCCJ4XUGNJHUQWBA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470482224 (15h5m38.21453909s) [noise=true , address=104.36.82.14:4254 , piece_id=AIB65II65FK6M3EQHX6PVX27N4Q4BQKI5QGVP3PD6D34ZKXVCLZA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470792753 (15h5m38.214849603s) [noise=true , address=104.36.82.19:4405 , piece_id=SA2CPGDA75CYSUDYYZ6V4NKTG5AEIZC7SVYNL2OLBK3E5MSK5FQQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470877563 (15h5m38.214934486s) [noise=true , address=104.36.82.14:4243 , piece_id=HSR3LCZHIDLJSJ7WISDYOWGPLBFZ73MIXTR7MLFQ2TLUBYLRL4DQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.471502176 (15h5m38.215559147s) [noise=true , address=104.36.82.45:4910 , piece_id=OGS6W5AGDOACXIMZ4OH3GNQ43YUU6WOYJHKAHNMHJZBQPJUVVCRA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.471850592 (15h5m38.215907452s) [noise=true , address=79.127.201.215:4358 , piece_id=Z3PRSJIAQMKCL2ZU5TA7WZ2IBCVIXB2K5UJBMS3UCI5H6HDNTWBA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:17.050924903 (15h5m38.794981774s) [upload=15533]: Next returned nil segment Aug 8 00:45:17.051086092 (15h5m38.795142971s) [upload=15533]: Waiting for error group managing segments... Aug 8 00:45:17.225555422 (15h5m38.969612281s) [noise=true , address=104.36.82.62:4430 , piece_id=JGWQSS5KYYCBOITFMENDLSGK6XSDVOT7TMRGJBZ55KBXX54ENY6A, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.232285957 (15h5m38.97634281s) [noise=true , address=45.91.116.6:4982 , piece_id=NV3WES7XAJBCNO5EQMWFK6SNUEYE63YKTQC5ZMRF7URP6SX3FXGA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.239931943 (15h5m38.983988797s) [noise=true , address=104.36.82.45:4910 , piece_id=OGS6W5AGDOACXIMZ4OH3GNQ43YUU6WOYJHKAHNMHJZBQPJUVVCRA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.240442032 (15h5m38.984498886s) [noise=true , address=104.36.82.34:4577 , piece_id=4PTCNUDROG6OXW5DBUEMBAGPEMEMVLIW5OXRP2LPC55J2PTK3PAA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.280384119 (15h5m39.024440979s) [noise=true , address=104.36.82.19:4405 , piece_id=SA2CPGDA75CYSUDYYZ6V4NKTG5AEIZC7SVYNL2OLBK3E5MSK5FQQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.284571751 (15h5m39.028628621s) [noise=true , address=104.36.82.16:4307 , piece_id=B7ZUFMTGVRPCNJSTBCXMQRZW2AGKPXM2LJRDLYZWPXQ4F5JJ75UQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.295563563 (15h5m39.03962043s) [noise=true , address=104.36.82.14:4254 , piece_id=AIB65II65FK6M3EQHX6PVX27N4Q4BQKI5QGVP3PD6D34ZKXVCLZA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.367859627 (15h5m39.11191649s) [noise=true , address=104.36.82.44:4887 , piece_id=FC4PG4NZJHR35DZLGLFZ6EJZYA2MJWLIDPBYHZCNJKJI4OVX5G4Q, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.486413539 (15h5m39.2304704s) [noise=true , address=104.36.82.9:4038 , piece_id=RVXFCLSREWSON62WSTYVISC52HN33VUN2BHJ7UR72MEBMTFKQV6A, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.510192603 (15h5m39.254249451s) [noise=true , address=79.127.201.215:4358 , piece_id=Z3PRSJIAQMKCL2ZU5TA7WZ2IBCVIXB2K5UJBMS3UCI5H6HDNTWBA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.609092139 (15h5m39.353148988s) [noise=true , address=104.36.82.54:4192 , piece_id=FDJHGRVMHAFB5JZM4Z77MF3NHRU3T4FNQPUAOXGP5NRCM5Q7RVRA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.647464269 (15h5m39.391521122s) [noise=true , address=104.36.82.14:4243 , piece_id=HSR3LCZHIDLJSJ7WISDYOWGPLBFZ73MIXTR7MLFQ2TLUBYLRL4DQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.681479215 (15h5m39.425536078s) [noise=true , address=104.36.82.43:4846 , piece_id=73PPCVC3HGFJDIB63URF7XA6PC37VCYLBCOMTSSFNC5UYZR6M5YA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.697970794 (15h5m39.442027648s) [noise=true , address=104.36.82.60:4369 , piece_id=WVPSCPUKKSYS4T3JIEEVDTS7WSFYXNGVQA6QNLGO3BKC3EKEYJGQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.697997236 (15h5m39.442054151s) [seg_pos={0 0}, upload=15533]: Segment reached optimal threshold of 14 pieces. Aug 8 00:45:17.698203533 (15h5m39.442260383s) [noise=true , address=79.127.205.233:4633 , piece_id=F4RDTCFPIWZS4U65QMU5YXWOZSPPDIDMLJ26G3DCN3YV2UMAEFWA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EEYb4mCySvZ54qzmygCQ7hLyTGYpdNJMyrvPvHJjtfGos9usk): protocol: expected piece hash; context canceled; EOF Aug 8 00:45:17.698290533 (15h5m39.442347383s) [noise=true , address=64.49.13.140:4515 , piece_id=AQTWEM35E7E6YUISEOQGTPTPKD6RDOMKXLK267UILGD77273OY4A, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mY7sRFAeDLmKPqq8BgwnGtkdPTsDRR6soTBshsn4qhP7evARb): protocol: expected piece hash; context canceled; EOF Aug 8 00:45:17.698544485 (15h5m39.442601338s) [noise=true , address=104.36.82.23:4464 , piece_id=4HNT67XCPCNCECLCE36RVAHTJ7VBAADPQ7VMC7IASAUPVVEKIVWQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mLbGHntspYmsTVg8433HYSdAEMrXjGnDhmmvX1JmmgcxtdLgk): protocol: write tcp 139.59.144.22:39260->104.36.82.23:4464: use of closed network connection; write tcp 139.59.144.22:39260->104.36.82.23:4464: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39260->104.36.82.23:4464: use of closed network connection Aug 8 00:45:17.698701613 (15h5m39.442758474s) [noise=true , address=104.36.82.23:4452 , piece_id=B4XGMOK4RCAIOV3VSJXFZ73O52E4WIRUGSPVTCCJ4XUGNJHUQWBA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sAh1mLmzbtmc4o2FAZssW8Y7iDDNmNTCXdCS8xrH8PBVQqxG5): protocol: context canceled; context canceled Aug 8 00:45:17.698723379 (15h5m39.44278023s) [seg_pos={0 0}, upload=15533]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 00:45:17.698815435 (15h5m39.442872285s) [upload=15533]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:45:17.698820522 (15h5m39.442877383s) [upload=15533]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:45:17.860271466 (15h5m39.604328326s) [upload=15533]: Done uploading segments... Aug 8 00:45:17.860291088 (15h5m39.604347935s) [upload=15533]: Upload finished. err: <nil> Duration: 2.439949536s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 885.110592ms FAIL (2025-08-08 00:50:14 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== stItem_ObjectCommit Aug 8 00:37:35.928930883 (14h57m57.672987755s) [upload=15397]: Done uploading segments... Aug 8 00:37:35.928967703 (14h57m57.673024563s) [upload=15397]: Upload finished. err: <nil> Aug 8 00:37:56.695554589 (14h58m18.439611449s) [upload=15402]: Starting upload Aug 8 00:37:56.695609369 (14h58m18.43966623s) [upload=15402]: Done starting upload Aug 8 00:37:56.695671726 (14h58m18.439728574s) [upload=15402]: Uploading segments... Aug 8 00:37:56.695713147 (14h58m18.439770016s) [upload=15402]: Got next segment. Inline: true Aug 8 00:37:56.695718707 (14h58m18.439775558s) [upload=15402]: Waiting for error group managing segments... Aug 8 00:37:56.695760492 (14h58m18.439817342s) [upload=15402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:37:56.695765786 (14h58m18.439822646s) [upload=15402]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:37:56.695769297 (14h58m18.439826158s) [upload=15402]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:37:56.839484031 (14h58m18.583540885s) [upload=15402]: Done uploading segments... Aug 8 00:37:56.839508115 (14h58m18.583564984s) [upload=15402]: Upload finished. err: <nil> Aug 8 00:39:08.555730292 (14h59m30.299787146s) [upload=15423]: Starting upload Aug 8 00:39:08.555824312 (14h59m30.299881164s) [upload=15423]: Done starting upload Aug 8 00:39:08.555930281 (14h59m30.299987134s) [upload=15423]: Uploading segments... Aug 8 00:39:08.555989277 (14h59m30.300046148s) [upload=15423]: Got next segment. Inline: true Aug 8 00:39:08.555995450 (14h59m30.300052299s) [upload=15423]: Waiting for error group managing segments... Aug 8 00:39:08.556058440 (14h59m30.30011529s) [upload=15423]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:39:08.556064571 (14h59m30.300121422s) [upload=15423]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:39:08.556069271 (14h59m30.300126124s) [upload=15423]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:39:08.697938297 (14h59m30.441995159s) [upload=15423]: Done uploading segments... Aug 8 00:39:08.697959132 (14h59m30.442015982s) [upload=15423]: Upload finished. err: <nil> Aug 8 00:40:19.170886308 (15h0m40.914943159s) [upload=15444]: Starting upload Aug 8 00:40:19.170949280 (15h0m40.91500614s) [upload=15444]: Done starting upload Aug 8 00:40:19.171171058 (15h0m40.915227903s) [upload=15444]: Uploading segments... Aug 8 00:40:19.171300409 (15h0m40.915357281s) [upload=15444]: Got next segment. Inline: true Aug 8 00:40:19.171309541 (15h0m40.915366409s) [upload=15444]: Waiting for error group managing segments... Aug 8 00:40:19.171385631 (15h0m40.915442482s) [upload=15444]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:40:19.171391854 (15h0m40.915448714s) [upload=15444]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:40:19.171395665 (15h0m40.915452525s) [upload=15444]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:40:19.331296850 (15h0m41.075353715s) [upload=15444]: Done uploading segments... Aug 8 00:40:19.331319872 (15h0m41.075376736s) [upload=15444]: Upload finished. err: <nil> Aug 8 00:41:14.342295084 (15h1m36.086351944s) [upload=15459]: Starting upload Aug 8 00:41:14.342370744 (15h1m36.086427596s) [upload=15459]: Done starting upload Aug 8 00:41:14.342453152 (15h1m36.086510013s) [upload=15459]: Uploading segments... Aug 8 00:41:14.342497220 (15h1m36.086554158s) [upload=15459]: Got next segment. Inline: true Aug 8 00:41:14.342504023 (15h1m36.08656089s) [upload=15459]: Waiting for error group managing segments... Aug 8 00:41:14.342588479 (15h1m36.086645343s) [upload=15459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:41:14.342612029 (15h1m36.086668893s) [upload=15459]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:41:14.342616257 (15h1m36.08667312s) [upload=15459]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:41:14.552238378 (15h1m36.296295229s) [upload=15459]: Done uploading segments... Aug 8 00:41:14.552258831 (15h1m36.296315682s) [upload=15459]: Upload finished. err: <nil> Aug 8 00:41:33.520869236 (15h1m55.264926087s) [upload=15470]: Starting upload Aug 8 00:41:33.520992007 (15h1m55.265048867s) [upload=15470]: Done starting upload Aug 8 00:41:33.521087418 (15h1m55.265144282s) [upload=15470]: Uploading segments... Aug 8 00:41:33.521174747 (15h1m55.265231612s) [upload=15470]: Got next segment. Inline: true Aug 8 00:41:33.521182648 (15h1m55.265239509s) [upload=15470]: Waiting for error group managing segments... Aug 8 00:41:33.521241390 (15h1m55.265298239s) [upload=15470]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:41:33.521247030 (15h1m55.26530389s) [upload=15470]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:41:33.521250440 (15h1m55.265307301s) [upload=15470]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:41:33.674374631 (15h1m55.418431478s) [upload=15470]: Done uploading segments... Aug 8 00:41:33.674392958 (15h1m55.418449804s) [upload=15470]: Upload finished. err: <nil> Aug 8 00:42:13.521243964 (15h2m35.26530081s) [upload=15480]: Starting upload Aug 8 00:42:13.521322202 (15h2m35.265379069s) [upload=15480]: Done starting upload Aug 8 00:42:13.521399949 (15h2m35.265456796s) [upload=15480]: Uploading segments... Aug 8 00:42:13.521486038 (15h2m35.265542908s) [upload=15480]: Got next segment. Inline: true Aug 8 00:42:13.521492566 (15h2m35.265549425s) [upload=15480]: Waiting for error group managing segments... Aug 8 00:42:13.521546734 (15h2m35.26560358s) [upload=15480]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:42:13.521552405 (15h2m35.265609264s) [upload=15480]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:42:13.521555275 (15h2m35.265612134s) [upload=15480]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:42:13.690373889 (15h2m35.434430739s) [upload=15480]: Done uploading segments... Aug 8 00:42:13.690405230 (15h2m35.434462081s) [upload=15480]: Upload finished. err: <nil> Aug 8 00:42:43.968444265 (15h3m5.712501112s) [upload=15489]: Starting upload Aug 8 00:42:43.968495052 (15h3m5.712551899s) [upload=15489]: Done starting upload Aug 8 00:42:43.968563760 (15h3m5.712620606s) [upload=15489]: Uploading segments... Aug 8 00:42:43.968604135 (15h3m5.712661001s) [upload=15489]: Got next segment. Inline: true Aug 8 00:42:43.968608788 (15h3m5.712665649s) [upload=15489]: Waiting for error group managing segments... Aug 8 00:42:43.968647123 (15h3m5.712703969s) [upload=15489]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:42:43.968652138 (15h3m5.712708998s) [upload=15489]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:42:43.968654778 (15h3m5.712711638s) [upload=15489]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:42:44.310945173 (15h3m6.055002023s) [upload=15489]: Done uploading segments... Aug 8 00:42:44.311090602 (15h3m6.055147463s) [upload=15489]: Upload finished. err: <nil> Aug 8 00:43:08.872038018 (15h3m30.616094879s) [upload=15496]: Starting upload Aug 8 00:43:08.872131319 (15h3m30.616188187s) [upload=15496]: Done starting upload Aug 8 00:43:08.872241899 (15h3m30.616298752s) [upload=15496]: Uploading segments... Aug 8 00:43:08.872314522 (15h3m30.616371385s) [upload=15496]: Got next segment. Inline: true Aug 8 00:43:08.872320103 (15h3m30.616376964s) [upload=15496]: Waiting for error group managing segments... Aug 8 00:43:08.872387129 (15h3m30.616443979s) [upload=15496]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:43:08.872392943 (15h3m30.616449792s) [upload=15496]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:43:08.872396750 (15h3m30.61645361s) [upload=15496]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:43:09.042725445 (15h3m30.786782296s) [upload=15496]: Done uploading segments... Aug 8 00:43:09.042744523 (15h3m30.786801373s) [upload=15496]: Upload finished. err: <nil> Aug 8 00:43:58.382659382 (15h4m20.126716247s) [upload=15511]: Starting upload Aug 8 00:43:58.382746751 (15h4m20.126803611s) [upload=15511]: Done starting upload Aug 8 00:43:58.382850091 (15h4m20.126906951s) [upload=15511]: Uploading segments... Aug 8 00:43:58.382921355 (15h4m20.126978226s) [upload=15511]: Got next segment. Inline: true Aug 8 00:43:58.382928916 (15h4m20.12698578s) [upload=15511]: Waiting for error group managing segments... Aug 8 00:43:58.383111923 (15h4m20.127168791s) [upload=15511]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:43:58.383121251 (15h4m20.127178114s) [upload=15511]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:43:58.383125081 (15h4m20.127181945s) [upload=15511]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:43:58.536263481 (15h4m20.280320332s) [upload=15511]: Done uploading segments... Aug 8 00:43:58.536281876 (15h4m20.28033873s) [upload=15511]: Upload finished. err: <nil> Aug 8 00:45:01.015818127 (15h5m22.759874977s) [upload=15529]: Starting upload Aug 8 00:45:01.015874109 (15h5m22.759930972s) [upload=15529]: Done starting upload Aug 8 00:45:01.015956182 (15h5m22.76001303s) [upload=15529]: Uploading segments... Aug 8 00:45:01.016043908 (15h5m22.760100776s) [upload=15529]: Got next segment. Inline: true Aug 8 00:45:01.016049919 (15h5m22.760106782s) [upload=15529]: Waiting for error group managing segments... Aug 8 00:45:01.016115031 (15h5m22.760171895s) [upload=15529]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:45:01.016122390 (15h5m22.760179257s) [upload=15529]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:45:01.016125863 (15h5m22.760182727s) [upload=15529]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:45:01.173239782 (15h5m22.91729665s) [upload=15529]: Done uploading segments... Aug 8 00:45:01.173279478 (15h5m22.917336328s) [upload=15529]: Upload finished. err: <nil> Duration: 885.176061ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........X...........................................X.....] duration: 2.70976716s FAIL (2025-08-08 00:49:55 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Done starting upload Aug 8 00:45:16.266121630 (15h5m38.01017848s) [upload=15533]: Uploading segments... Aug 8 00:45:16.266428624 (15h5m38.010485484s) [upload=15533]: Got next segment. Inline: false Aug 8 00:45:16.266497281 (15h5m38.010554191s) [upload=15533]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:45:16.266512043 (15h5m38.010568914s) [upload=15533]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:45:16.467813786 (15h5m38.211870647s) [seg_pos={0 0}, upload=15533]: Begin upload segment... Aug 8 00:45:16.467883230 (15h5m38.211940081s) [seg_pos={0 0}, upload=15533]: Done begin upload segment. Aug 8 00:45:16.467935054 (15h5m38.211991969s) [noise=true , address=104.36.82.60:4369 , piece_id=WVPSCPUKKSYS4T3JIEEVDTS7WSFYXNGVQA6QNLGO3BKC3EKEYJGQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.467945924 (15h5m38.212002789s) [noise=true , address=104.36.82.54:4192 , piece_id=FDJHGRVMHAFB5JZM4Z77MF3NHRU3T4FNQPUAOXGP5NRCM5Q7RVRA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468440400 (15h5m38.212497272s) [noise=true , address=104.36.82.34:4577 , piece_id=4PTCNUDROG6OXW5DBUEMBAGPEMEMVLIW5OXRP2LPC55J2PTK3PAA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468457544 (15h5m38.21251441s) [noise=true , address=104.36.82.44:4887 , piece_id=FC4PG4NZJHR35DZLGLFZ6EJZYA2MJWLIDPBYHZCNJKJI4OVX5G4Q, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468847577 (15h5m38.212904441s) [noise=true , address=104.36.82.62:4430 , piece_id=JGWQSS5KYYCBOITFMENDLSGK6XSDVOT7TMRGJBZ55KBXX54ENY6A, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468887696 (15h5m38.21294456s) [noise=true , address=79.127.205.233:4633 , piece_id=F4RDTCFPIWZS4U65QMU5YXWOZSPPDIDMLJ26G3DCN3YV2UMAEFWA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469228336 (15h5m38.2132852s) [noise=true , address=104.36.82.43:4846 , piece_id=73PPCVC3HGFJDIB63URF7XA6PC37VCYLBCOMTSSFNC5UYZR6M5YA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469268209 (15h5m38.213325082s) [noise=true , address=64.49.13.140:4515 , piece_id=AQTWEM35E7E6YUISEOQGTPTPKD6RDOMKXLK267UILGD77273OY4A, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469613163 (15h5m38.21367003s) [noise=true , address=104.36.82.16:4307 , piece_id=B7ZUFMTGVRPCNJSTBCXMQRZW2AGKPXM2LJRDLYZWPXQ4F5JJ75UQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469702757 (15h5m38.213759621s) [noise=true , address=104.36.82.9:4038 , piece_id=RVXFCLSREWSON62WSTYVISC52HN33VUN2BHJ7UR72MEBMTFKQV6A, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470036233 (15h5m38.214093098s) [noise=true , address=104.36.82.23:4464 , piece_id=4HNT67XCPCNCECLCE36RVAHTJ7VBAADPQ7VMC7IASAUPVVEKIVWQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470129321 (15h5m38.214186184s) [noise=true , address=45.91.116.6:4982 , piece_id=NV3WES7XAJBCNO5EQMWFK6SNUEYE63YKTQC5ZMRF7URP6SX3FXGA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470411978 (15h5m38.214468842s) [noise=true , address=104.36.82.23:4452 , piece_id=B4XGMOK4RCAIOV3VSJXFZ73O52E4WIRUGSPVTCCJ4XUGNJHUQWBA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470482224 (15h5m38.21453909s) [noise=true , address=104.36.82.14:4254 , piece_id=AIB65II65FK6M3EQHX6PVX27N4Q4BQKI5QGVP3PD6D34ZKXVCLZA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470792753 (15h5m38.214849603s) [noise=true , address=104.36.82.19:4405 , piece_id=SA2CPGDA75CYSUDYYZ6V4NKTG5AEIZC7SVYNL2OLBK3E5MSK5FQQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470877563 (15h5m38.214934486s) [noise=true , address=104.36.82.14:4243 , piece_id=HSR3LCZHIDLJSJ7WISDYOWGPLBFZ73MIXTR7MLFQ2TLUBYLRL4DQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.471502176 (15h5m38.215559147s) [noise=true , address=104.36.82.45:4910 , piece_id=OGS6W5AGDOACXIMZ4OH3GNQ43YUU6WOYJHKAHNMHJZBQPJUVVCRA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.471850592 (15h5m38.215907452s) [noise=true , address=79.127.201.215:4358 , piece_id=Z3PRSJIAQMKCL2ZU5TA7WZ2IBCVIXB2K5UJBMS3UCI5H6HDNTWBA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:17.050924903 (15h5m38.794981774s) [upload=15533]: Next returned nil segment Aug 8 00:45:17.051086092 (15h5m38.795142971s) [upload=15533]: Waiting for error group managing segments... Aug 8 00:45:17.225555422 (15h5m38.969612281s) [noise=true , address=104.36.82.62:4430 , piece_id=JGWQSS5KYYCBOITFMENDLSGK6XSDVOT7TMRGJBZ55KBXX54ENY6A, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.232285957 (15h5m38.97634281s) [noise=true , address=45.91.116.6:4982 , piece_id=NV3WES7XAJBCNO5EQMWFK6SNUEYE63YKTQC5ZMRF7URP6SX3FXGA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.239931943 (15h5m38.983988797s) [noise=true , address=104.36.82.45:4910 , piece_id=OGS6W5AGDOACXIMZ4OH3GNQ43YUU6WOYJHKAHNMHJZBQPJUVVCRA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.240442032 (15h5m38.984498886s) [noise=true , address=104.36.82.34:4577 , piece_id=4PTCNUDROG6OXW5DBUEMBAGPEMEMVLIW5OXRP2LPC55J2PTK3PAA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.280384119 (15h5m39.024440979s) [noise=true , address=104.36.82.19:4405 , piece_id=SA2CPGDA75CYSUDYYZ6V4NKTG5AEIZC7SVYNL2OLBK3E5MSK5FQQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.284571751 (15h5m39.028628621s) [noise=true , address=104.36.82.16:4307 , piece_id=B7ZUFMTGVRPCNJSTBCXMQRZW2AGKPXM2LJRDLYZWPXQ4F5JJ75UQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.295563563 (15h5m39.03962043s) [noise=true , address=104.36.82.14:4254 , piece_id=AIB65II65FK6M3EQHX6PVX27N4Q4BQKI5QGVP3PD6D34ZKXVCLZA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.367859627 (15h5m39.11191649s) [noise=true , address=104.36.82.44:4887 , piece_id=FC4PG4NZJHR35DZLGLFZ6EJZYA2MJWLIDPBYHZCNJKJI4OVX5G4Q, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.486413539 (15h5m39.2304704s) [noise=true , address=104.36.82.9:4038 , piece_id=RVXFCLSREWSON62WSTYVISC52HN33VUN2BHJ7UR72MEBMTFKQV6A, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.510192603 (15h5m39.254249451s) [noise=true , address=79.127.201.215:4358 , piece_id=Z3PRSJIAQMKCL2ZU5TA7WZ2IBCVIXB2K5UJBMS3UCI5H6HDNTWBA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.609092139 (15h5m39.353148988s) [noise=true , address=104.36.82.54:4192 , piece_id=FDJHGRVMHAFB5JZM4Z77MF3NHRU3T4FNQPUAOXGP5NRCM5Q7RVRA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.647464269 (15h5m39.391521122s) [noise=true , address=104.36.82.14:4243 , piece_id=HSR3LCZHIDLJSJ7WISDYOWGPLBFZ73MIXTR7MLFQ2TLUBYLRL4DQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.681479215 (15h5m39.425536078s) [noise=true , address=104.36.82.43:4846 , piece_id=73PPCVC3HGFJDIB63URF7XA6PC37VCYLBCOMTSSFNC5UYZR6M5YA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.697970794 (15h5m39.442027648s) [noise=true , address=104.36.82.60:4369 , piece_id=WVPSCPUKKSYS4T3JIEEVDTS7WSFYXNGVQA6QNLGO3BKC3EKEYJGQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.697997236 (15h5m39.442054151s) [seg_pos={0 0}, upload=15533]: Segment reached optimal threshold of 14 pieces. Aug 8 00:45:17.698203533 (15h5m39.442260383s) [noise=true , address=79.127.205.233:4633 , piece_id=F4RDTCFPIWZS4U65QMU5YXWOZSPPDIDMLJ26G3DCN3YV2UMAEFWA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EEYb4mCySvZ54qzmygCQ7hLyTGYpdNJMyrvPvHJjtfGos9usk): protocol: expected piece hash; context canceled; EOF Aug 8 00:45:17.698290533 (15h5m39.442347383s) [noise=true , address=64.49.13.140:4515 , piece_id=AQTWEM35E7E6YUISEOQGTPTPKD6RDOMKXLK267UILGD77273OY4A, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mY7sRFAeDLmKPqq8BgwnGtkdPTsDRR6soTBshsn4qhP7evARb): protocol: expected piece hash; context canceled; EOF Aug 8 00:45:17.698544485 (15h5m39.442601338s) [noise=true , address=104.36.82.23:4464 , piece_id=4HNT67XCPCNCECLCE36RVAHTJ7VBAADPQ7VMC7IASAUPVVEKIVWQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mLbGHntspYmsTVg8433HYSdAEMrXjGnDhmmvX1JmmgcxtdLgk): protocol: write tcp 139.59.144.22:39260->104.36.82.23:4464: use of closed network connection; write tcp 139.59.144.22:39260->104.36.82.23:4464: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39260->104.36.82.23:4464: use of closed network connection Aug 8 00:45:17.698701613 (15h5m39.442758474s) [noise=true , address=104.36.82.23:4452 , piece_id=B4XGMOK4RCAIOV3VSJXFZ73O52E4WIRUGSPVTCCJ4XUGNJHUQWBA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sAh1mLmzbtmc4o2FAZssW8Y7iDDNmNTCXdCS8xrH8PBVQqxG5): protocol: context canceled; context canceled Aug 8 00:45:17.698723379 (15h5m39.44278023s) [seg_pos={0 0}, upload=15533]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 00:45:17.698815435 (15h5m39.442872285s) [upload=15533]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:45:17.698820522 (15h5m39.442877383s) [upload=15533]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:45:17.860271466 (15h5m39.604328326s) [upload=15533]: Done uploading segments... Aug 8 00:45:17.860291088 (15h5m39.604347935s) [upload=15533]: Upload finished. err: <nil> Duration: 2.709798249s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........X...........................................X....] duration: 3.702846021s FAIL (2025-08-08 00:49:29 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Done starting upload Aug 8 00:45:16.266121630 (15h5m38.01017848s) [upload=15533]: Uploading segments... Aug 8 00:45:16.266428624 (15h5m38.010485484s) [upload=15533]: Got next segment. Inline: false Aug 8 00:45:16.266497281 (15h5m38.010554191s) [upload=15533]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:45:16.266512043 (15h5m38.010568914s) [upload=15533]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:45:16.467813786 (15h5m38.211870647s) [seg_pos={0 0}, upload=15533]: Begin upload segment... Aug 8 00:45:16.467883230 (15h5m38.211940081s) [seg_pos={0 0}, upload=15533]: Done begin upload segment. Aug 8 00:45:16.467935054 (15h5m38.211991969s) [noise=true , address=104.36.82.60:4369 , piece_id=WVPSCPUKKSYS4T3JIEEVDTS7WSFYXNGVQA6QNLGO3BKC3EKEYJGQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.467945924 (15h5m38.212002789s) [noise=true , address=104.36.82.54:4192 , piece_id=FDJHGRVMHAFB5JZM4Z77MF3NHRU3T4FNQPUAOXGP5NRCM5Q7RVRA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468440400 (15h5m38.212497272s) [noise=true , address=104.36.82.34:4577 , piece_id=4PTCNUDROG6OXW5DBUEMBAGPEMEMVLIW5OXRP2LPC55J2PTK3PAA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468457544 (15h5m38.21251441s) [noise=true , address=104.36.82.44:4887 , piece_id=FC4PG4NZJHR35DZLGLFZ6EJZYA2MJWLIDPBYHZCNJKJI4OVX5G4Q, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468847577 (15h5m38.212904441s) [noise=true , address=104.36.82.62:4430 , piece_id=JGWQSS5KYYCBOITFMENDLSGK6XSDVOT7TMRGJBZ55KBXX54ENY6A, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468887696 (15h5m38.21294456s) [noise=true , address=79.127.205.233:4633 , piece_id=F4RDTCFPIWZS4U65QMU5YXWOZSPPDIDMLJ26G3DCN3YV2UMAEFWA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469228336 (15h5m38.2132852s) [noise=true , address=104.36.82.43:4846 , piece_id=73PPCVC3HGFJDIB63URF7XA6PC37VCYLBCOMTSSFNC5UYZR6M5YA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469268209 (15h5m38.213325082s) [noise=true , address=64.49.13.140:4515 , piece_id=AQTWEM35E7E6YUISEOQGTPTPKD6RDOMKXLK267UILGD77273OY4A, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469613163 (15h5m38.21367003s) [noise=true , address=104.36.82.16:4307 , piece_id=B7ZUFMTGVRPCNJSTBCXMQRZW2AGKPXM2LJRDLYZWPXQ4F5JJ75UQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469702757 (15h5m38.213759621s) [noise=true , address=104.36.82.9:4038 , piece_id=RVXFCLSREWSON62WSTYVISC52HN33VUN2BHJ7UR72MEBMTFKQV6A, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470036233 (15h5m38.214093098s) [noise=true , address=104.36.82.23:4464 , piece_id=4HNT67XCPCNCECLCE36RVAHTJ7VBAADPQ7VMC7IASAUPVVEKIVWQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470129321 (15h5m38.214186184s) [noise=true , address=45.91.116.6:4982 , piece_id=NV3WES7XAJBCNO5EQMWFK6SNUEYE63YKTQC5ZMRF7URP6SX3FXGA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470411978 (15h5m38.214468842s) [noise=true , address=104.36.82.23:4452 , piece_id=B4XGMOK4RCAIOV3VSJXFZ73O52E4WIRUGSPVTCCJ4XUGNJHUQWBA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470482224 (15h5m38.21453909s) [noise=true , address=104.36.82.14:4254 , piece_id=AIB65II65FK6M3EQHX6PVX27N4Q4BQKI5QGVP3PD6D34ZKXVCLZA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470792753 (15h5m38.214849603s) [noise=true , address=104.36.82.19:4405 , piece_id=SA2CPGDA75CYSUDYYZ6V4NKTG5AEIZC7SVYNL2OLBK3E5MSK5FQQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470877563 (15h5m38.214934486s) [noise=true , address=104.36.82.14:4243 , piece_id=HSR3LCZHIDLJSJ7WISDYOWGPLBFZ73MIXTR7MLFQ2TLUBYLRL4DQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.471502176 (15h5m38.215559147s) [noise=true , address=104.36.82.45:4910 , piece_id=OGS6W5AGDOACXIMZ4OH3GNQ43YUU6WOYJHKAHNMHJZBQPJUVVCRA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.471850592 (15h5m38.215907452s) [noise=true , address=79.127.201.215:4358 , piece_id=Z3PRSJIAQMKCL2ZU5TA7WZ2IBCVIXB2K5UJBMS3UCI5H6HDNTWBA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:17.050924903 (15h5m38.794981774s) [upload=15533]: Next returned nil segment Aug 8 00:45:17.051086092 (15h5m38.795142971s) [upload=15533]: Waiting for error group managing segments... Aug 8 00:45:17.225555422 (15h5m38.969612281s) [noise=true , address=104.36.82.62:4430 , piece_id=JGWQSS5KYYCBOITFMENDLSGK6XSDVOT7TMRGJBZ55KBXX54ENY6A, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.232285957 (15h5m38.97634281s) [noise=true , address=45.91.116.6:4982 , piece_id=NV3WES7XAJBCNO5EQMWFK6SNUEYE63YKTQC5ZMRF7URP6SX3FXGA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.239931943 (15h5m38.983988797s) [noise=true , address=104.36.82.45:4910 , piece_id=OGS6W5AGDOACXIMZ4OH3GNQ43YUU6WOYJHKAHNMHJZBQPJUVVCRA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.240442032 (15h5m38.984498886s) [noise=true , address=104.36.82.34:4577 , piece_id=4PTCNUDROG6OXW5DBUEMBAGPEMEMVLIW5OXRP2LPC55J2PTK3PAA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.280384119 (15h5m39.024440979s) [noise=true , address=104.36.82.19:4405 , piece_id=SA2CPGDA75CYSUDYYZ6V4NKTG5AEIZC7SVYNL2OLBK3E5MSK5FQQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.284571751 (15h5m39.028628621s) [noise=true , address=104.36.82.16:4307 , piece_id=B7ZUFMTGVRPCNJSTBCXMQRZW2AGKPXM2LJRDLYZWPXQ4F5JJ75UQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.295563563 (15h5m39.03962043s) [noise=true , address=104.36.82.14:4254 , piece_id=AIB65II65FK6M3EQHX6PVX27N4Q4BQKI5QGVP3PD6D34ZKXVCLZA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.367859627 (15h5m39.11191649s) [noise=true , address=104.36.82.44:4887 , piece_id=FC4PG4NZJHR35DZLGLFZ6EJZYA2MJWLIDPBYHZCNJKJI4OVX5G4Q, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.486413539 (15h5m39.2304704s) [noise=true , address=104.36.82.9:4038 , piece_id=RVXFCLSREWSON62WSTYVISC52HN33VUN2BHJ7UR72MEBMTFKQV6A, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.510192603 (15h5m39.254249451s) [noise=true , address=79.127.201.215:4358 , piece_id=Z3PRSJIAQMKCL2ZU5TA7WZ2IBCVIXB2K5UJBMS3UCI5H6HDNTWBA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.609092139 (15h5m39.353148988s) [noise=true , address=104.36.82.54:4192 , piece_id=FDJHGRVMHAFB5JZM4Z77MF3NHRU3T4FNQPUAOXGP5NRCM5Q7RVRA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.647464269 (15h5m39.391521122s) [noise=true , address=104.36.82.14:4243 , piece_id=HSR3LCZHIDLJSJ7WISDYOWGPLBFZ73MIXTR7MLFQ2TLUBYLRL4DQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.681479215 (15h5m39.425536078s) [noise=true , address=104.36.82.43:4846 , piece_id=73PPCVC3HGFJDIB63URF7XA6PC37VCYLBCOMTSSFNC5UYZR6M5YA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.697970794 (15h5m39.442027648s) [noise=true , address=104.36.82.60:4369 , piece_id=WVPSCPUKKSYS4T3JIEEVDTS7WSFYXNGVQA6QNLGO3BKC3EKEYJGQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.697997236 (15h5m39.442054151s) [seg_pos={0 0}, upload=15533]: Segment reached optimal threshold of 14 pieces. Aug 8 00:45:17.698203533 (15h5m39.442260383s) [noise=true , address=79.127.205.233:4633 , piece_id=F4RDTCFPIWZS4U65QMU5YXWOZSPPDIDMLJ26G3DCN3YV2UMAEFWA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EEYb4mCySvZ54qzmygCQ7hLyTGYpdNJMyrvPvHJjtfGos9usk): protocol: expected piece hash; context canceled; EOF Aug 8 00:45:17.698290533 (15h5m39.442347383s) [noise=true , address=64.49.13.140:4515 , piece_id=AQTWEM35E7E6YUISEOQGTPTPKD6RDOMKXLK267UILGD77273OY4A, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mY7sRFAeDLmKPqq8BgwnGtkdPTsDRR6soTBshsn4qhP7evARb): protocol: expected piece hash; context canceled; EOF Aug 8 00:45:17.698544485 (15h5m39.442601338s) [noise=true , address=104.36.82.23:4464 , piece_id=4HNT67XCPCNCECLCE36RVAHTJ7VBAADPQ7VMC7IASAUPVVEKIVWQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mLbGHntspYmsTVg8433HYSdAEMrXjGnDhmmvX1JmmgcxtdLgk): protocol: write tcp 139.59.144.22:39260->104.36.82.23:4464: use of closed network connection; write tcp 139.59.144.22:39260->104.36.82.23:4464: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39260->104.36.82.23:4464: use of closed network connection Aug 8 00:45:17.698701613 (15h5m39.442758474s) [noise=true , address=104.36.82.23:4452 , piece_id=B4XGMOK4RCAIOV3VSJXFZ73O52E4WIRUGSPVTCCJ4XUGNJHUQWBA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sAh1mLmzbtmc4o2FAZssW8Y7iDDNmNTCXdCS8xrH8PBVQqxG5): protocol: context canceled; context canceled Aug 8 00:45:17.698723379 (15h5m39.44278023s) [seg_pos={0 0}, upload=15533]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 00:45:17.698815435 (15h5m39.442872285s) [upload=15533]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:45:17.698820522 (15h5m39.442877383s) [upload=15533]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:45:17.860271466 (15h5m39.604328326s) [upload=15533]: Done uploading segments... Aug 8 00:45:17.860291088 (15h5m39.604347935s) [upload=15533]: Upload finished. err: <nil> Duration: 3.70287344s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 831.155844ms FAIL (2025-08-08 00:49:18 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== stItem_ObjectCommit Aug 8 00:37:35.928930883 (14h57m57.672987755s) [upload=15397]: Done uploading segments... Aug 8 00:37:35.928967703 (14h57m57.673024563s) [upload=15397]: Upload finished. err: <nil> Aug 8 00:37:56.695554589 (14h58m18.439611449s) [upload=15402]: Starting upload Aug 8 00:37:56.695609369 (14h58m18.43966623s) [upload=15402]: Done starting upload Aug 8 00:37:56.695671726 (14h58m18.439728574s) [upload=15402]: Uploading segments... Aug 8 00:37:56.695713147 (14h58m18.439770016s) [upload=15402]: Got next segment. Inline: true Aug 8 00:37:56.695718707 (14h58m18.439775558s) [upload=15402]: Waiting for error group managing segments... Aug 8 00:37:56.695760492 (14h58m18.439817342s) [upload=15402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:37:56.695765786 (14h58m18.439822646s) [upload=15402]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:37:56.695769297 (14h58m18.439826158s) [upload=15402]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:37:56.839484031 (14h58m18.583540885s) [upload=15402]: Done uploading segments... Aug 8 00:37:56.839508115 (14h58m18.583564984s) [upload=15402]: Upload finished. err: <nil> Aug 8 00:39:08.555730292 (14h59m30.299787146s) [upload=15423]: Starting upload Aug 8 00:39:08.555824312 (14h59m30.299881164s) [upload=15423]: Done starting upload Aug 8 00:39:08.555930281 (14h59m30.299987134s) [upload=15423]: Uploading segments... Aug 8 00:39:08.555989277 (14h59m30.300046148s) [upload=15423]: Got next segment. Inline: true Aug 8 00:39:08.555995450 (14h59m30.300052299s) [upload=15423]: Waiting for error group managing segments... Aug 8 00:39:08.556058440 (14h59m30.30011529s) [upload=15423]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:39:08.556064571 (14h59m30.300121422s) [upload=15423]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:39:08.556069271 (14h59m30.300126124s) [upload=15423]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:39:08.697938297 (14h59m30.441995159s) [upload=15423]: Done uploading segments... Aug 8 00:39:08.697959132 (14h59m30.442015982s) [upload=15423]: Upload finished. err: <nil> Aug 8 00:40:19.170886308 (15h0m40.914943159s) [upload=15444]: Starting upload Aug 8 00:40:19.170949280 (15h0m40.91500614s) [upload=15444]: Done starting upload Aug 8 00:40:19.171171058 (15h0m40.915227903s) [upload=15444]: Uploading segments... Aug 8 00:40:19.171300409 (15h0m40.915357281s) [upload=15444]: Got next segment. Inline: true Aug 8 00:40:19.171309541 (15h0m40.915366409s) [upload=15444]: Waiting for error group managing segments... Aug 8 00:40:19.171385631 (15h0m40.915442482s) [upload=15444]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:40:19.171391854 (15h0m40.915448714s) [upload=15444]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:40:19.171395665 (15h0m40.915452525s) [upload=15444]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:40:19.331296850 (15h0m41.075353715s) [upload=15444]: Done uploading segments... Aug 8 00:40:19.331319872 (15h0m41.075376736s) [upload=15444]: Upload finished. err: <nil> Aug 8 00:41:14.342295084 (15h1m36.086351944s) [upload=15459]: Starting upload Aug 8 00:41:14.342370744 (15h1m36.086427596s) [upload=15459]: Done starting upload Aug 8 00:41:14.342453152 (15h1m36.086510013s) [upload=15459]: Uploading segments... Aug 8 00:41:14.342497220 (15h1m36.086554158s) [upload=15459]: Got next segment. Inline: true Aug 8 00:41:14.342504023 (15h1m36.08656089s) [upload=15459]: Waiting for error group managing segments... Aug 8 00:41:14.342588479 (15h1m36.086645343s) [upload=15459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:41:14.342612029 (15h1m36.086668893s) [upload=15459]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:41:14.342616257 (15h1m36.08667312s) [upload=15459]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:41:14.552238378 (15h1m36.296295229s) [upload=15459]: Done uploading segments... Aug 8 00:41:14.552258831 (15h1m36.296315682s) [upload=15459]: Upload finished. err: <nil> Aug 8 00:41:33.520869236 (15h1m55.264926087s) [upload=15470]: Starting upload Aug 8 00:41:33.520992007 (15h1m55.265048867s) [upload=15470]: Done starting upload Aug 8 00:41:33.521087418 (15h1m55.265144282s) [upload=15470]: Uploading segments... Aug 8 00:41:33.521174747 (15h1m55.265231612s) [upload=15470]: Got next segment. Inline: true Aug 8 00:41:33.521182648 (15h1m55.265239509s) [upload=15470]: Waiting for error group managing segments... Aug 8 00:41:33.521241390 (15h1m55.265298239s) [upload=15470]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:41:33.521247030 (15h1m55.26530389s) [upload=15470]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:41:33.521250440 (15h1m55.265307301s) [upload=15470]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:41:33.674374631 (15h1m55.418431478s) [upload=15470]: Done uploading segments... Aug 8 00:41:33.674392958 (15h1m55.418449804s) [upload=15470]: Upload finished. err: <nil> Aug 8 00:42:13.521243964 (15h2m35.26530081s) [upload=15480]: Starting upload Aug 8 00:42:13.521322202 (15h2m35.265379069s) [upload=15480]: Done starting upload Aug 8 00:42:13.521399949 (15h2m35.265456796s) [upload=15480]: Uploading segments... Aug 8 00:42:13.521486038 (15h2m35.265542908s) [upload=15480]: Got next segment. Inline: true Aug 8 00:42:13.521492566 (15h2m35.265549425s) [upload=15480]: Waiting for error group managing segments... Aug 8 00:42:13.521546734 (15h2m35.26560358s) [upload=15480]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:42:13.521552405 (15h2m35.265609264s) [upload=15480]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:42:13.521555275 (15h2m35.265612134s) [upload=15480]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:42:13.690373889 (15h2m35.434430739s) [upload=15480]: Done uploading segments... Aug 8 00:42:13.690405230 (15h2m35.434462081s) [upload=15480]: Upload finished. err: <nil> Aug 8 00:42:43.968444265 (15h3m5.712501112s) [upload=15489]: Starting upload Aug 8 00:42:43.968495052 (15h3m5.712551899s) [upload=15489]: Done starting upload Aug 8 00:42:43.968563760 (15h3m5.712620606s) [upload=15489]: Uploading segments... Aug 8 00:42:43.968604135 (15h3m5.712661001s) [upload=15489]: Got next segment. Inline: true Aug 8 00:42:43.968608788 (15h3m5.712665649s) [upload=15489]: Waiting for error group managing segments... Aug 8 00:42:43.968647123 (15h3m5.712703969s) [upload=15489]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:42:43.968652138 (15h3m5.712708998s) [upload=15489]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:42:43.968654778 (15h3m5.712711638s) [upload=15489]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:42:44.310945173 (15h3m6.055002023s) [upload=15489]: Done uploading segments... Aug 8 00:42:44.311090602 (15h3m6.055147463s) [upload=15489]: Upload finished. err: <nil> Aug 8 00:43:08.872038018 (15h3m30.616094879s) [upload=15496]: Starting upload Aug 8 00:43:08.872131319 (15h3m30.616188187s) [upload=15496]: Done starting upload Aug 8 00:43:08.872241899 (15h3m30.616298752s) [upload=15496]: Uploading segments... Aug 8 00:43:08.872314522 (15h3m30.616371385s) [upload=15496]: Got next segment. Inline: true Aug 8 00:43:08.872320103 (15h3m30.616376964s) [upload=15496]: Waiting for error group managing segments... Aug 8 00:43:08.872387129 (15h3m30.616443979s) [upload=15496]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:43:08.872392943 (15h3m30.616449792s) [upload=15496]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:43:08.872396750 (15h3m30.61645361s) [upload=15496]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:43:09.042725445 (15h3m30.786782296s) [upload=15496]: Done uploading segments... Aug 8 00:43:09.042744523 (15h3m30.786801373s) [upload=15496]: Upload finished. err: <nil> Aug 8 00:43:58.382659382 (15h4m20.126716247s) [upload=15511]: Starting upload Aug 8 00:43:58.382746751 (15h4m20.126803611s) [upload=15511]: Done starting upload Aug 8 00:43:58.382850091 (15h4m20.126906951s) [upload=15511]: Uploading segments... Aug 8 00:43:58.382921355 (15h4m20.126978226s) [upload=15511]: Got next segment. Inline: true Aug 8 00:43:58.382928916 (15h4m20.12698578s) [upload=15511]: Waiting for error group managing segments... Aug 8 00:43:58.383111923 (15h4m20.127168791s) [upload=15511]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:43:58.383121251 (15h4m20.127178114s) [upload=15511]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:43:58.383125081 (15h4m20.127181945s) [upload=15511]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:43:58.536263481 (15h4m20.280320332s) [upload=15511]: Done uploading segments... Aug 8 00:43:58.536281876 (15h4m20.28033873s) [upload=15511]: Upload finished. err: <nil> Aug 8 00:45:01.015818127 (15h5m22.759874977s) [upload=15529]: Starting upload Aug 8 00:45:01.015874109 (15h5m22.759930972s) [upload=15529]: Done starting upload Aug 8 00:45:01.015956182 (15h5m22.76001303s) [upload=15529]: Uploading segments... Aug 8 00:45:01.016043908 (15h5m22.760100776s) [upload=15529]: Got next segment. Inline: true Aug 8 00:45:01.016049919 (15h5m22.760106782s) [upload=15529]: Waiting for error group managing segments... Aug 8 00:45:01.016115031 (15h5m22.760171895s) [upload=15529]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:45:01.016122390 (15h5m22.760179257s) [upload=15529]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:45:01.016125863 (15h5m22.760182727s) [upload=15529]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:45:01.173239782 (15h5m22.91729665s) [upload=15529]: Done uploading segments... Aug 8 00:45:01.173279478 (15h5m22.917336328s) [upload=15529]: Upload finished. err: <nil> Duration: 831.212646ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 913.710059ms FAIL (2025-08-08 00:48:26 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== stItem_ObjectCommit Aug 8 00:37:35.928930883 (14h57m57.672987755s) [upload=15397]: Done uploading segments... Aug 8 00:37:35.928967703 (14h57m57.673024563s) [upload=15397]: Upload finished. err: <nil> Aug 8 00:37:56.695554589 (14h58m18.439611449s) [upload=15402]: Starting upload Aug 8 00:37:56.695609369 (14h58m18.43966623s) [upload=15402]: Done starting upload Aug 8 00:37:56.695671726 (14h58m18.439728574s) [upload=15402]: Uploading segments... Aug 8 00:37:56.695713147 (14h58m18.439770016s) [upload=15402]: Got next segment. Inline: true Aug 8 00:37:56.695718707 (14h58m18.439775558s) [upload=15402]: Waiting for error group managing segments... Aug 8 00:37:56.695760492 (14h58m18.439817342s) [upload=15402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:37:56.695765786 (14h58m18.439822646s) [upload=15402]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:37:56.695769297 (14h58m18.439826158s) [upload=15402]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:37:56.839484031 (14h58m18.583540885s) [upload=15402]: Done uploading segments... Aug 8 00:37:56.839508115 (14h58m18.583564984s) [upload=15402]: Upload finished. err: <nil> Aug 8 00:39:08.555730292 (14h59m30.299787146s) [upload=15423]: Starting upload Aug 8 00:39:08.555824312 (14h59m30.299881164s) [upload=15423]: Done starting upload Aug 8 00:39:08.555930281 (14h59m30.299987134s) [upload=15423]: Uploading segments... Aug 8 00:39:08.555989277 (14h59m30.300046148s) [upload=15423]: Got next segment. Inline: true Aug 8 00:39:08.555995450 (14h59m30.300052299s) [upload=15423]: Waiting for error group managing segments... Aug 8 00:39:08.556058440 (14h59m30.30011529s) [upload=15423]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:39:08.556064571 (14h59m30.300121422s) [upload=15423]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:39:08.556069271 (14h59m30.300126124s) [upload=15423]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:39:08.697938297 (14h59m30.441995159s) [upload=15423]: Done uploading segments... Aug 8 00:39:08.697959132 (14h59m30.442015982s) [upload=15423]: Upload finished. err: <nil> Aug 8 00:40:19.170886308 (15h0m40.914943159s) [upload=15444]: Starting upload Aug 8 00:40:19.170949280 (15h0m40.91500614s) [upload=15444]: Done starting upload Aug 8 00:40:19.171171058 (15h0m40.915227903s) [upload=15444]: Uploading segments... Aug 8 00:40:19.171300409 (15h0m40.915357281s) [upload=15444]: Got next segment. Inline: true Aug 8 00:40:19.171309541 (15h0m40.915366409s) [upload=15444]: Waiting for error group managing segments... Aug 8 00:40:19.171385631 (15h0m40.915442482s) [upload=15444]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:40:19.171391854 (15h0m40.915448714s) [upload=15444]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:40:19.171395665 (15h0m40.915452525s) [upload=15444]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:40:19.331296850 (15h0m41.075353715s) [upload=15444]: Done uploading segments... Aug 8 00:40:19.331319872 (15h0m41.075376736s) [upload=15444]: Upload finished. err: <nil> Aug 8 00:41:14.342295084 (15h1m36.086351944s) [upload=15459]: Starting upload Aug 8 00:41:14.342370744 (15h1m36.086427596s) [upload=15459]: Done starting upload Aug 8 00:41:14.342453152 (15h1m36.086510013s) [upload=15459]: Uploading segments... Aug 8 00:41:14.342497220 (15h1m36.086554158s) [upload=15459]: Got next segment. Inline: true Aug 8 00:41:14.342504023 (15h1m36.08656089s) [upload=15459]: Waiting for error group managing segments... Aug 8 00:41:14.342588479 (15h1m36.086645343s) [upload=15459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:41:14.342612029 (15h1m36.086668893s) [upload=15459]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:41:14.342616257 (15h1m36.08667312s) [upload=15459]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:41:14.552238378 (15h1m36.296295229s) [upload=15459]: Done uploading segments... Aug 8 00:41:14.552258831 (15h1m36.296315682s) [upload=15459]: Upload finished. err: <nil> Aug 8 00:41:33.520869236 (15h1m55.264926087s) [upload=15470]: Starting upload Aug 8 00:41:33.520992007 (15h1m55.265048867s) [upload=15470]: Done starting upload Aug 8 00:41:33.521087418 (15h1m55.265144282s) [upload=15470]: Uploading segments... Aug 8 00:41:33.521174747 (15h1m55.265231612s) [upload=15470]: Got next segment. Inline: true Aug 8 00:41:33.521182648 (15h1m55.265239509s) [upload=15470]: Waiting for error group managing segments... Aug 8 00:41:33.521241390 (15h1m55.265298239s) [upload=15470]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:41:33.521247030 (15h1m55.26530389s) [upload=15470]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:41:33.521250440 (15h1m55.265307301s) [upload=15470]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:41:33.674374631 (15h1m55.418431478s) [upload=15470]: Done uploading segments... Aug 8 00:41:33.674392958 (15h1m55.418449804s) [upload=15470]: Upload finished. err: <nil> Aug 8 00:42:13.521243964 (15h2m35.26530081s) [upload=15480]: Starting upload Aug 8 00:42:13.521322202 (15h2m35.265379069s) [upload=15480]: Done starting upload Aug 8 00:42:13.521399949 (15h2m35.265456796s) [upload=15480]: Uploading segments... Aug 8 00:42:13.521486038 (15h2m35.265542908s) [upload=15480]: Got next segment. Inline: true Aug 8 00:42:13.521492566 (15h2m35.265549425s) [upload=15480]: Waiting for error group managing segments... Aug 8 00:42:13.521546734 (15h2m35.26560358s) [upload=15480]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:42:13.521552405 (15h2m35.265609264s) [upload=15480]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:42:13.521555275 (15h2m35.265612134s) [upload=15480]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:42:13.690373889 (15h2m35.434430739s) [upload=15480]: Done uploading segments... Aug 8 00:42:13.690405230 (15h2m35.434462081s) [upload=15480]: Upload finished. err: <nil> Aug 8 00:42:43.968444265 (15h3m5.712501112s) [upload=15489]: Starting upload Aug 8 00:42:43.968495052 (15h3m5.712551899s) [upload=15489]: Done starting upload Aug 8 00:42:43.968563760 (15h3m5.712620606s) [upload=15489]: Uploading segments... Aug 8 00:42:43.968604135 (15h3m5.712661001s) [upload=15489]: Got next segment. Inline: true Aug 8 00:42:43.968608788 (15h3m5.712665649s) [upload=15489]: Waiting for error group managing segments... Aug 8 00:42:43.968647123 (15h3m5.712703969s) [upload=15489]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:42:43.968652138 (15h3m5.712708998s) [upload=15489]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:42:43.968654778 (15h3m5.712711638s) [upload=15489]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:42:44.310945173 (15h3m6.055002023s) [upload=15489]: Done uploading segments... Aug 8 00:42:44.311090602 (15h3m6.055147463s) [upload=15489]: Upload finished. err: <nil> Aug 8 00:43:08.872038018 (15h3m30.616094879s) [upload=15496]: Starting upload Aug 8 00:43:08.872131319 (15h3m30.616188187s) [upload=15496]: Done starting upload Aug 8 00:43:08.872241899 (15h3m30.616298752s) [upload=15496]: Uploading segments... Aug 8 00:43:08.872314522 (15h3m30.616371385s) [upload=15496]: Got next segment. Inline: true Aug 8 00:43:08.872320103 (15h3m30.616376964s) [upload=15496]: Waiting for error group managing segments... Aug 8 00:43:08.872387129 (15h3m30.616443979s) [upload=15496]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:43:08.872392943 (15h3m30.616449792s) [upload=15496]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:43:08.872396750 (15h3m30.61645361s) [upload=15496]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:43:09.042725445 (15h3m30.786782296s) [upload=15496]: Done uploading segments... Aug 8 00:43:09.042744523 (15h3m30.786801373s) [upload=15496]: Upload finished. err: <nil> Aug 8 00:43:58.382659382 (15h4m20.126716247s) [upload=15511]: Starting upload Aug 8 00:43:58.382746751 (15h4m20.126803611s) [upload=15511]: Done starting upload Aug 8 00:43:58.382850091 (15h4m20.126906951s) [upload=15511]: Uploading segments... Aug 8 00:43:58.382921355 (15h4m20.126978226s) [upload=15511]: Got next segment. Inline: true Aug 8 00:43:58.382928916 (15h4m20.12698578s) [upload=15511]: Waiting for error group managing segments... Aug 8 00:43:58.383111923 (15h4m20.127168791s) [upload=15511]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:43:58.383121251 (15h4m20.127178114s) [upload=15511]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:43:58.383125081 (15h4m20.127181945s) [upload=15511]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:43:58.536263481 (15h4m20.280320332s) [upload=15511]: Done uploading segments... Aug 8 00:43:58.536281876 (15h4m20.28033873s) [upload=15511]: Upload finished. err: <nil> Aug 8 00:45:01.015818127 (15h5m22.759874977s) [upload=15529]: Starting upload Aug 8 00:45:01.015874109 (15h5m22.759930972s) [upload=15529]: Done starting upload Aug 8 00:45:01.015956182 (15h5m22.76001303s) [upload=15529]: Uploading segments... Aug 8 00:45:01.016043908 (15h5m22.760100776s) [upload=15529]: Got next segment. Inline: true Aug 8 00:45:01.016049919 (15h5m22.760106782s) [upload=15529]: Waiting for error group managing segments... Aug 8 00:45:01.016115031 (15h5m22.760171895s) [upload=15529]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:45:01.016122390 (15h5m22.760179257s) [upload=15529]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:45:01.016125863 (15h5m22.760182727s) [upload=15529]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:45:01.173239782 (15h5m22.91729665s) [upload=15529]: Done uploading segments... Aug 8 00:45:01.173279478 (15h5m22.917336328s) [upload=15529]: Upload finished. err: <nil> Duration: 913.755339ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [............X...........................................X...] duration: 2.440906281s FAIL (2025-08-08 00:48:18 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Done starting upload Aug 8 00:45:16.266121630 (15h5m38.01017848s) [upload=15533]: Uploading segments... Aug 8 00:45:16.266428624 (15h5m38.010485484s) [upload=15533]: Got next segment. Inline: false Aug 8 00:45:16.266497281 (15h5m38.010554191s) [upload=15533]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:45:16.266512043 (15h5m38.010568914s) [upload=15533]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:45:16.467813786 (15h5m38.211870647s) [seg_pos={0 0}, upload=15533]: Begin upload segment... Aug 8 00:45:16.467883230 (15h5m38.211940081s) [seg_pos={0 0}, upload=15533]: Done begin upload segment. Aug 8 00:45:16.467935054 (15h5m38.211991969s) [noise=true , address=104.36.82.60:4369 , piece_id=WVPSCPUKKSYS4T3JIEEVDTS7WSFYXNGVQA6QNLGO3BKC3EKEYJGQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.467945924 (15h5m38.212002789s) [noise=true , address=104.36.82.54:4192 , piece_id=FDJHGRVMHAFB5JZM4Z77MF3NHRU3T4FNQPUAOXGP5NRCM5Q7RVRA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468440400 (15h5m38.212497272s) [noise=true , address=104.36.82.34:4577 , piece_id=4PTCNUDROG6OXW5DBUEMBAGPEMEMVLIW5OXRP2LPC55J2PTK3PAA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468457544 (15h5m38.21251441s) [noise=true , address=104.36.82.44:4887 , piece_id=FC4PG4NZJHR35DZLGLFZ6EJZYA2MJWLIDPBYHZCNJKJI4OVX5G4Q, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468847577 (15h5m38.212904441s) [noise=true , address=104.36.82.62:4430 , piece_id=JGWQSS5KYYCBOITFMENDLSGK6XSDVOT7TMRGJBZ55KBXX54ENY6A, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468887696 (15h5m38.21294456s) [noise=true , address=79.127.205.233:4633 , piece_id=F4RDTCFPIWZS4U65QMU5YXWOZSPPDIDMLJ26G3DCN3YV2UMAEFWA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469228336 (15h5m38.2132852s) [noise=true , address=104.36.82.43:4846 , piece_id=73PPCVC3HGFJDIB63URF7XA6PC37VCYLBCOMTSSFNC5UYZR6M5YA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469268209 (15h5m38.213325082s) [noise=true , address=64.49.13.140:4515 , piece_id=AQTWEM35E7E6YUISEOQGTPTPKD6RDOMKXLK267UILGD77273OY4A, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469613163 (15h5m38.21367003s) [noise=true , address=104.36.82.16:4307 , piece_id=B7ZUFMTGVRPCNJSTBCXMQRZW2AGKPXM2LJRDLYZWPXQ4F5JJ75UQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469702757 (15h5m38.213759621s) [noise=true , address=104.36.82.9:4038 , piece_id=RVXFCLSREWSON62WSTYVISC52HN33VUN2BHJ7UR72MEBMTFKQV6A, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470036233 (15h5m38.214093098s) [noise=true , address=104.36.82.23:4464 , piece_id=4HNT67XCPCNCECLCE36RVAHTJ7VBAADPQ7VMC7IASAUPVVEKIVWQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470129321 (15h5m38.214186184s) [noise=true , address=45.91.116.6:4982 , piece_id=NV3WES7XAJBCNO5EQMWFK6SNUEYE63YKTQC5ZMRF7URP6SX3FXGA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470411978 (15h5m38.214468842s) [noise=true , address=104.36.82.23:4452 , piece_id=B4XGMOK4RCAIOV3VSJXFZ73O52E4WIRUGSPVTCCJ4XUGNJHUQWBA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470482224 (15h5m38.21453909s) [noise=true , address=104.36.82.14:4254 , piece_id=AIB65II65FK6M3EQHX6PVX27N4Q4BQKI5QGVP3PD6D34ZKXVCLZA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470792753 (15h5m38.214849603s) [noise=true , address=104.36.82.19:4405 , piece_id=SA2CPGDA75CYSUDYYZ6V4NKTG5AEIZC7SVYNL2OLBK3E5MSK5FQQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470877563 (15h5m38.214934486s) [noise=true , address=104.36.82.14:4243 , piece_id=HSR3LCZHIDLJSJ7WISDYOWGPLBFZ73MIXTR7MLFQ2TLUBYLRL4DQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.471502176 (15h5m38.215559147s) [noise=true , address=104.36.82.45:4910 , piece_id=OGS6W5AGDOACXIMZ4OH3GNQ43YUU6WOYJHKAHNMHJZBQPJUVVCRA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.471850592 (15h5m38.215907452s) [noise=true , address=79.127.201.215:4358 , piece_id=Z3PRSJIAQMKCL2ZU5TA7WZ2IBCVIXB2K5UJBMS3UCI5H6HDNTWBA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:17.050924903 (15h5m38.794981774s) [upload=15533]: Next returned nil segment Aug 8 00:45:17.051086092 (15h5m38.795142971s) [upload=15533]: Waiting for error group managing segments... Aug 8 00:45:17.225555422 (15h5m38.969612281s) [noise=true , address=104.36.82.62:4430 , piece_id=JGWQSS5KYYCBOITFMENDLSGK6XSDVOT7TMRGJBZ55KBXX54ENY6A, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.232285957 (15h5m38.97634281s) [noise=true , address=45.91.116.6:4982 , piece_id=NV3WES7XAJBCNO5EQMWFK6SNUEYE63YKTQC5ZMRF7URP6SX3FXGA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.239931943 (15h5m38.983988797s) [noise=true , address=104.36.82.45:4910 , piece_id=OGS6W5AGDOACXIMZ4OH3GNQ43YUU6WOYJHKAHNMHJZBQPJUVVCRA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.240442032 (15h5m38.984498886s) [noise=true , address=104.36.82.34:4577 , piece_id=4PTCNUDROG6OXW5DBUEMBAGPEMEMVLIW5OXRP2LPC55J2PTK3PAA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.280384119 (15h5m39.024440979s) [noise=true , address=104.36.82.19:4405 , piece_id=SA2CPGDA75CYSUDYYZ6V4NKTG5AEIZC7SVYNL2OLBK3E5MSK5FQQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.284571751 (15h5m39.028628621s) [noise=true , address=104.36.82.16:4307 , piece_id=B7ZUFMTGVRPCNJSTBCXMQRZW2AGKPXM2LJRDLYZWPXQ4F5JJ75UQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.295563563 (15h5m39.03962043s) [noise=true , address=104.36.82.14:4254 , piece_id=AIB65II65FK6M3EQHX6PVX27N4Q4BQKI5QGVP3PD6D34ZKXVCLZA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.367859627 (15h5m39.11191649s) [noise=true , address=104.36.82.44:4887 , piece_id=FC4PG4NZJHR35DZLGLFZ6EJZYA2MJWLIDPBYHZCNJKJI4OVX5G4Q, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.486413539 (15h5m39.2304704s) [noise=true , address=104.36.82.9:4038 , piece_id=RVXFCLSREWSON62WSTYVISC52HN33VUN2BHJ7UR72MEBMTFKQV6A, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.510192603 (15h5m39.254249451s) [noise=true , address=79.127.201.215:4358 , piece_id=Z3PRSJIAQMKCL2ZU5TA7WZ2IBCVIXB2K5UJBMS3UCI5H6HDNTWBA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.609092139 (15h5m39.353148988s) [noise=true , address=104.36.82.54:4192 , piece_id=FDJHGRVMHAFB5JZM4Z77MF3NHRU3T4FNQPUAOXGP5NRCM5Q7RVRA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.647464269 (15h5m39.391521122s) [noise=true , address=104.36.82.14:4243 , piece_id=HSR3LCZHIDLJSJ7WISDYOWGPLBFZ73MIXTR7MLFQ2TLUBYLRL4DQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.681479215 (15h5m39.425536078s) [noise=true , address=104.36.82.43:4846 , piece_id=73PPCVC3HGFJDIB63URF7XA6PC37VCYLBCOMTSSFNC5UYZR6M5YA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.697970794 (15h5m39.442027648s) [noise=true , address=104.36.82.60:4369 , piece_id=WVPSCPUKKSYS4T3JIEEVDTS7WSFYXNGVQA6QNLGO3BKC3EKEYJGQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.697997236 (15h5m39.442054151s) [seg_pos={0 0}, upload=15533]: Segment reached optimal threshold of 14 pieces. Aug 8 00:45:17.698203533 (15h5m39.442260383s) [noise=true , address=79.127.205.233:4633 , piece_id=F4RDTCFPIWZS4U65QMU5YXWOZSPPDIDMLJ26G3DCN3YV2UMAEFWA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EEYb4mCySvZ54qzmygCQ7hLyTGYpdNJMyrvPvHJjtfGos9usk): protocol: expected piece hash; context canceled; EOF Aug 8 00:45:17.698290533 (15h5m39.442347383s) [noise=true , address=64.49.13.140:4515 , piece_id=AQTWEM35E7E6YUISEOQGTPTPKD6RDOMKXLK267UILGD77273OY4A, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mY7sRFAeDLmKPqq8BgwnGtkdPTsDRR6soTBshsn4qhP7evARb): protocol: expected piece hash; context canceled; EOF Aug 8 00:45:17.698544485 (15h5m39.442601338s) [noise=true , address=104.36.82.23:4464 , piece_id=4HNT67XCPCNCECLCE36RVAHTJ7VBAADPQ7VMC7IASAUPVVEKIVWQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mLbGHntspYmsTVg8433HYSdAEMrXjGnDhmmvX1JmmgcxtdLgk): protocol: write tcp 139.59.144.22:39260->104.36.82.23:4464: use of closed network connection; write tcp 139.59.144.22:39260->104.36.82.23:4464: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39260->104.36.82.23:4464: use of closed network connection Aug 8 00:45:17.698701613 (15h5m39.442758474s) [noise=true , address=104.36.82.23:4452 , piece_id=B4XGMOK4RCAIOV3VSJXFZ73O52E4WIRUGSPVTCCJ4XUGNJHUQWBA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sAh1mLmzbtmc4o2FAZssW8Y7iDDNmNTCXdCS8xrH8PBVQqxG5): protocol: context canceled; context canceled Aug 8 00:45:17.698723379 (15h5m39.44278023s) [seg_pos={0 0}, upload=15533]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 00:45:17.698815435 (15h5m39.442872285s) [upload=15533]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:45:17.698820522 (15h5m39.442877383s) [upload=15533]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:45:17.860271466 (15h5m39.604328326s) [upload=15533]: Done uploading segments... Aug 8 00:45:17.860291088 (15h5m39.604347935s) [upload=15533]: Upload finished. err: <nil> Duration: 2.440936782s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 850.063152ms FAIL (2025-08-08 00:48:05 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== stItem_ObjectCommit Aug 8 00:37:35.928930883 (14h57m57.672987755s) [upload=15397]: Done uploading segments... Aug 8 00:37:35.928967703 (14h57m57.673024563s) [upload=15397]: Upload finished. err: <nil> Aug 8 00:37:56.695554589 (14h58m18.439611449s) [upload=15402]: Starting upload Aug 8 00:37:56.695609369 (14h58m18.43966623s) [upload=15402]: Done starting upload Aug 8 00:37:56.695671726 (14h58m18.439728574s) [upload=15402]: Uploading segments... Aug 8 00:37:56.695713147 (14h58m18.439770016s) [upload=15402]: Got next segment. Inline: true Aug 8 00:37:56.695718707 (14h58m18.439775558s) [upload=15402]: Waiting for error group managing segments... Aug 8 00:37:56.695760492 (14h58m18.439817342s) [upload=15402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:37:56.695765786 (14h58m18.439822646s) [upload=15402]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:37:56.695769297 (14h58m18.439826158s) [upload=15402]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:37:56.839484031 (14h58m18.583540885s) [upload=15402]: Done uploading segments... Aug 8 00:37:56.839508115 (14h58m18.583564984s) [upload=15402]: Upload finished. err: <nil> Aug 8 00:39:08.555730292 (14h59m30.299787146s) [upload=15423]: Starting upload Aug 8 00:39:08.555824312 (14h59m30.299881164s) [upload=15423]: Done starting upload Aug 8 00:39:08.555930281 (14h59m30.299987134s) [upload=15423]: Uploading segments... Aug 8 00:39:08.555989277 (14h59m30.300046148s) [upload=15423]: Got next segment. Inline: true Aug 8 00:39:08.555995450 (14h59m30.300052299s) [upload=15423]: Waiting for error group managing segments... Aug 8 00:39:08.556058440 (14h59m30.30011529s) [upload=15423]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:39:08.556064571 (14h59m30.300121422s) [upload=15423]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:39:08.556069271 (14h59m30.300126124s) [upload=15423]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:39:08.697938297 (14h59m30.441995159s) [upload=15423]: Done uploading segments... Aug 8 00:39:08.697959132 (14h59m30.442015982s) [upload=15423]: Upload finished. err: <nil> Aug 8 00:40:19.170886308 (15h0m40.914943159s) [upload=15444]: Starting upload Aug 8 00:40:19.170949280 (15h0m40.91500614s) [upload=15444]: Done starting upload Aug 8 00:40:19.171171058 (15h0m40.915227903s) [upload=15444]: Uploading segments... Aug 8 00:40:19.171300409 (15h0m40.915357281s) [upload=15444]: Got next segment. Inline: true Aug 8 00:40:19.171309541 (15h0m40.915366409s) [upload=15444]: Waiting for error group managing segments... Aug 8 00:40:19.171385631 (15h0m40.915442482s) [upload=15444]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:40:19.171391854 (15h0m40.915448714s) [upload=15444]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:40:19.171395665 (15h0m40.915452525s) [upload=15444]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:40:19.331296850 (15h0m41.075353715s) [upload=15444]: Done uploading segments... Aug 8 00:40:19.331319872 (15h0m41.075376736s) [upload=15444]: Upload finished. err: <nil> Aug 8 00:41:14.342295084 (15h1m36.086351944s) [upload=15459]: Starting upload Aug 8 00:41:14.342370744 (15h1m36.086427596s) [upload=15459]: Done starting upload Aug 8 00:41:14.342453152 (15h1m36.086510013s) [upload=15459]: Uploading segments... Aug 8 00:41:14.342497220 (15h1m36.086554158s) [upload=15459]: Got next segment. Inline: true Aug 8 00:41:14.342504023 (15h1m36.08656089s) [upload=15459]: Waiting for error group managing segments... Aug 8 00:41:14.342588479 (15h1m36.086645343s) [upload=15459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:41:14.342612029 (15h1m36.086668893s) [upload=15459]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:41:14.342616257 (15h1m36.08667312s) [upload=15459]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:41:14.552238378 (15h1m36.296295229s) [upload=15459]: Done uploading segments... Aug 8 00:41:14.552258831 (15h1m36.296315682s) [upload=15459]: Upload finished. err: <nil> Aug 8 00:41:33.520869236 (15h1m55.264926087s) [upload=15470]: Starting upload Aug 8 00:41:33.520992007 (15h1m55.265048867s) [upload=15470]: Done starting upload Aug 8 00:41:33.521087418 (15h1m55.265144282s) [upload=15470]: Uploading segments... Aug 8 00:41:33.521174747 (15h1m55.265231612s) [upload=15470]: Got next segment. Inline: true Aug 8 00:41:33.521182648 (15h1m55.265239509s) [upload=15470]: Waiting for error group managing segments... Aug 8 00:41:33.521241390 (15h1m55.265298239s) [upload=15470]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:41:33.521247030 (15h1m55.26530389s) [upload=15470]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:41:33.521250440 (15h1m55.265307301s) [upload=15470]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:41:33.674374631 (15h1m55.418431478s) [upload=15470]: Done uploading segments... Aug 8 00:41:33.674392958 (15h1m55.418449804s) [upload=15470]: Upload finished. err: <nil> Aug 8 00:42:13.521243964 (15h2m35.26530081s) [upload=15480]: Starting upload Aug 8 00:42:13.521322202 (15h2m35.265379069s) [upload=15480]: Done starting upload Aug 8 00:42:13.521399949 (15h2m35.265456796s) [upload=15480]: Uploading segments... Aug 8 00:42:13.521486038 (15h2m35.265542908s) [upload=15480]: Got next segment. Inline: true Aug 8 00:42:13.521492566 (15h2m35.265549425s) [upload=15480]: Waiting for error group managing segments... Aug 8 00:42:13.521546734 (15h2m35.26560358s) [upload=15480]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:42:13.521552405 (15h2m35.265609264s) [upload=15480]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:42:13.521555275 (15h2m35.265612134s) [upload=15480]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:42:13.690373889 (15h2m35.434430739s) [upload=15480]: Done uploading segments... Aug 8 00:42:13.690405230 (15h2m35.434462081s) [upload=15480]: Upload finished. err: <nil> Aug 8 00:42:43.968444265 (15h3m5.712501112s) [upload=15489]: Starting upload Aug 8 00:42:43.968495052 (15h3m5.712551899s) [upload=15489]: Done starting upload Aug 8 00:42:43.968563760 (15h3m5.712620606s) [upload=15489]: Uploading segments... Aug 8 00:42:43.968604135 (15h3m5.712661001s) [upload=15489]: Got next segment. Inline: true Aug 8 00:42:43.968608788 (15h3m5.712665649s) [upload=15489]: Waiting for error group managing segments... Aug 8 00:42:43.968647123 (15h3m5.712703969s) [upload=15489]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:42:43.968652138 (15h3m5.712708998s) [upload=15489]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:42:43.968654778 (15h3m5.712711638s) [upload=15489]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:42:44.310945173 (15h3m6.055002023s) [upload=15489]: Done uploading segments... Aug 8 00:42:44.311090602 (15h3m6.055147463s) [upload=15489]: Upload finished. err: <nil> Aug 8 00:43:08.872038018 (15h3m30.616094879s) [upload=15496]: Starting upload Aug 8 00:43:08.872131319 (15h3m30.616188187s) [upload=15496]: Done starting upload Aug 8 00:43:08.872241899 (15h3m30.616298752s) [upload=15496]: Uploading segments... Aug 8 00:43:08.872314522 (15h3m30.616371385s) [upload=15496]: Got next segment. Inline: true Aug 8 00:43:08.872320103 (15h3m30.616376964s) [upload=15496]: Waiting for error group managing segments... Aug 8 00:43:08.872387129 (15h3m30.616443979s) [upload=15496]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:43:08.872392943 (15h3m30.616449792s) [upload=15496]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:43:08.872396750 (15h3m30.61645361s) [upload=15496]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:43:09.042725445 (15h3m30.786782296s) [upload=15496]: Done uploading segments... Aug 8 00:43:09.042744523 (15h3m30.786801373s) [upload=15496]: Upload finished. err: <nil> Aug 8 00:43:58.382659382 (15h4m20.126716247s) [upload=15511]: Starting upload Aug 8 00:43:58.382746751 (15h4m20.126803611s) [upload=15511]: Done starting upload Aug 8 00:43:58.382850091 (15h4m20.126906951s) [upload=15511]: Uploading segments... Aug 8 00:43:58.382921355 (15h4m20.126978226s) [upload=15511]: Got next segment. Inline: true Aug 8 00:43:58.382928916 (15h4m20.12698578s) [upload=15511]: Waiting for error group managing segments... Aug 8 00:43:58.383111923 (15h4m20.127168791s) [upload=15511]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:43:58.383121251 (15h4m20.127178114s) [upload=15511]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:43:58.383125081 (15h4m20.127181945s) [upload=15511]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:43:58.536263481 (15h4m20.280320332s) [upload=15511]: Done uploading segments... Aug 8 00:43:58.536281876 (15h4m20.28033873s) [upload=15511]: Upload finished. err: <nil> Aug 8 00:45:01.015818127 (15h5m22.759874977s) [upload=15529]: Starting upload Aug 8 00:45:01.015874109 (15h5m22.759930972s) [upload=15529]: Done starting upload Aug 8 00:45:01.015956182 (15h5m22.76001303s) [upload=15529]: Uploading segments... Aug 8 00:45:01.016043908 (15h5m22.760100776s) [upload=15529]: Got next segment. Inline: true Aug 8 00:45:01.016049919 (15h5m22.760106782s) [upload=15529]: Waiting for error group managing segments... Aug 8 00:45:01.016115031 (15h5m22.760171895s) [upload=15529]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:45:01.016122390 (15h5m22.760179257s) [upload=15529]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:45:01.016125863 (15h5m22.760182727s) [upload=15529]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:45:01.173239782 (15h5m22.91729665s) [upload=15529]: Done uploading segments... Aug 8 00:45:01.173279478 (15h5m22.917336328s) [upload=15529]: Upload finished. err: <nil> Duration: 850.126207ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 1.168460663s FAIL (2025-08-08 00:48:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.168482381s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............X...........................................X..] duration: 2.572127449s FAIL (2025-08-08 00:47:33 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Done starting upload Aug 8 00:45:16.266121630 (15h5m38.01017848s) [upload=15533]: Uploading segments... Aug 8 00:45:16.266428624 (15h5m38.010485484s) [upload=15533]: Got next segment. Inline: false Aug 8 00:45:16.266497281 (15h5m38.010554191s) [upload=15533]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:45:16.266512043 (15h5m38.010568914s) [upload=15533]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:45:16.467813786 (15h5m38.211870647s) [seg_pos={0 0}, upload=15533]: Begin upload segment... Aug 8 00:45:16.467883230 (15h5m38.211940081s) [seg_pos={0 0}, upload=15533]: Done begin upload segment. Aug 8 00:45:16.467935054 (15h5m38.211991969s) [noise=true , address=104.36.82.60:4369 , piece_id=WVPSCPUKKSYS4T3JIEEVDTS7WSFYXNGVQA6QNLGO3BKC3EKEYJGQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.467945924 (15h5m38.212002789s) [noise=true , address=104.36.82.54:4192 , piece_id=FDJHGRVMHAFB5JZM4Z77MF3NHRU3T4FNQPUAOXGP5NRCM5Q7RVRA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468440400 (15h5m38.212497272s) [noise=true , address=104.36.82.34:4577 , piece_id=4PTCNUDROG6OXW5DBUEMBAGPEMEMVLIW5OXRP2LPC55J2PTK3PAA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468457544 (15h5m38.21251441s) [noise=true , address=104.36.82.44:4887 , piece_id=FC4PG4NZJHR35DZLGLFZ6EJZYA2MJWLIDPBYHZCNJKJI4OVX5G4Q, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468847577 (15h5m38.212904441s) [noise=true , address=104.36.82.62:4430 , piece_id=JGWQSS5KYYCBOITFMENDLSGK6XSDVOT7TMRGJBZ55KBXX54ENY6A, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468887696 (15h5m38.21294456s) [noise=true , address=79.127.205.233:4633 , piece_id=F4RDTCFPIWZS4U65QMU5YXWOZSPPDIDMLJ26G3DCN3YV2UMAEFWA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469228336 (15h5m38.2132852s) [noise=true , address=104.36.82.43:4846 , piece_id=73PPCVC3HGFJDIB63URF7XA6PC37VCYLBCOMTSSFNC5UYZR6M5YA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469268209 (15h5m38.213325082s) [noise=true , address=64.49.13.140:4515 , piece_id=AQTWEM35E7E6YUISEOQGTPTPKD6RDOMKXLK267UILGD77273OY4A, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469613163 (15h5m38.21367003s) [noise=true , address=104.36.82.16:4307 , piece_id=B7ZUFMTGVRPCNJSTBCXMQRZW2AGKPXM2LJRDLYZWPXQ4F5JJ75UQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469702757 (15h5m38.213759621s) [noise=true , address=104.36.82.9:4038 , piece_id=RVXFCLSREWSON62WSTYVISC52HN33VUN2BHJ7UR72MEBMTFKQV6A, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470036233 (15h5m38.214093098s) [noise=true , address=104.36.82.23:4464 , piece_id=4HNT67XCPCNCECLCE36RVAHTJ7VBAADPQ7VMC7IASAUPVVEKIVWQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470129321 (15h5m38.214186184s) [noise=true , address=45.91.116.6:4982 , piece_id=NV3WES7XAJBCNO5EQMWFK6SNUEYE63YKTQC5ZMRF7URP6SX3FXGA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470411978 (15h5m38.214468842s) [noise=true , address=104.36.82.23:4452 , piece_id=B4XGMOK4RCAIOV3VSJXFZ73O52E4WIRUGSPVTCCJ4XUGNJHUQWBA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470482224 (15h5m38.21453909s) [noise=true , address=104.36.82.14:4254 , piece_id=AIB65II65FK6M3EQHX6PVX27N4Q4BQKI5QGVP3PD6D34ZKXVCLZA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470792753 (15h5m38.214849603s) [noise=true , address=104.36.82.19:4405 , piece_id=SA2CPGDA75CYSUDYYZ6V4NKTG5AEIZC7SVYNL2OLBK3E5MSK5FQQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470877563 (15h5m38.214934486s) [noise=true , address=104.36.82.14:4243 , piece_id=HSR3LCZHIDLJSJ7WISDYOWGPLBFZ73MIXTR7MLFQ2TLUBYLRL4DQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.471502176 (15h5m38.215559147s) [noise=true , address=104.36.82.45:4910 , piece_id=OGS6W5AGDOACXIMZ4OH3GNQ43YUU6WOYJHKAHNMHJZBQPJUVVCRA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.471850592 (15h5m38.215907452s) [noise=true , address=79.127.201.215:4358 , piece_id=Z3PRSJIAQMKCL2ZU5TA7WZ2IBCVIXB2K5UJBMS3UCI5H6HDNTWBA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:17.050924903 (15h5m38.794981774s) [upload=15533]: Next returned nil segment Aug 8 00:45:17.051086092 (15h5m38.795142971s) [upload=15533]: Waiting for error group managing segments... Aug 8 00:45:17.225555422 (15h5m38.969612281s) [noise=true , address=104.36.82.62:4430 , piece_id=JGWQSS5KYYCBOITFMENDLSGK6XSDVOT7TMRGJBZ55KBXX54ENY6A, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.232285957 (15h5m38.97634281s) [noise=true , address=45.91.116.6:4982 , piece_id=NV3WES7XAJBCNO5EQMWFK6SNUEYE63YKTQC5ZMRF7URP6SX3FXGA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.239931943 (15h5m38.983988797s) [noise=true , address=104.36.82.45:4910 , piece_id=OGS6W5AGDOACXIMZ4OH3GNQ43YUU6WOYJHKAHNMHJZBQPJUVVCRA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.240442032 (15h5m38.984498886s) [noise=true , address=104.36.82.34:4577 , piece_id=4PTCNUDROG6OXW5DBUEMBAGPEMEMVLIW5OXRP2LPC55J2PTK3PAA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.280384119 (15h5m39.024440979s) [noise=true , address=104.36.82.19:4405 , piece_id=SA2CPGDA75CYSUDYYZ6V4NKTG5AEIZC7SVYNL2OLBK3E5MSK5FQQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.284571751 (15h5m39.028628621s) [noise=true , address=104.36.82.16:4307 , piece_id=B7ZUFMTGVRPCNJSTBCXMQRZW2AGKPXM2LJRDLYZWPXQ4F5JJ75UQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.295563563 (15h5m39.03962043s) [noise=true , address=104.36.82.14:4254 , piece_id=AIB65II65FK6M3EQHX6PVX27N4Q4BQKI5QGVP3PD6D34ZKXVCLZA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.367859627 (15h5m39.11191649s) [noise=true , address=104.36.82.44:4887 , piece_id=FC4PG4NZJHR35DZLGLFZ6EJZYA2MJWLIDPBYHZCNJKJI4OVX5G4Q, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.486413539 (15h5m39.2304704s) [noise=true , address=104.36.82.9:4038 , piece_id=RVXFCLSREWSON62WSTYVISC52HN33VUN2BHJ7UR72MEBMTFKQV6A, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.510192603 (15h5m39.254249451s) [noise=true , address=79.127.201.215:4358 , piece_id=Z3PRSJIAQMKCL2ZU5TA7WZ2IBCVIXB2K5UJBMS3UCI5H6HDNTWBA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.609092139 (15h5m39.353148988s) [noise=true , address=104.36.82.54:4192 , piece_id=FDJHGRVMHAFB5JZM4Z77MF3NHRU3T4FNQPUAOXGP5NRCM5Q7RVRA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.647464269 (15h5m39.391521122s) [noise=true , address=104.36.82.14:4243 , piece_id=HSR3LCZHIDLJSJ7WISDYOWGPLBFZ73MIXTR7MLFQ2TLUBYLRL4DQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.681479215 (15h5m39.425536078s) [noise=true , address=104.36.82.43:4846 , piece_id=73PPCVC3HGFJDIB63URF7XA6PC37VCYLBCOMTSSFNC5UYZR6M5YA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.697970794 (15h5m39.442027648s) [noise=true , address=104.36.82.60:4369 , piece_id=WVPSCPUKKSYS4T3JIEEVDTS7WSFYXNGVQA6QNLGO3BKC3EKEYJGQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.697997236 (15h5m39.442054151s) [seg_pos={0 0}, upload=15533]: Segment reached optimal threshold of 14 pieces. Aug 8 00:45:17.698203533 (15h5m39.442260383s) [noise=true , address=79.127.205.233:4633 , piece_id=F4RDTCFPIWZS4U65QMU5YXWOZSPPDIDMLJ26G3DCN3YV2UMAEFWA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EEYb4mCySvZ54qzmygCQ7hLyTGYpdNJMyrvPvHJjtfGos9usk): protocol: expected piece hash; context canceled; EOF Aug 8 00:45:17.698290533 (15h5m39.442347383s) [noise=true , address=64.49.13.140:4515 , piece_id=AQTWEM35E7E6YUISEOQGTPTPKD6RDOMKXLK267UILGD77273OY4A, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mY7sRFAeDLmKPqq8BgwnGtkdPTsDRR6soTBshsn4qhP7evARb): protocol: expected piece hash; context canceled; EOF Aug 8 00:45:17.698544485 (15h5m39.442601338s) [noise=true , address=104.36.82.23:4464 , piece_id=4HNT67XCPCNCECLCE36RVAHTJ7VBAADPQ7VMC7IASAUPVVEKIVWQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mLbGHntspYmsTVg8433HYSdAEMrXjGnDhmmvX1JmmgcxtdLgk): protocol: write tcp 139.59.144.22:39260->104.36.82.23:4464: use of closed network connection; write tcp 139.59.144.22:39260->104.36.82.23:4464: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39260->104.36.82.23:4464: use of closed network connection Aug 8 00:45:17.698701613 (15h5m39.442758474s) [noise=true , address=104.36.82.23:4452 , piece_id=B4XGMOK4RCAIOV3VSJXFZ73O52E4WIRUGSPVTCCJ4XUGNJHUQWBA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sAh1mLmzbtmc4o2FAZssW8Y7iDDNmNTCXdCS8xrH8PBVQqxG5): protocol: context canceled; context canceled Aug 8 00:45:17.698723379 (15h5m39.44278023s) [seg_pos={0 0}, upload=15533]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 00:45:17.698815435 (15h5m39.442872285s) [upload=15533]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:45:17.698820522 (15h5m39.442877383s) [upload=15533]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:45:17.860271466 (15h5m39.604328326s) [upload=15533]: Done uploading segments... Aug 8 00:45:17.860291088 (15h5m39.604347935s) [upload=15533]: Upload finished. err: <nil> Duration: 2.572182376s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 1.069603327s FAIL (2025-08-08 00:47:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.069638439s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 854.787494ms FAIL (2025-08-08 00:47:15 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== stItem_ObjectCommit Aug 8 00:37:35.928930883 (14h57m57.672987755s) [upload=15397]: Done uploading segments... Aug 8 00:37:35.928967703 (14h57m57.673024563s) [upload=15397]: Upload finished. err: <nil> Aug 8 00:37:56.695554589 (14h58m18.439611449s) [upload=15402]: Starting upload Aug 8 00:37:56.695609369 (14h58m18.43966623s) [upload=15402]: Done starting upload Aug 8 00:37:56.695671726 (14h58m18.439728574s) [upload=15402]: Uploading segments... Aug 8 00:37:56.695713147 (14h58m18.439770016s) [upload=15402]: Got next segment. Inline: true Aug 8 00:37:56.695718707 (14h58m18.439775558s) [upload=15402]: Waiting for error group managing segments... Aug 8 00:37:56.695760492 (14h58m18.439817342s) [upload=15402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:37:56.695765786 (14h58m18.439822646s) [upload=15402]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:37:56.695769297 (14h58m18.439826158s) [upload=15402]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:37:56.839484031 (14h58m18.583540885s) [upload=15402]: Done uploading segments... Aug 8 00:37:56.839508115 (14h58m18.583564984s) [upload=15402]: Upload finished. err: <nil> Aug 8 00:39:08.555730292 (14h59m30.299787146s) [upload=15423]: Starting upload Aug 8 00:39:08.555824312 (14h59m30.299881164s) [upload=15423]: Done starting upload Aug 8 00:39:08.555930281 (14h59m30.299987134s) [upload=15423]: Uploading segments... Aug 8 00:39:08.555989277 (14h59m30.300046148s) [upload=15423]: Got next segment. Inline: true Aug 8 00:39:08.555995450 (14h59m30.300052299s) [upload=15423]: Waiting for error group managing segments... Aug 8 00:39:08.556058440 (14h59m30.30011529s) [upload=15423]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:39:08.556064571 (14h59m30.300121422s) [upload=15423]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:39:08.556069271 (14h59m30.300126124s) [upload=15423]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:39:08.697938297 (14h59m30.441995159s) [upload=15423]: Done uploading segments... Aug 8 00:39:08.697959132 (14h59m30.442015982s) [upload=15423]: Upload finished. err: <nil> Aug 8 00:40:19.170886308 (15h0m40.914943159s) [upload=15444]: Starting upload Aug 8 00:40:19.170949280 (15h0m40.91500614s) [upload=15444]: Done starting upload Aug 8 00:40:19.171171058 (15h0m40.915227903s) [upload=15444]: Uploading segments... Aug 8 00:40:19.171300409 (15h0m40.915357281s) [upload=15444]: Got next segment. Inline: true Aug 8 00:40:19.171309541 (15h0m40.915366409s) [upload=15444]: Waiting for error group managing segments... Aug 8 00:40:19.171385631 (15h0m40.915442482s) [upload=15444]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:40:19.171391854 (15h0m40.915448714s) [upload=15444]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:40:19.171395665 (15h0m40.915452525s) [upload=15444]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:40:19.331296850 (15h0m41.075353715s) [upload=15444]: Done uploading segments... Aug 8 00:40:19.331319872 (15h0m41.075376736s) [upload=15444]: Upload finished. err: <nil> Aug 8 00:41:14.342295084 (15h1m36.086351944s) [upload=15459]: Starting upload Aug 8 00:41:14.342370744 (15h1m36.086427596s) [upload=15459]: Done starting upload Aug 8 00:41:14.342453152 (15h1m36.086510013s) [upload=15459]: Uploading segments... Aug 8 00:41:14.342497220 (15h1m36.086554158s) [upload=15459]: Got next segment. Inline: true Aug 8 00:41:14.342504023 (15h1m36.08656089s) [upload=15459]: Waiting for error group managing segments... Aug 8 00:41:14.342588479 (15h1m36.086645343s) [upload=15459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:41:14.342612029 (15h1m36.086668893s) [upload=15459]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:41:14.342616257 (15h1m36.08667312s) [upload=15459]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:41:14.552238378 (15h1m36.296295229s) [upload=15459]: Done uploading segments... Aug 8 00:41:14.552258831 (15h1m36.296315682s) [upload=15459]: Upload finished. err: <nil> Aug 8 00:41:33.520869236 (15h1m55.264926087s) [upload=15470]: Starting upload Aug 8 00:41:33.520992007 (15h1m55.265048867s) [upload=15470]: Done starting upload Aug 8 00:41:33.521087418 (15h1m55.265144282s) [upload=15470]: Uploading segments... Aug 8 00:41:33.521174747 (15h1m55.265231612s) [upload=15470]: Got next segment. Inline: true Aug 8 00:41:33.521182648 (15h1m55.265239509s) [upload=15470]: Waiting for error group managing segments... Aug 8 00:41:33.521241390 (15h1m55.265298239s) [upload=15470]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:41:33.521247030 (15h1m55.26530389s) [upload=15470]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:41:33.521250440 (15h1m55.265307301s) [upload=15470]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:41:33.674374631 (15h1m55.418431478s) [upload=15470]: Done uploading segments... Aug 8 00:41:33.674392958 (15h1m55.418449804s) [upload=15470]: Upload finished. err: <nil> Aug 8 00:42:13.521243964 (15h2m35.26530081s) [upload=15480]: Starting upload Aug 8 00:42:13.521322202 (15h2m35.265379069s) [upload=15480]: Done starting upload Aug 8 00:42:13.521399949 (15h2m35.265456796s) [upload=15480]: Uploading segments... Aug 8 00:42:13.521486038 (15h2m35.265542908s) [upload=15480]: Got next segment. Inline: true Aug 8 00:42:13.521492566 (15h2m35.265549425s) [upload=15480]: Waiting for error group managing segments... Aug 8 00:42:13.521546734 (15h2m35.26560358s) [upload=15480]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:42:13.521552405 (15h2m35.265609264s) [upload=15480]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:42:13.521555275 (15h2m35.265612134s) [upload=15480]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:42:13.690373889 (15h2m35.434430739s) [upload=15480]: Done uploading segments... Aug 8 00:42:13.690405230 (15h2m35.434462081s) [upload=15480]: Upload finished. err: <nil> Aug 8 00:42:43.968444265 (15h3m5.712501112s) [upload=15489]: Starting upload Aug 8 00:42:43.968495052 (15h3m5.712551899s) [upload=15489]: Done starting upload Aug 8 00:42:43.968563760 (15h3m5.712620606s) [upload=15489]: Uploading segments... Aug 8 00:42:43.968604135 (15h3m5.712661001s) [upload=15489]: Got next segment. Inline: true Aug 8 00:42:43.968608788 (15h3m5.712665649s) [upload=15489]: Waiting for error group managing segments... Aug 8 00:42:43.968647123 (15h3m5.712703969s) [upload=15489]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:42:43.968652138 (15h3m5.712708998s) [upload=15489]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:42:43.968654778 (15h3m5.712711638s) [upload=15489]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:42:44.310945173 (15h3m6.055002023s) [upload=15489]: Done uploading segments... Aug 8 00:42:44.311090602 (15h3m6.055147463s) [upload=15489]: Upload finished. err: <nil> Aug 8 00:43:08.872038018 (15h3m30.616094879s) [upload=15496]: Starting upload Aug 8 00:43:08.872131319 (15h3m30.616188187s) [upload=15496]: Done starting upload Aug 8 00:43:08.872241899 (15h3m30.616298752s) [upload=15496]: Uploading segments... Aug 8 00:43:08.872314522 (15h3m30.616371385s) [upload=15496]: Got next segment. Inline: true Aug 8 00:43:08.872320103 (15h3m30.616376964s) [upload=15496]: Waiting for error group managing segments... Aug 8 00:43:08.872387129 (15h3m30.616443979s) [upload=15496]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:43:08.872392943 (15h3m30.616449792s) [upload=15496]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:43:08.872396750 (15h3m30.61645361s) [upload=15496]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:43:09.042725445 (15h3m30.786782296s) [upload=15496]: Done uploading segments... Aug 8 00:43:09.042744523 (15h3m30.786801373s) [upload=15496]: Upload finished. err: <nil> Aug 8 00:43:58.382659382 (15h4m20.126716247s) [upload=15511]: Starting upload Aug 8 00:43:58.382746751 (15h4m20.126803611s) [upload=15511]: Done starting upload Aug 8 00:43:58.382850091 (15h4m20.126906951s) [upload=15511]: Uploading segments... Aug 8 00:43:58.382921355 (15h4m20.126978226s) [upload=15511]: Got next segment. Inline: true Aug 8 00:43:58.382928916 (15h4m20.12698578s) [upload=15511]: Waiting for error group managing segments... Aug 8 00:43:58.383111923 (15h4m20.127168791s) [upload=15511]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:43:58.383121251 (15h4m20.127178114s) [upload=15511]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:43:58.383125081 (15h4m20.127181945s) [upload=15511]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:43:58.536263481 (15h4m20.280320332s) [upload=15511]: Done uploading segments... Aug 8 00:43:58.536281876 (15h4m20.28033873s) [upload=15511]: Upload finished. err: <nil> Aug 8 00:45:01.015818127 (15h5m22.759874977s) [upload=15529]: Starting upload Aug 8 00:45:01.015874109 (15h5m22.759930972s) [upload=15529]: Done starting upload Aug 8 00:45:01.015956182 (15h5m22.76001303s) [upload=15529]: Uploading segments... Aug 8 00:45:01.016043908 (15h5m22.760100776s) [upload=15529]: Got next segment. Inline: true Aug 8 00:45:01.016049919 (15h5m22.760106782s) [upload=15529]: Waiting for error group managing segments... Aug 8 00:45:01.016115031 (15h5m22.760171895s) [upload=15529]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:45:01.016122390 (15h5m22.760179257s) [upload=15529]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:45:01.016125863 (15h5m22.760182727s) [upload=15529]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:45:01.173239782 (15h5m22.91729665s) [upload=15529]: Done uploading segments... Aug 8 00:45:01.173279478 (15h5m22.917336328s) [upload=15529]: Upload finished. err: <nil> Duration: 854.820503ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............X...........................................X.] duration: 2.648379869s FAIL (2025-08-08 00:46:27 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Done starting upload Aug 8 00:45:16.266121630 (15h5m38.01017848s) [upload=15533]: Uploading segments... Aug 8 00:45:16.266428624 (15h5m38.010485484s) [upload=15533]: Got next segment. Inline: false Aug 8 00:45:16.266497281 (15h5m38.010554191s) [upload=15533]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:45:16.266512043 (15h5m38.010568914s) [upload=15533]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:45:16.467813786 (15h5m38.211870647s) [seg_pos={0 0}, upload=15533]: Begin upload segment... Aug 8 00:45:16.467883230 (15h5m38.211940081s) [seg_pos={0 0}, upload=15533]: Done begin upload segment. Aug 8 00:45:16.467935054 (15h5m38.211991969s) [noise=true , address=104.36.82.60:4369 , piece_id=WVPSCPUKKSYS4T3JIEEVDTS7WSFYXNGVQA6QNLGO3BKC3EKEYJGQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.467945924 (15h5m38.212002789s) [noise=true , address=104.36.82.54:4192 , piece_id=FDJHGRVMHAFB5JZM4Z77MF3NHRU3T4FNQPUAOXGP5NRCM5Q7RVRA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468440400 (15h5m38.212497272s) [noise=true , address=104.36.82.34:4577 , piece_id=4PTCNUDROG6OXW5DBUEMBAGPEMEMVLIW5OXRP2LPC55J2PTK3PAA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468457544 (15h5m38.21251441s) [noise=true , address=104.36.82.44:4887 , piece_id=FC4PG4NZJHR35DZLGLFZ6EJZYA2MJWLIDPBYHZCNJKJI4OVX5G4Q, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468847577 (15h5m38.212904441s) [noise=true , address=104.36.82.62:4430 , piece_id=JGWQSS5KYYCBOITFMENDLSGK6XSDVOT7TMRGJBZ55KBXX54ENY6A, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468887696 (15h5m38.21294456s) [noise=true , address=79.127.205.233:4633 , piece_id=F4RDTCFPIWZS4U65QMU5YXWOZSPPDIDMLJ26G3DCN3YV2UMAEFWA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469228336 (15h5m38.2132852s) [noise=true , address=104.36.82.43:4846 , piece_id=73PPCVC3HGFJDIB63URF7XA6PC37VCYLBCOMTSSFNC5UYZR6M5YA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469268209 (15h5m38.213325082s) [noise=true , address=64.49.13.140:4515 , piece_id=AQTWEM35E7E6YUISEOQGTPTPKD6RDOMKXLK267UILGD77273OY4A, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469613163 (15h5m38.21367003s) [noise=true , address=104.36.82.16:4307 , piece_id=B7ZUFMTGVRPCNJSTBCXMQRZW2AGKPXM2LJRDLYZWPXQ4F5JJ75UQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469702757 (15h5m38.213759621s) [noise=true , address=104.36.82.9:4038 , piece_id=RVXFCLSREWSON62WSTYVISC52HN33VUN2BHJ7UR72MEBMTFKQV6A, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470036233 (15h5m38.214093098s) [noise=true , address=104.36.82.23:4464 , piece_id=4HNT67XCPCNCECLCE36RVAHTJ7VBAADPQ7VMC7IASAUPVVEKIVWQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470129321 (15h5m38.214186184s) [noise=true , address=45.91.116.6:4982 , piece_id=NV3WES7XAJBCNO5EQMWFK6SNUEYE63YKTQC5ZMRF7URP6SX3FXGA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470411978 (15h5m38.214468842s) [noise=true , address=104.36.82.23:4452 , piece_id=B4XGMOK4RCAIOV3VSJXFZ73O52E4WIRUGSPVTCCJ4XUGNJHUQWBA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470482224 (15h5m38.21453909s) [noise=true , address=104.36.82.14:4254 , piece_id=AIB65II65FK6M3EQHX6PVX27N4Q4BQKI5QGVP3PD6D34ZKXVCLZA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470792753 (15h5m38.214849603s) [noise=true , address=104.36.82.19:4405 , piece_id=SA2CPGDA75CYSUDYYZ6V4NKTG5AEIZC7SVYNL2OLBK3E5MSK5FQQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470877563 (15h5m38.214934486s) [noise=true , address=104.36.82.14:4243 , piece_id=HSR3LCZHIDLJSJ7WISDYOWGPLBFZ73MIXTR7MLFQ2TLUBYLRL4DQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.471502176 (15h5m38.215559147s) [noise=true , address=104.36.82.45:4910 , piece_id=OGS6W5AGDOACXIMZ4OH3GNQ43YUU6WOYJHKAHNMHJZBQPJUVVCRA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.471850592 (15h5m38.215907452s) [noise=true , address=79.127.201.215:4358 , piece_id=Z3PRSJIAQMKCL2ZU5TA7WZ2IBCVIXB2K5UJBMS3UCI5H6HDNTWBA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:17.050924903 (15h5m38.794981774s) [upload=15533]: Next returned nil segment Aug 8 00:45:17.051086092 (15h5m38.795142971s) [upload=15533]: Waiting for error group managing segments... Aug 8 00:45:17.225555422 (15h5m38.969612281s) [noise=true , address=104.36.82.62:4430 , piece_id=JGWQSS5KYYCBOITFMENDLSGK6XSDVOT7TMRGJBZ55KBXX54ENY6A, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.232285957 (15h5m38.97634281s) [noise=true , address=45.91.116.6:4982 , piece_id=NV3WES7XAJBCNO5EQMWFK6SNUEYE63YKTQC5ZMRF7URP6SX3FXGA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.239931943 (15h5m38.983988797s) [noise=true , address=104.36.82.45:4910 , piece_id=OGS6W5AGDOACXIMZ4OH3GNQ43YUU6WOYJHKAHNMHJZBQPJUVVCRA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.240442032 (15h5m38.984498886s) [noise=true , address=104.36.82.34:4577 , piece_id=4PTCNUDROG6OXW5DBUEMBAGPEMEMVLIW5OXRP2LPC55J2PTK3PAA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.280384119 (15h5m39.024440979s) [noise=true , address=104.36.82.19:4405 , piece_id=SA2CPGDA75CYSUDYYZ6V4NKTG5AEIZC7SVYNL2OLBK3E5MSK5FQQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.284571751 (15h5m39.028628621s) [noise=true , address=104.36.82.16:4307 , piece_id=B7ZUFMTGVRPCNJSTBCXMQRZW2AGKPXM2LJRDLYZWPXQ4F5JJ75UQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.295563563 (15h5m39.03962043s) [noise=true , address=104.36.82.14:4254 , piece_id=AIB65II65FK6M3EQHX6PVX27N4Q4BQKI5QGVP3PD6D34ZKXVCLZA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.367859627 (15h5m39.11191649s) [noise=true , address=104.36.82.44:4887 , piece_id=FC4PG4NZJHR35DZLGLFZ6EJZYA2MJWLIDPBYHZCNJKJI4OVX5G4Q, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.486413539 (15h5m39.2304704s) [noise=true , address=104.36.82.9:4038 , piece_id=RVXFCLSREWSON62WSTYVISC52HN33VUN2BHJ7UR72MEBMTFKQV6A, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.510192603 (15h5m39.254249451s) [noise=true , address=79.127.201.215:4358 , piece_id=Z3PRSJIAQMKCL2ZU5TA7WZ2IBCVIXB2K5UJBMS3UCI5H6HDNTWBA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.609092139 (15h5m39.353148988s) [noise=true , address=104.36.82.54:4192 , piece_id=FDJHGRVMHAFB5JZM4Z77MF3NHRU3T4FNQPUAOXGP5NRCM5Q7RVRA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.647464269 (15h5m39.391521122s) [noise=true , address=104.36.82.14:4243 , piece_id=HSR3LCZHIDLJSJ7WISDYOWGPLBFZ73MIXTR7MLFQ2TLUBYLRL4DQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.681479215 (15h5m39.425536078s) [noise=true , address=104.36.82.43:4846 , piece_id=73PPCVC3HGFJDIB63URF7XA6PC37VCYLBCOMTSSFNC5UYZR6M5YA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.697970794 (15h5m39.442027648s) [noise=true , address=104.36.82.60:4369 , piece_id=WVPSCPUKKSYS4T3JIEEVDTS7WSFYXNGVQA6QNLGO3BKC3EKEYJGQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.697997236 (15h5m39.442054151s) [seg_pos={0 0}, upload=15533]: Segment reached optimal threshold of 14 pieces. Aug 8 00:45:17.698203533 (15h5m39.442260383s) [noise=true , address=79.127.205.233:4633 , piece_id=F4RDTCFPIWZS4U65QMU5YXWOZSPPDIDMLJ26G3DCN3YV2UMAEFWA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EEYb4mCySvZ54qzmygCQ7hLyTGYpdNJMyrvPvHJjtfGos9usk): protocol: expected piece hash; context canceled; EOF Aug 8 00:45:17.698290533 (15h5m39.442347383s) [noise=true , address=64.49.13.140:4515 , piece_id=AQTWEM35E7E6YUISEOQGTPTPKD6RDOMKXLK267UILGD77273OY4A, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mY7sRFAeDLmKPqq8BgwnGtkdPTsDRR6soTBshsn4qhP7evARb): protocol: expected piece hash; context canceled; EOF Aug 8 00:45:17.698544485 (15h5m39.442601338s) [noise=true , address=104.36.82.23:4464 , piece_id=4HNT67XCPCNCECLCE36RVAHTJ7VBAADPQ7VMC7IASAUPVVEKIVWQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mLbGHntspYmsTVg8433HYSdAEMrXjGnDhmmvX1JmmgcxtdLgk): protocol: write tcp 139.59.144.22:39260->104.36.82.23:4464: use of closed network connection; write tcp 139.59.144.22:39260->104.36.82.23:4464: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39260->104.36.82.23:4464: use of closed network connection Aug 8 00:45:17.698701613 (15h5m39.442758474s) [noise=true , address=104.36.82.23:4452 , piece_id=B4XGMOK4RCAIOV3VSJXFZ73O52E4WIRUGSPVTCCJ4XUGNJHUQWBA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sAh1mLmzbtmc4o2FAZssW8Y7iDDNmNTCXdCS8xrH8PBVQqxG5): protocol: context canceled; context canceled Aug 8 00:45:17.698723379 (15h5m39.44278023s) [seg_pos={0 0}, upload=15533]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 00:45:17.698815435 (15h5m39.442872285s) [upload=15533]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:45:17.698820522 (15h5m39.442877383s) [upload=15533]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:45:17.860271466 (15h5m39.604328326s) [upload=15533]: Done uploading segments... Aug 8 00:45:17.860291088 (15h5m39.604347935s) [upload=15533]: Upload finished. err: <nil> Duration: 2.648412587s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 1.121291807s FAIL (2025-08-08 00:46:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.121324806s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 910.305861ms FAIL (2025-08-08 00:46:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 0]: Starting upload Aug 7 23:53:17.153036870 (14h13m38.89709373s) [upload=14650]: Done starting upload Aug 7 23:53:17.153154678 (14h13m38.897211544s) [upload=14650]: Uploading segments... Aug 7 23:53:17.153233089 (14h13m38.897289957s) [upload=14650]: Got next segment. Inline: true Aug 7 23:53:17.153240256 (14h13m38.897297124s) [upload=14650]: Waiting for error group managing segments... Aug 7 23:53:17.153318892 (14h13m38.897375757s) [upload=14650]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:17.153325816 (14h13m38.89738268s) [upload=14650]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:17.153330453 (14h13m38.897387314s) [upload=14650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:16.875757982 (14h14m38.619814845s) [upload=14650]: Done uploading segments... Aug 7 23:54:16.875781849 (14h14m38.619838701s) [upload=14650]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:24.158448119 (14h14m45.90250498s) [upload=14667]: Starting upload Aug 7 23:54:24.158525946 (14h14m45.902582806s) [upload=14667]: Done starting upload Aug 7 23:54:24.158598972 (14h14m45.902655824s) [upload=14667]: Uploading segments... Aug 7 23:54:24.158654148 (14h14m45.902711017s) [upload=14667]: Got next segment. Inline: true Aug 7 23:54:24.158659995 (14h14m45.902716856s) [upload=14667]: Waiting for error group managing segments... Aug 7 23:54:24.158714896 (14h14m45.902771747s) [upload=14667]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:24.158720357 (14h14m45.902777209s) [upload=14667]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:24.158724105 (14h14m45.902780965s) [upload=14667]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:23.888400190 (14h15m45.63245704s) [upload=14667]: Done uploading segments... Aug 7 23:55:23.888419087 (14h15m45.632475938s) [upload=14667]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:24.297146309 (14h15m46.041203174s) [upload=14687]: Starting upload Aug 7 23:55:24.297232144 (14h15m46.04128901s) [upload=14687]: Done starting upload Aug 7 23:55:24.297315772 (14h15m46.041372619s) [upload=14687]: Uploading segments... Aug 7 23:55:24.297396238 (14h15m46.041453151s) [upload=14687]: Got next segment. Inline: true Aug 7 23:55:24.297403729 (14h15m46.04146059s) [upload=14687]: Waiting for error group managing segments... Aug 7 23:55:24.297444116 (14h15m46.041500976s) [upload=14687]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:24.297450971 (14h15m46.041507831s) [upload=14687]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:24.297453883 (14h15m46.041510749s) [upload=14687]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:24.439126204 (14h15m46.183183055s) [upload=14687]: Done uploading segments... Aug 7 23:55:24.439141121 (14h15m46.183197972s) [upload=14687]: Upload finished. err: <nil> Aug 7 23:56:03.068174127 (14h16m24.812230977s) [upload=14695]: Starting upload Aug 7 23:56:03.068258755 (14h16m24.812315617s) [upload=14695]: Done starting upload Aug 7 23:56:03.068368961 (14h16m24.812425812s) [upload=14695]: Uploading segments... Aug 7 23:56:03.068437851 (14h16m24.812494719s) [upload=14695]: Got next segment. Inline: true Aug 7 23:56:03.068445058 (14h16m24.812501927s) [upload=14695]: Waiting for error group managing segments... Aug 7 23:56:03.068516175 (14h16m24.812573049s) [upload=14695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:03.068523275 (14h16m24.812580137s) [upload=14695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:03.068527513 (14h16m24.812584488s) [upload=14695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:03.370378590 (14h16m25.114435452s) [upload=14695]: Done uploading segments... Aug 7 23:56:03.370401742 (14h16m25.114458656s) [upload=14695]: Upload finished. err: <nil> Aug 7 23:56:41.636734874 (14h17m3.380791742s) [upload=14707]: Starting upload Aug 7 23:56:41.636844337 (14h17m3.380901201s) [upload=14707]: Done starting upload Aug 7 23:56:41.636938361 (14h17m3.380995223s) [upload=14707]: Uploading segments... Aug 7 23:56:41.637004566 (14h17m3.381061436s) [upload=14707]: Got next segment. Inline: true Aug 7 23:56:41.637011720 (14h17m3.381068581s) [upload=14707]: Waiting for error group managing segments... Aug 7 23:56:41.637073380 (14h17m3.381130242s) [upload=14707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:41.637079684 (14h17m3.381136546s) [upload=14707]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:41.637083892 (14h17m3.381140753s) [upload=14707]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:41.369074470 (14h18m3.113131318s) [upload=14707]: Done uploading segments... Aug 7 23:57:41.369096313 (14h18m3.11315316s) [upload=14707]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:57:41.785857243 (14h18m3.529914104s) [upload=14726]: Starting upload Aug 7 23:57:41.785973295 (14h18m3.530030155s) [upload=14726]: Done starting upload Aug 7 23:57:41.786053418 (14h18m3.530110268s) [upload=14726]: Uploading segments... Aug 7 23:57:41.786115407 (14h18m3.530172273s) [upload=14726]: Got next segment. Inline: true Aug 7 23:57:41.786121268 (14h18m3.530178127s) [upload=14726]: Waiting for error group managing segments... Aug 7 23:57:41.786162041 (14h18m3.530218901s) [upload=14726]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:57:41.786167247 (14h18m3.530224108s) [upload=14726]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:57:41.786170583 (14h18m3.530227443s) [upload=14726]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:41.518215592 (14h19m3.262272452s) [upload=14726]: Done uploading segments... Aug 7 23:58:41.518237426 (14h19m3.262294275s) [upload=14726]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:58:49.080161447 (14h19m10.824218292s) [upload=14741]: Starting upload Aug 7 23:58:49.080236560 (14h19m10.824293407s) [upload=14741]: Done starting upload Aug 7 23:58:49.080313897 (14h19m10.824370744s) [upload=14741]: Uploading segments... Aug 7 23:58:49.080371825 (14h19m10.824428691s) [upload=14741]: Got next segment. Inline: true Aug 7 23:58:49.080376725 (14h19m10.824433586s) [upload=14741]: Waiting for error group managing segments... Aug 7 23:58:49.080418334 (14h19m10.824475197s) [upload=14741]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:49.080422560 (14h19m10.824479419s) [upload=14741]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:49.080438323 (14h19m10.824495186s) [upload=14741]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:49.220444289 (14h19m10.964501139s) [upload=14741]: Done uploading segments... Aug 7 23:58:49.220462660 (14h19m10.96451951s) [upload=14741]: Upload finished. err: <nil> Aug 7 23:59:26.271206280 (14h19m48.015263132s) [upload=14752]: Starting upload Aug 7 23:59:26.271300842 (14h19m48.015357706s) [upload=14752]: Done starting upload Aug 7 23:59:26.271409542 (14h19m48.015466389s) [upload=14752]: Uploading segments... Aug 7 23:59:26.271471969 (14h19m48.015528839s) [upload=14752]: Got next segment. Inline: true Aug 7 23:59:26.271478422 (14h19m48.015535286s) [upload=14752]: Waiting for error group managing segments... Aug 7 23:59:26.271524304 (14h19m48.015581168s) [upload=14752]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:26.271531803 (14h19m48.01558867s) [upload=14752]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:26.271535740 (14h19m48.015592604s) [upload=14752]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:26.006104454 (14h20m47.750161323s) [upload=14752]: Done uploading segments... Aug 8 00:00:26.006140966 (14h20m47.750197816s) [upload=14752]: Upload finished. err: metaclient: context deadline exceeded Aug 8 00:00:26.408809053 (14h20m48.152865927s) [upload=14766]: Starting upload Aug 8 00:00:26.408886561 (14h20m48.152943434s) [upload=14766]: Done starting upload Aug 8 00:00:26.408975679 (14h20m48.15303253s) [upload=14766]: Uploading segments... Aug 8 00:00:26.409028539 (14h20m48.153085476s) [upload=14766]: Got next segment. Inline: true Aug 8 00:00:26.409034850 (14h20m48.153091712s) [upload=14766]: Waiting for error group managing segments... Aug 8 00:00:26.409085190 (14h20m48.153142042s) [upload=14766]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:00:26.409090930 (14h20m48.153147793s) [upload=14766]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:00:26.409094742 (14h20m48.153151605s) [upload=14766]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:26.549853950 (14h20m48.2939108s) [upload=14766]: Done uploading segments... Aug 8 00:00:26.549878419 (14h20m48.293935265s) [upload=14766]: Upload finished. err: <nil> Aug 8 00:01:08.274172938 (14h21m30.018229799s) [upload=14780]: Starting upload Aug 8 00:01:08.274248443 (14h21m30.018305302s) [upload=14780]: Done starting upload Aug 8 00:01:08.274326739 (14h21m30.018383585s) [upload=14780]: Uploading segments... Aug 8 00:01:08.274380689 (14h21m30.018437549s) [upload=14780]: Got next segment. Inline: true Aug 8 00:01:08.274385704 (14h21m30.018442565s) [upload=14780]: Waiting for error group managing segments... Aug 8 00:01:08.274437335 (14h21m30.018494182s) [upload=14780]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:01:08.274442821 (14h21m30.018499667s) [upload=14780]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:01:08.274445609 (14h21m30.018502454s) [upload=14780]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:08.005343335 (14h22m29.749400197s) [upload=14780]: Done uploading segments... Aug 8 00:02:08.005365121 (14h22m29.749421982s) [upload=14780]: Upload finished. err: metaclient: context deadline exceeded Duration: 910.334872ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 965.390887ms FAIL (2025-08-08 00:46:01 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== stItem_ObjectCommit Aug 8 00:37:35.928930883 (14h57m57.672987755s) [upload=15397]: Done uploading segments... Aug 8 00:37:35.928967703 (14h57m57.673024563s) [upload=15397]: Upload finished. err: <nil> Aug 8 00:37:56.695554589 (14h58m18.439611449s) [upload=15402]: Starting upload Aug 8 00:37:56.695609369 (14h58m18.43966623s) [upload=15402]: Done starting upload Aug 8 00:37:56.695671726 (14h58m18.439728574s) [upload=15402]: Uploading segments... Aug 8 00:37:56.695713147 (14h58m18.439770016s) [upload=15402]: Got next segment. Inline: true Aug 8 00:37:56.695718707 (14h58m18.439775558s) [upload=15402]: Waiting for error group managing segments... Aug 8 00:37:56.695760492 (14h58m18.439817342s) [upload=15402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:37:56.695765786 (14h58m18.439822646s) [upload=15402]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:37:56.695769297 (14h58m18.439826158s) [upload=15402]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:37:56.839484031 (14h58m18.583540885s) [upload=15402]: Done uploading segments... Aug 8 00:37:56.839508115 (14h58m18.583564984s) [upload=15402]: Upload finished. err: <nil> Aug 8 00:39:08.555730292 (14h59m30.299787146s) [upload=15423]: Starting upload Aug 8 00:39:08.555824312 (14h59m30.299881164s) [upload=15423]: Done starting upload Aug 8 00:39:08.555930281 (14h59m30.299987134s) [upload=15423]: Uploading segments... Aug 8 00:39:08.555989277 (14h59m30.300046148s) [upload=15423]: Got next segment. Inline: true Aug 8 00:39:08.555995450 (14h59m30.300052299s) [upload=15423]: Waiting for error group managing segments... Aug 8 00:39:08.556058440 (14h59m30.30011529s) [upload=15423]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:39:08.556064571 (14h59m30.300121422s) [upload=15423]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:39:08.556069271 (14h59m30.300126124s) [upload=15423]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:39:08.697938297 (14h59m30.441995159s) [upload=15423]: Done uploading segments... Aug 8 00:39:08.697959132 (14h59m30.442015982s) [upload=15423]: Upload finished. err: <nil> Aug 8 00:40:19.170886308 (15h0m40.914943159s) [upload=15444]: Starting upload Aug 8 00:40:19.170949280 (15h0m40.91500614s) [upload=15444]: Done starting upload Aug 8 00:40:19.171171058 (15h0m40.915227903s) [upload=15444]: Uploading segments... Aug 8 00:40:19.171300409 (15h0m40.915357281s) [upload=15444]: Got next segment. Inline: true Aug 8 00:40:19.171309541 (15h0m40.915366409s) [upload=15444]: Waiting for error group managing segments... Aug 8 00:40:19.171385631 (15h0m40.915442482s) [upload=15444]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:40:19.171391854 (15h0m40.915448714s) [upload=15444]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:40:19.171395665 (15h0m40.915452525s) [upload=15444]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:40:19.331296850 (15h0m41.075353715s) [upload=15444]: Done uploading segments... Aug 8 00:40:19.331319872 (15h0m41.075376736s) [upload=15444]: Upload finished. err: <nil> Aug 8 00:41:14.342295084 (15h1m36.086351944s) [upload=15459]: Starting upload Aug 8 00:41:14.342370744 (15h1m36.086427596s) [upload=15459]: Done starting upload Aug 8 00:41:14.342453152 (15h1m36.086510013s) [upload=15459]: Uploading segments... Aug 8 00:41:14.342497220 (15h1m36.086554158s) [upload=15459]: Got next segment. Inline: true Aug 8 00:41:14.342504023 (15h1m36.08656089s) [upload=15459]: Waiting for error group managing segments... Aug 8 00:41:14.342588479 (15h1m36.086645343s) [upload=15459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:41:14.342612029 (15h1m36.086668893s) [upload=15459]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:41:14.342616257 (15h1m36.08667312s) [upload=15459]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:41:14.552238378 (15h1m36.296295229s) [upload=15459]: Done uploading segments... Aug 8 00:41:14.552258831 (15h1m36.296315682s) [upload=15459]: Upload finished. err: <nil> Aug 8 00:41:33.520869236 (15h1m55.264926087s) [upload=15470]: Starting upload Aug 8 00:41:33.520992007 (15h1m55.265048867s) [upload=15470]: Done starting upload Aug 8 00:41:33.521087418 (15h1m55.265144282s) [upload=15470]: Uploading segments... Aug 8 00:41:33.521174747 (15h1m55.265231612s) [upload=15470]: Got next segment. Inline: true Aug 8 00:41:33.521182648 (15h1m55.265239509s) [upload=15470]: Waiting for error group managing segments... Aug 8 00:41:33.521241390 (15h1m55.265298239s) [upload=15470]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:41:33.521247030 (15h1m55.26530389s) [upload=15470]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:41:33.521250440 (15h1m55.265307301s) [upload=15470]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:41:33.674374631 (15h1m55.418431478s) [upload=15470]: Done uploading segments... Aug 8 00:41:33.674392958 (15h1m55.418449804s) [upload=15470]: Upload finished. err: <nil> Aug 8 00:42:13.521243964 (15h2m35.26530081s) [upload=15480]: Starting upload Aug 8 00:42:13.521322202 (15h2m35.265379069s) [upload=15480]: Done starting upload Aug 8 00:42:13.521399949 (15h2m35.265456796s) [upload=15480]: Uploading segments... Aug 8 00:42:13.521486038 (15h2m35.265542908s) [upload=15480]: Got next segment. Inline: true Aug 8 00:42:13.521492566 (15h2m35.265549425s) [upload=15480]: Waiting for error group managing segments... Aug 8 00:42:13.521546734 (15h2m35.26560358s) [upload=15480]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:42:13.521552405 (15h2m35.265609264s) [upload=15480]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:42:13.521555275 (15h2m35.265612134s) [upload=15480]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:42:13.690373889 (15h2m35.434430739s) [upload=15480]: Done uploading segments... Aug 8 00:42:13.690405230 (15h2m35.434462081s) [upload=15480]: Upload finished. err: <nil> Aug 8 00:42:43.968444265 (15h3m5.712501112s) [upload=15489]: Starting upload Aug 8 00:42:43.968495052 (15h3m5.712551899s) [upload=15489]: Done starting upload Aug 8 00:42:43.968563760 (15h3m5.712620606s) [upload=15489]: Uploading segments... Aug 8 00:42:43.968604135 (15h3m5.712661001s) [upload=15489]: Got next segment. Inline: true Aug 8 00:42:43.968608788 (15h3m5.712665649s) [upload=15489]: Waiting for error group managing segments... Aug 8 00:42:43.968647123 (15h3m5.712703969s) [upload=15489]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:42:43.968652138 (15h3m5.712708998s) [upload=15489]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:42:43.968654778 (15h3m5.712711638s) [upload=15489]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:42:44.310945173 (15h3m6.055002023s) [upload=15489]: Done uploading segments... Aug 8 00:42:44.311090602 (15h3m6.055147463s) [upload=15489]: Upload finished. err: <nil> Aug 8 00:43:08.872038018 (15h3m30.616094879s) [upload=15496]: Starting upload Aug 8 00:43:08.872131319 (15h3m30.616188187s) [upload=15496]: Done starting upload Aug 8 00:43:08.872241899 (15h3m30.616298752s) [upload=15496]: Uploading segments... Aug 8 00:43:08.872314522 (15h3m30.616371385s) [upload=15496]: Got next segment. Inline: true Aug 8 00:43:08.872320103 (15h3m30.616376964s) [upload=15496]: Waiting for error group managing segments... Aug 8 00:43:08.872387129 (15h3m30.616443979s) [upload=15496]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:43:08.872392943 (15h3m30.616449792s) [upload=15496]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:43:08.872396750 (15h3m30.61645361s) [upload=15496]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:43:09.042725445 (15h3m30.786782296s) [upload=15496]: Done uploading segments... Aug 8 00:43:09.042744523 (15h3m30.786801373s) [upload=15496]: Upload finished. err: <nil> Aug 8 00:43:58.382659382 (15h4m20.126716247s) [upload=15511]: Starting upload Aug 8 00:43:58.382746751 (15h4m20.126803611s) [upload=15511]: Done starting upload Aug 8 00:43:58.382850091 (15h4m20.126906951s) [upload=15511]: Uploading segments... Aug 8 00:43:58.382921355 (15h4m20.126978226s) [upload=15511]: Got next segment. Inline: true Aug 8 00:43:58.382928916 (15h4m20.12698578s) [upload=15511]: Waiting for error group managing segments... Aug 8 00:43:58.383111923 (15h4m20.127168791s) [upload=15511]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:43:58.383121251 (15h4m20.127178114s) [upload=15511]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:43:58.383125081 (15h4m20.127181945s) [upload=15511]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:43:58.536263481 (15h4m20.280320332s) [upload=15511]: Done uploading segments... Aug 8 00:43:58.536281876 (15h4m20.28033873s) [upload=15511]: Upload finished. err: <nil> Aug 8 00:45:01.015818127 (15h5m22.759874977s) [upload=15529]: Starting upload Aug 8 00:45:01.015874109 (15h5m22.759930972s) [upload=15529]: Done starting upload Aug 8 00:45:01.015956182 (15h5m22.76001303s) [upload=15529]: Uploading segments... Aug 8 00:45:01.016043908 (15h5m22.760100776s) [upload=15529]: Got next segment. Inline: true Aug 8 00:45:01.016049919 (15h5m22.760106782s) [upload=15529]: Waiting for error group managing segments... Aug 8 00:45:01.016115031 (15h5m22.760171895s) [upload=15529]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:45:01.016122390 (15h5m22.760179257s) [upload=15529]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:45:01.016125863 (15h5m22.760182727s) [upload=15529]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:45:01.173239782 (15h5m22.91729665s) [upload=15529]: Done uploading segments... Aug 8 00:45:01.173279478 (15h5m22.917336328s) [upload=15529]: Upload finished. err: <nil> Duration: 965.423654ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.585340157s FAIL (2025-08-08 00:45:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.585367423s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 877.691388ms FAIL (2025-08-08 00:45:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 0]: Starting upload Aug 7 23:53:17.153036870 (14h13m38.89709373s) [upload=14650]: Done starting upload Aug 7 23:53:17.153154678 (14h13m38.897211544s) [upload=14650]: Uploading segments... Aug 7 23:53:17.153233089 (14h13m38.897289957s) [upload=14650]: Got next segment. Inline: true Aug 7 23:53:17.153240256 (14h13m38.897297124s) [upload=14650]: Waiting for error group managing segments... Aug 7 23:53:17.153318892 (14h13m38.897375757s) [upload=14650]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:17.153325816 (14h13m38.89738268s) [upload=14650]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:17.153330453 (14h13m38.897387314s) [upload=14650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:16.875757982 (14h14m38.619814845s) [upload=14650]: Done uploading segments... Aug 7 23:54:16.875781849 (14h14m38.619838701s) [upload=14650]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:24.158448119 (14h14m45.90250498s) [upload=14667]: Starting upload Aug 7 23:54:24.158525946 (14h14m45.902582806s) [upload=14667]: Done starting upload Aug 7 23:54:24.158598972 (14h14m45.902655824s) [upload=14667]: Uploading segments... Aug 7 23:54:24.158654148 (14h14m45.902711017s) [upload=14667]: Got next segment. Inline: true Aug 7 23:54:24.158659995 (14h14m45.902716856s) [upload=14667]: Waiting for error group managing segments... Aug 7 23:54:24.158714896 (14h14m45.902771747s) [upload=14667]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:24.158720357 (14h14m45.902777209s) [upload=14667]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:24.158724105 (14h14m45.902780965s) [upload=14667]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:23.888400190 (14h15m45.63245704s) [upload=14667]: Done uploading segments... Aug 7 23:55:23.888419087 (14h15m45.632475938s) [upload=14667]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:24.297146309 (14h15m46.041203174s) [upload=14687]: Starting upload Aug 7 23:55:24.297232144 (14h15m46.04128901s) [upload=14687]: Done starting upload Aug 7 23:55:24.297315772 (14h15m46.041372619s) [upload=14687]: Uploading segments... Aug 7 23:55:24.297396238 (14h15m46.041453151s) [upload=14687]: Got next segment. Inline: true Aug 7 23:55:24.297403729 (14h15m46.04146059s) [upload=14687]: Waiting for error group managing segments... Aug 7 23:55:24.297444116 (14h15m46.041500976s) [upload=14687]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:24.297450971 (14h15m46.041507831s) [upload=14687]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:24.297453883 (14h15m46.041510749s) [upload=14687]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:24.439126204 (14h15m46.183183055s) [upload=14687]: Done uploading segments... Aug 7 23:55:24.439141121 (14h15m46.183197972s) [upload=14687]: Upload finished. err: <nil> Aug 7 23:56:03.068174127 (14h16m24.812230977s) [upload=14695]: Starting upload Aug 7 23:56:03.068258755 (14h16m24.812315617s) [upload=14695]: Done starting upload Aug 7 23:56:03.068368961 (14h16m24.812425812s) [upload=14695]: Uploading segments... Aug 7 23:56:03.068437851 (14h16m24.812494719s) [upload=14695]: Got next segment. Inline: true Aug 7 23:56:03.068445058 (14h16m24.812501927s) [upload=14695]: Waiting for error group managing segments... Aug 7 23:56:03.068516175 (14h16m24.812573049s) [upload=14695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:03.068523275 (14h16m24.812580137s) [upload=14695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:03.068527513 (14h16m24.812584488s) [upload=14695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:03.370378590 (14h16m25.114435452s) [upload=14695]: Done uploading segments... Aug 7 23:56:03.370401742 (14h16m25.114458656s) [upload=14695]: Upload finished. err: <nil> Aug 7 23:56:41.636734874 (14h17m3.380791742s) [upload=14707]: Starting upload Aug 7 23:56:41.636844337 (14h17m3.380901201s) [upload=14707]: Done starting upload Aug 7 23:56:41.636938361 (14h17m3.380995223s) [upload=14707]: Uploading segments... Aug 7 23:56:41.637004566 (14h17m3.381061436s) [upload=14707]: Got next segment. Inline: true Aug 7 23:56:41.637011720 (14h17m3.381068581s) [upload=14707]: Waiting for error group managing segments... Aug 7 23:56:41.637073380 (14h17m3.381130242s) [upload=14707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:41.637079684 (14h17m3.381136546s) [upload=14707]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:41.637083892 (14h17m3.381140753s) [upload=14707]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:41.369074470 (14h18m3.113131318s) [upload=14707]: Done uploading segments... Aug 7 23:57:41.369096313 (14h18m3.11315316s) [upload=14707]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:57:41.785857243 (14h18m3.529914104s) [upload=14726]: Starting upload Aug 7 23:57:41.785973295 (14h18m3.530030155s) [upload=14726]: Done starting upload Aug 7 23:57:41.786053418 (14h18m3.530110268s) [upload=14726]: Uploading segments... Aug 7 23:57:41.786115407 (14h18m3.530172273s) [upload=14726]: Got next segment. Inline: true Aug 7 23:57:41.786121268 (14h18m3.530178127s) [upload=14726]: Waiting for error group managing segments... Aug 7 23:57:41.786162041 (14h18m3.530218901s) [upload=14726]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:57:41.786167247 (14h18m3.530224108s) [upload=14726]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:57:41.786170583 (14h18m3.530227443s) [upload=14726]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:41.518215592 (14h19m3.262272452s) [upload=14726]: Done uploading segments... Aug 7 23:58:41.518237426 (14h19m3.262294275s) [upload=14726]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:58:49.080161447 (14h19m10.824218292s) [upload=14741]: Starting upload Aug 7 23:58:49.080236560 (14h19m10.824293407s) [upload=14741]: Done starting upload Aug 7 23:58:49.080313897 (14h19m10.824370744s) [upload=14741]: Uploading segments... Aug 7 23:58:49.080371825 (14h19m10.824428691s) [upload=14741]: Got next segment. Inline: true Aug 7 23:58:49.080376725 (14h19m10.824433586s) [upload=14741]: Waiting for error group managing segments... Aug 7 23:58:49.080418334 (14h19m10.824475197s) [upload=14741]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:49.080422560 (14h19m10.824479419s) [upload=14741]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:49.080438323 (14h19m10.824495186s) [upload=14741]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:49.220444289 (14h19m10.964501139s) [upload=14741]: Done uploading segments... Aug 7 23:58:49.220462660 (14h19m10.96451951s) [upload=14741]: Upload finished. err: <nil> Aug 7 23:59:26.271206280 (14h19m48.015263132s) [upload=14752]: Starting upload Aug 7 23:59:26.271300842 (14h19m48.015357706s) [upload=14752]: Done starting upload Aug 7 23:59:26.271409542 (14h19m48.015466389s) [upload=14752]: Uploading segments... Aug 7 23:59:26.271471969 (14h19m48.015528839s) [upload=14752]: Got next segment. Inline: true Aug 7 23:59:26.271478422 (14h19m48.015535286s) [upload=14752]: Waiting for error group managing segments... Aug 7 23:59:26.271524304 (14h19m48.015581168s) [upload=14752]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:26.271531803 (14h19m48.01558867s) [upload=14752]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:26.271535740 (14h19m48.015592604s) [upload=14752]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:26.006104454 (14h20m47.750161323s) [upload=14752]: Done uploading segments... Aug 8 00:00:26.006140966 (14h20m47.750197816s) [upload=14752]: Upload finished. err: metaclient: context deadline exceeded Aug 8 00:00:26.408809053 (14h20m48.152865927s) [upload=14766]: Starting upload Aug 8 00:00:26.408886561 (14h20m48.152943434s) [upload=14766]: Done starting upload Aug 8 00:00:26.408975679 (14h20m48.15303253s) [upload=14766]: Uploading segments... Aug 8 00:00:26.409028539 (14h20m48.153085476s) [upload=14766]: Got next segment. Inline: true Aug 8 00:00:26.409034850 (14h20m48.153091712s) [upload=14766]: Waiting for error group managing segments... Aug 8 00:00:26.409085190 (14h20m48.153142042s) [upload=14766]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:00:26.409090930 (14h20m48.153147793s) [upload=14766]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:00:26.409094742 (14h20m48.153151605s) [upload=14766]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:26.549853950 (14h20m48.2939108s) [upload=14766]: Done uploading segments... Aug 8 00:00:26.549878419 (14h20m48.293935265s) [upload=14766]: Upload finished. err: <nil> Aug 8 00:01:08.274172938 (14h21m30.018229799s) [upload=14780]: Starting upload Aug 8 00:01:08.274248443 (14h21m30.018305302s) [upload=14780]: Done starting upload Aug 8 00:01:08.274326739 (14h21m30.018383585s) [upload=14780]: Uploading segments... Aug 8 00:01:08.274380689 (14h21m30.018437549s) [upload=14780]: Got next segment. Inline: true Aug 8 00:01:08.274385704 (14h21m30.018442565s) [upload=14780]: Waiting for error group managing segments... Aug 8 00:01:08.274437335 (14h21m30.018494182s) [upload=14780]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:01:08.274442821 (14h21m30.018499667s) [upload=14780]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:01:08.274445609 (14h21m30.018502454s) [upload=14780]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:08.005343335 (14h22m29.749400197s) [upload=14780]: Done uploading segments... Aug 8 00:02:08.005365121 (14h22m29.749421982s) [upload=14780]: Upload finished. err: metaclient: context deadline exceeded Duration: 877.719668ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.350424217s FAIL (2025-08-08 00:45:28 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.350457345s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 1.004293524s FAIL (2025-08-08 00:45:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 140355 (14h13m13.036197215s) [upload=14641]: Starting upload Aug 7 23:52:51.292216133 (14h13m13.036272985s) [upload=14641]: Done starting upload Aug 7 23:52:51.292307446 (14h13m13.036364307s) [upload=14641]: Uploading segments... Aug 7 23:52:51.292389087 (14h13m13.036445956s) [upload=14641]: Got next segment. Inline: true Aug 7 23:52:51.292394933 (14h13m13.036451793s) [upload=14641]: Waiting for error group managing segments... Aug 7 23:52:51.292456032 (14h13m13.036512883s) [upload=14641]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:51.292461787 (14h13m13.036518647s) [upload=14641]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:51.292467676 (14h13m13.036524537s) [upload=14641]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:51.436915828 (14h13m13.180972678s) [upload=14641]: Done uploading segments... Aug 7 23:52:51.436934458 (14h13m13.18099131s) [upload=14641]: Upload finished. err: <nil> Aug 7 23:53:10.946261153 (14h13m32.690318028s) [upload=14646]: Starting upload Aug 7 23:53:10.946350198 (14h13m32.690407065s) [upload=14646]: Done starting upload Aug 7 23:53:10.946435423 (14h13m32.690492276s) [upload=14646]: Uploading segments... Aug 7 23:53:10.946489255 (14h13m32.690546124s) [upload=14646]: Got next segment. Inline: true Aug 7 23:53:10.946497660 (14h13m32.690554519s) [upload=14646]: Waiting for error group managing segments... Aug 7 23:53:10.946542678 (14h13m32.690599529s) [upload=14646]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:10.946548267 (14h13m32.690605128s) [upload=14646]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:10.946553792 (14h13m32.690610652s) [upload=14646]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:14.554651436 (14h13m36.298708309s) [upload=14646]: Done uploading segments... Aug 7 23:53:14.554675183 (14h13m36.298732045s) [upload=14646]: Upload finished. err: <nil> Aug 7 23:53:53.584502029 (14h14m15.328558897s) [upload=14658]: Starting upload Aug 7 23:53:53.584599757 (14h14m15.328656616s) [upload=14658]: Done starting upload Aug 7 23:53:53.584702364 (14h14m15.328759218s) [upload=14658]: Uploading segments... Aug 7 23:53:53.584798610 (14h14m15.32885548s) [upload=14658]: Got next segment. Inline: true Aug 7 23:53:53.584806594 (14h14m15.328863461s) [upload=14658]: Waiting for error group managing segments... Aug 7 23:53:53.584875193 (14h14m15.328932062s) [upload=14658]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:53.584882324 (14h14m15.328939193s) [upload=14658]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:53.584901657 (14h14m15.328958528s) [upload=14658]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:23.318821682 (14h14m45.062878542s) [upload=14658]: Done uploading segments... Aug 7 23:54:23.318863033 (14h14m45.062919884s) [upload=14658]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:36.775209359 (14h14m58.519266209s) [upload=14671]: Starting upload Aug 7 23:54:36.775292235 (14h14m58.519349112s) [upload=14671]: Done starting upload Aug 7 23:54:36.775412031 (14h14m58.519468883s) [upload=14671]: Uploading segments... Aug 7 23:54:36.775480767 (14h14m58.519537636s) [upload=14671]: Got next segment. Inline: true Aug 7 23:54:36.775487320 (14h14m58.519544181s) [upload=14671]: Waiting for error group managing segments... Aug 7 23:54:36.775539752 (14h14m58.519596612s) [upload=14671]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:36.775545893 (14h14m58.519602753s) [upload=14671]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:36.775549619 (14h14m58.519606479s) [upload=14671]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:36.926276427 (14h14m58.670333277s) [upload=14671]: Done uploading segments... Aug 7 23:54:36.926295206 (14h14m58.670352057s) [upload=14671]: Upload finished. err: <nil> Aug 7 23:54:57.675347843 (14h15m19.419404694s) [upload=14676]: Starting upload Aug 7 23:54:57.675412352 (14h15m19.419469212s) [upload=14676]: Done starting upload Aug 7 23:54:57.675475670 (14h15m19.41953252s) [upload=14676]: Uploading segments... Aug 7 23:54:57.675516336 (14h15m19.419573205s) [upload=14676]: Got next segment. Inline: true Aug 7 23:54:57.675521071 (14h15m19.419577931s) [upload=14676]: Waiting for error group managing segments... Aug 7 23:54:57.675565512 (14h15m19.419622375s) [upload=14676]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:57.675574255 (14h15m19.419631202s) [upload=14676]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:57.675580694 (14h15m19.419637563s) [upload=14676]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:27.407552485 (14h15m49.151609344s) [upload=14676]: Done uploading segments... Aug 7 23:55:27.407592978 (14h15m49.151649829s) [upload=14676]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:56:06.851646232 (14h16m28.595703082s) [upload=14696]: Starting upload Aug 7 23:56:06.851717330 (14h16m28.595774191s) [upload=14696]: Done starting upload Aug 7 23:56:06.851786008 (14h16m28.59584286s) [upload=14696]: Uploading segments... Aug 7 23:56:06.851834232 (14h16m28.595891101s) [upload=14696]: Got next segment. Inline: true Aug 7 23:56:06.851839675 (14h16m28.595896535s) [upload=14696]: Waiting for error group managing segments... Aug 7 23:56:06.851896275 (14h16m28.595953125s) [upload=14696]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:06.851901659 (14h16m28.59595852s) [upload=14696]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:06.851905505 (14h16m28.595962364s) [upload=14696]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:07.006202008 (14h16m28.750258868s) [upload=14696]: Done uploading segments... Aug 7 23:56:07.006222000 (14h16m28.750278861s) [upload=14696]: Upload finished. err: <nil> Aug 7 23:56:59.020763574 (14h17m20.764820421s) [upload=14712]: Starting upload Aug 7 23:56:59.020858955 (14h17m20.764915806s) [upload=14712]: Done starting upload Aug 7 23:56:59.020935444 (14h17m20.764992295s) [upload=14712]: Uploading segments... Aug 7 23:56:59.021007501 (14h17m20.76506437s) [upload=14712]: Got next segment. Inline: true Aug 7 23:56:59.021013631 (14h17m20.765070491s) [upload=14712]: Waiting for error group managing segments... Aug 7 23:56:59.021070471 (14h17m20.76512733s) [upload=14712]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:59.021076966 (14h17m20.765133821s) [upload=14712]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:59.021081305 (14h17m20.76513816s) [upload=14712]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:59.154820708 (14h17m20.898877569s) [upload=14712]: Done uploading segments... Aug 7 23:56:59.154839835 (14h17m20.898896685s) [upload=14712]: Upload finished. err: <nil> Aug 7 23:58:10.963677657 (14h18m32.707734508s) [upload=14728]: Starting upload Aug 7 23:58:10.963770332 (14h18m32.70782718s) [upload=14728]: Done starting upload Aug 7 23:58:10.963851725 (14h18m32.707908586s) [upload=14728]: Uploading segments... Aug 7 23:58:10.963901559 (14h18m32.707958431s) [upload=14728]: Got next segment. Inline: true Aug 7 23:58:10.963907663 (14h18m32.707964525s) [upload=14728]: Waiting for error group managing segments... Aug 7 23:58:10.963960167 (14h18m32.708017032s) [upload=14728]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:10.963966034 (14h18m32.708022896s) [upload=14728]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:10.963969944 (14h18m32.708026806s) [upload=14728]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:11.106069715 (14h18m32.850126571s) [upload=14728]: Done uploading segments... Aug 7 23:58:11.106112099 (14h18m32.850168961s) [upload=14728]: Upload finished. err: <nil> Aug 7 23:59:19.078128070 (14h19m40.822184931s) [upload=14749]: Starting upload Aug 7 23:59:19.078217499 (14h19m40.822274359s) [upload=14749]: Done starting upload Aug 7 23:59:19.078291040 (14h19m40.822347895s) [upload=14749]: Uploading segments... Aug 7 23:59:19.078333863 (14h19m40.822390732s) [upload=14749]: Got next segment. Inline: true Aug 7 23:59:19.078339134 (14h19m40.822395984s) [upload=14749]: Waiting for error group managing segments... Aug 7 23:59:19.078391868 (14h19m40.822448717s) [upload=14749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:19.078397213 (14h19m40.822454064s) [upload=14749]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:19.078401054 (14h19m40.822457905s) [upload=14749]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:59:48.808547234 (14h20m10.552604081s) [upload=14749]: Done uploading segments... Aug 7 23:59:48.808568563 (14h20m10.552625409s) [upload=14749]: Upload finished. err: metaclient: context deadline exceeded Aug 8 00:00:20.896255037 (14h20m42.640311905s) [upload=14765]: Starting upload Aug 8 00:00:20.896381648 (14h20m42.640438517s) [upload=14765]: Done starting upload Aug 8 00:00:20.896532416 (14h20m42.640589282s) [upload=14765]: Uploading segments... Aug 8 00:00:20.896610121 (14h20m42.640666994s) [upload=14765]: Got next segment. Inline: true Aug 8 00:00:20.896618785 (14h20m42.640675649s) [upload=14765]: Waiting for error group managing segments... Aug 8 00:00:20.896682486 (14h20m42.640739601s) [upload=14765]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:00:20.896690306 (14h20m42.640747187s) [upload=14765]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:00:20.896694946 (14h20m42.640751814s) [upload=14765]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:50.607106781 (14h21m12.351163663s) [upload=14765]: Done uploading segments... Aug 8 00:00:50.607134684 (14h21m12.351191558s) [upload=14765]: Upload finished. err: metaclient: context deadline exceeded Duration: 1.004329991s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [X..............X...........................................X] duration: 2.171402302s FAIL (2025-08-08 00:45:18 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Done starting upload Aug 8 00:45:16.266121630 (15h5m38.01017848s) [upload=15533]: Uploading segments... Aug 8 00:45:16.266428624 (15h5m38.010485484s) [upload=15533]: Got next segment. Inline: false Aug 8 00:45:16.266497281 (15h5m38.010554191s) [upload=15533]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:45:16.266512043 (15h5m38.010568914s) [upload=15533]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:45:16.467813786 (15h5m38.211870647s) [seg_pos={0 0}, upload=15533]: Begin upload segment... Aug 8 00:45:16.467883230 (15h5m38.211940081s) [seg_pos={0 0}, upload=15533]: Done begin upload segment. Aug 8 00:45:16.467935054 (15h5m38.211991969s) [noise=true , address=104.36.82.60:4369 , piece_id=WVPSCPUKKSYS4T3JIEEVDTS7WSFYXNGVQA6QNLGO3BKC3EKEYJGQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.467945924 (15h5m38.212002789s) [noise=true , address=104.36.82.54:4192 , piece_id=FDJHGRVMHAFB5JZM4Z77MF3NHRU3T4FNQPUAOXGP5NRCM5Q7RVRA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468440400 (15h5m38.212497272s) [noise=true , address=104.36.82.34:4577 , piece_id=4PTCNUDROG6OXW5DBUEMBAGPEMEMVLIW5OXRP2LPC55J2PTK3PAA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468457544 (15h5m38.21251441s) [noise=true , address=104.36.82.44:4887 , piece_id=FC4PG4NZJHR35DZLGLFZ6EJZYA2MJWLIDPBYHZCNJKJI4OVX5G4Q, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468847577 (15h5m38.212904441s) [noise=true , address=104.36.82.62:4430 , piece_id=JGWQSS5KYYCBOITFMENDLSGK6XSDVOT7TMRGJBZ55KBXX54ENY6A, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.468887696 (15h5m38.21294456s) [noise=true , address=79.127.205.233:4633 , piece_id=F4RDTCFPIWZS4U65QMU5YXWOZSPPDIDMLJ26G3DCN3YV2UMAEFWA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469228336 (15h5m38.2132852s) [noise=true , address=104.36.82.43:4846 , piece_id=73PPCVC3HGFJDIB63URF7XA6PC37VCYLBCOMTSSFNC5UYZR6M5YA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469268209 (15h5m38.213325082s) [noise=true , address=64.49.13.140:4515 , piece_id=AQTWEM35E7E6YUISEOQGTPTPKD6RDOMKXLK267UILGD77273OY4A, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469613163 (15h5m38.21367003s) [noise=true , address=104.36.82.16:4307 , piece_id=B7ZUFMTGVRPCNJSTBCXMQRZW2AGKPXM2LJRDLYZWPXQ4F5JJ75UQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.469702757 (15h5m38.213759621s) [noise=true , address=104.36.82.9:4038 , piece_id=RVXFCLSREWSON62WSTYVISC52HN33VUN2BHJ7UR72MEBMTFKQV6A, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470036233 (15h5m38.214093098s) [noise=true , address=104.36.82.23:4464 , piece_id=4HNT67XCPCNCECLCE36RVAHTJ7VBAADPQ7VMC7IASAUPVVEKIVWQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470129321 (15h5m38.214186184s) [noise=true , address=45.91.116.6:4982 , piece_id=NV3WES7XAJBCNO5EQMWFK6SNUEYE63YKTQC5ZMRF7URP6SX3FXGA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470411978 (15h5m38.214468842s) [noise=true , address=104.36.82.23:4452 , piece_id=B4XGMOK4RCAIOV3VSJXFZ73O52E4WIRUGSPVTCCJ4XUGNJHUQWBA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470482224 (15h5m38.21453909s) [noise=true , address=104.36.82.14:4254 , piece_id=AIB65II65FK6M3EQHX6PVX27N4Q4BQKI5QGVP3PD6D34ZKXVCLZA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470792753 (15h5m38.214849603s) [noise=true , address=104.36.82.19:4405 , piece_id=SA2CPGDA75CYSUDYYZ6V4NKTG5AEIZC7SVYNL2OLBK3E5MSK5FQQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.470877563 (15h5m38.214934486s) [noise=true , address=104.36.82.14:4243 , piece_id=HSR3LCZHIDLJSJ7WISDYOWGPLBFZ73MIXTR7MLFQ2TLUBYLRL4DQ, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.471502176 (15h5m38.215559147s) [noise=true , address=104.36.82.45:4910 , piece_id=OGS6W5AGDOACXIMZ4OH3GNQ43YUU6WOYJHKAHNMHJZBQPJUVVCRA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:16.471850592 (15h5m38.215907452s) [noise=true , address=79.127.201.215:4358 , piece_id=Z3PRSJIAQMKCL2ZU5TA7WZ2IBCVIXB2K5UJBMS3UCI5H6HDNTWBA, seg_pos={0 0}, upload=15533]: Uploading piece... Aug 8 00:45:17.050924903 (15h5m38.794981774s) [upload=15533]: Next returned nil segment Aug 8 00:45:17.051086092 (15h5m38.795142971s) [upload=15533]: Waiting for error group managing segments... Aug 8 00:45:17.225555422 (15h5m38.969612281s) [noise=true , address=104.36.82.62:4430 , piece_id=JGWQSS5KYYCBOITFMENDLSGK6XSDVOT7TMRGJBZ55KBXX54ENY6A, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.232285957 (15h5m38.97634281s) [noise=true , address=45.91.116.6:4982 , piece_id=NV3WES7XAJBCNO5EQMWFK6SNUEYE63YKTQC5ZMRF7URP6SX3FXGA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.239931943 (15h5m38.983988797s) [noise=true , address=104.36.82.45:4910 , piece_id=OGS6W5AGDOACXIMZ4OH3GNQ43YUU6WOYJHKAHNMHJZBQPJUVVCRA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.240442032 (15h5m38.984498886s) [noise=true , address=104.36.82.34:4577 , piece_id=4PTCNUDROG6OXW5DBUEMBAGPEMEMVLIW5OXRP2LPC55J2PTK3PAA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.280384119 (15h5m39.024440979s) [noise=true , address=104.36.82.19:4405 , piece_id=SA2CPGDA75CYSUDYYZ6V4NKTG5AEIZC7SVYNL2OLBK3E5MSK5FQQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.284571751 (15h5m39.028628621s) [noise=true , address=104.36.82.16:4307 , piece_id=B7ZUFMTGVRPCNJSTBCXMQRZW2AGKPXM2LJRDLYZWPXQ4F5JJ75UQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.295563563 (15h5m39.03962043s) [noise=true , address=104.36.82.14:4254 , piece_id=AIB65II65FK6M3EQHX6PVX27N4Q4BQKI5QGVP3PD6D34ZKXVCLZA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.367859627 (15h5m39.11191649s) [noise=true , address=104.36.82.44:4887 , piece_id=FC4PG4NZJHR35DZLGLFZ6EJZYA2MJWLIDPBYHZCNJKJI4OVX5G4Q, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.486413539 (15h5m39.2304704s) [noise=true , address=104.36.82.9:4038 , piece_id=RVXFCLSREWSON62WSTYVISC52HN33VUN2BHJ7UR72MEBMTFKQV6A, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.510192603 (15h5m39.254249451s) [noise=true , address=79.127.201.215:4358 , piece_id=Z3PRSJIAQMKCL2ZU5TA7WZ2IBCVIXB2K5UJBMS3UCI5H6HDNTWBA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.609092139 (15h5m39.353148988s) [noise=true , address=104.36.82.54:4192 , piece_id=FDJHGRVMHAFB5JZM4Z77MF3NHRU3T4FNQPUAOXGP5NRCM5Q7RVRA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.647464269 (15h5m39.391521122s) [noise=true , address=104.36.82.14:4243 , piece_id=HSR3LCZHIDLJSJ7WISDYOWGPLBFZ73MIXTR7MLFQ2TLUBYLRL4DQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.681479215 (15h5m39.425536078s) [noise=true , address=104.36.82.43:4846 , piece_id=73PPCVC3HGFJDIB63URF7XA6PC37VCYLBCOMTSSFNC5UYZR6M5YA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.697970794 (15h5m39.442027648s) [noise=true , address=104.36.82.60:4369 , piece_id=WVPSCPUKKSYS4T3JIEEVDTS7WSFYXNGVQA6QNLGO3BKC3EKEYJGQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: <nil> Aug 8 00:45:17.697997236 (15h5m39.442054151s) [seg_pos={0 0}, upload=15533]: Segment reached optimal threshold of 14 pieces. Aug 8 00:45:17.698203533 (15h5m39.442260383s) [noise=true , address=79.127.205.233:4633 , piece_id=F4RDTCFPIWZS4U65QMU5YXWOZSPPDIDMLJ26G3DCN3YV2UMAEFWA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EEYb4mCySvZ54qzmygCQ7hLyTGYpdNJMyrvPvHJjtfGos9usk): protocol: expected piece hash; context canceled; EOF Aug 8 00:45:17.698290533 (15h5m39.442347383s) [noise=true , address=64.49.13.140:4515 , piece_id=AQTWEM35E7E6YUISEOQGTPTPKD6RDOMKXLK267UILGD77273OY4A, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mY7sRFAeDLmKPqq8BgwnGtkdPTsDRR6soTBshsn4qhP7evARb): protocol: expected piece hash; context canceled; EOF Aug 8 00:45:17.698544485 (15h5m39.442601338s) [noise=true , address=104.36.82.23:4464 , piece_id=4HNT67XCPCNCECLCE36RVAHTJ7VBAADPQ7VMC7IASAUPVVEKIVWQ, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mLbGHntspYmsTVg8433HYSdAEMrXjGnDhmmvX1JmmgcxtdLgk): protocol: write tcp 139.59.144.22:39260->104.36.82.23:4464: use of closed network connection; write tcp 139.59.144.22:39260->104.36.82.23:4464: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39260->104.36.82.23:4464: use of closed network connection Aug 8 00:45:17.698701613 (15h5m39.442758474s) [noise=true , address=104.36.82.23:4452 , piece_id=B4XGMOK4RCAIOV3VSJXFZ73O52E4WIRUGSPVTCCJ4XUGNJHUQWBA, seg_pos={0 0}, upload=15533]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sAh1mLmzbtmc4o2FAZssW8Y7iDDNmNTCXdCS8xrH8PBVQqxG5): protocol: context canceled; context canceled Aug 8 00:45:17.698723379 (15h5m39.44278023s) [seg_pos={0 0}, upload=15533]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 00:45:17.698815435 (15h5m39.442872285s) [upload=15533]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:45:17.698820522 (15h5m39.442877383s) [upload=15533]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:45:17.860271466 (15h5m39.604328326s) [upload=15533]: Done uploading segments... Aug 8 00:45:17.860291088 (15h5m39.604347935s) [upload=15533]: Upload finished. err: <nil> Duration: 2.171431032s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 1.041046866s FAIL (2025-08-08 00:45:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.041078428s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 2.39983886s FAIL (2025-08-08 00:45:11 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.399957502s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 3.567452643s FAIL (2025-08-08 00:45:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.567541568s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 648.289526ms FAIL (2025-08-08 00:45:01 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== stItem_ObjectCommit Aug 8 00:37:35.928930883 (14h57m57.672987755s) [upload=15397]: Done uploading segments... Aug 8 00:37:35.928967703 (14h57m57.673024563s) [upload=15397]: Upload finished. err: <nil> Aug 8 00:37:56.695554589 (14h58m18.439611449s) [upload=15402]: Starting upload Aug 8 00:37:56.695609369 (14h58m18.43966623s) [upload=15402]: Done starting upload Aug 8 00:37:56.695671726 (14h58m18.439728574s) [upload=15402]: Uploading segments... Aug 8 00:37:56.695713147 (14h58m18.439770016s) [upload=15402]: Got next segment. Inline: true Aug 8 00:37:56.695718707 (14h58m18.439775558s) [upload=15402]: Waiting for error group managing segments... Aug 8 00:37:56.695760492 (14h58m18.439817342s) [upload=15402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:37:56.695765786 (14h58m18.439822646s) [upload=15402]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:37:56.695769297 (14h58m18.439826158s) [upload=15402]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:37:56.839484031 (14h58m18.583540885s) [upload=15402]: Done uploading segments... Aug 8 00:37:56.839508115 (14h58m18.583564984s) [upload=15402]: Upload finished. err: <nil> Aug 8 00:39:08.555730292 (14h59m30.299787146s) [upload=15423]: Starting upload Aug 8 00:39:08.555824312 (14h59m30.299881164s) [upload=15423]: Done starting upload Aug 8 00:39:08.555930281 (14h59m30.299987134s) [upload=15423]: Uploading segments... Aug 8 00:39:08.555989277 (14h59m30.300046148s) [upload=15423]: Got next segment. Inline: true Aug 8 00:39:08.555995450 (14h59m30.300052299s) [upload=15423]: Waiting for error group managing segments... Aug 8 00:39:08.556058440 (14h59m30.30011529s) [upload=15423]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:39:08.556064571 (14h59m30.300121422s) [upload=15423]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:39:08.556069271 (14h59m30.300126124s) [upload=15423]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:39:08.697938297 (14h59m30.441995159s) [upload=15423]: Done uploading segments... Aug 8 00:39:08.697959132 (14h59m30.442015982s) [upload=15423]: Upload finished. err: <nil> Aug 8 00:40:19.170886308 (15h0m40.914943159s) [upload=15444]: Starting upload Aug 8 00:40:19.170949280 (15h0m40.91500614s) [upload=15444]: Done starting upload Aug 8 00:40:19.171171058 (15h0m40.915227903s) [upload=15444]: Uploading segments... Aug 8 00:40:19.171300409 (15h0m40.915357281s) [upload=15444]: Got next segment. Inline: true Aug 8 00:40:19.171309541 (15h0m40.915366409s) [upload=15444]: Waiting for error group managing segments... Aug 8 00:40:19.171385631 (15h0m40.915442482s) [upload=15444]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:40:19.171391854 (15h0m40.915448714s) [upload=15444]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:40:19.171395665 (15h0m40.915452525s) [upload=15444]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:40:19.331296850 (15h0m41.075353715s) [upload=15444]: Done uploading segments... Aug 8 00:40:19.331319872 (15h0m41.075376736s) [upload=15444]: Upload finished. err: <nil> Aug 8 00:41:14.342295084 (15h1m36.086351944s) [upload=15459]: Starting upload Aug 8 00:41:14.342370744 (15h1m36.086427596s) [upload=15459]: Done starting upload Aug 8 00:41:14.342453152 (15h1m36.086510013s) [upload=15459]: Uploading segments... Aug 8 00:41:14.342497220 (15h1m36.086554158s) [upload=15459]: Got next segment. Inline: true Aug 8 00:41:14.342504023 (15h1m36.08656089s) [upload=15459]: Waiting for error group managing segments... Aug 8 00:41:14.342588479 (15h1m36.086645343s) [upload=15459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:41:14.342612029 (15h1m36.086668893s) [upload=15459]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:41:14.342616257 (15h1m36.08667312s) [upload=15459]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:41:14.552238378 (15h1m36.296295229s) [upload=15459]: Done uploading segments... Aug 8 00:41:14.552258831 (15h1m36.296315682s) [upload=15459]: Upload finished. err: <nil> Aug 8 00:41:33.520869236 (15h1m55.264926087s) [upload=15470]: Starting upload Aug 8 00:41:33.520992007 (15h1m55.265048867s) [upload=15470]: Done starting upload Aug 8 00:41:33.521087418 (15h1m55.265144282s) [upload=15470]: Uploading segments... Aug 8 00:41:33.521174747 (15h1m55.265231612s) [upload=15470]: Got next segment. Inline: true Aug 8 00:41:33.521182648 (15h1m55.265239509s) [upload=15470]: Waiting for error group managing segments... Aug 8 00:41:33.521241390 (15h1m55.265298239s) [upload=15470]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:41:33.521247030 (15h1m55.26530389s) [upload=15470]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:41:33.521250440 (15h1m55.265307301s) [upload=15470]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:41:33.674374631 (15h1m55.418431478s) [upload=15470]: Done uploading segments... Aug 8 00:41:33.674392958 (15h1m55.418449804s) [upload=15470]: Upload finished. err: <nil> Aug 8 00:42:13.521243964 (15h2m35.26530081s) [upload=15480]: Starting upload Aug 8 00:42:13.521322202 (15h2m35.265379069s) [upload=15480]: Done starting upload Aug 8 00:42:13.521399949 (15h2m35.265456796s) [upload=15480]: Uploading segments... Aug 8 00:42:13.521486038 (15h2m35.265542908s) [upload=15480]: Got next segment. Inline: true Aug 8 00:42:13.521492566 (15h2m35.265549425s) [upload=15480]: Waiting for error group managing segments... Aug 8 00:42:13.521546734 (15h2m35.26560358s) [upload=15480]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:42:13.521552405 (15h2m35.265609264s) [upload=15480]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:42:13.521555275 (15h2m35.265612134s) [upload=15480]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:42:13.690373889 (15h2m35.434430739s) [upload=15480]: Done uploading segments... Aug 8 00:42:13.690405230 (15h2m35.434462081s) [upload=15480]: Upload finished. err: <nil> Aug 8 00:42:43.968444265 (15h3m5.712501112s) [upload=15489]: Starting upload Aug 8 00:42:43.968495052 (15h3m5.712551899s) [upload=15489]: Done starting upload Aug 8 00:42:43.968563760 (15h3m5.712620606s) [upload=15489]: Uploading segments... Aug 8 00:42:43.968604135 (15h3m5.712661001s) [upload=15489]: Got next segment. Inline: true Aug 8 00:42:43.968608788 (15h3m5.712665649s) [upload=15489]: Waiting for error group managing segments... Aug 8 00:42:43.968647123 (15h3m5.712703969s) [upload=15489]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:42:43.968652138 (15h3m5.712708998s) [upload=15489]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:42:43.968654778 (15h3m5.712711638s) [upload=15489]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:42:44.310945173 (15h3m6.055002023s) [upload=15489]: Done uploading segments... Aug 8 00:42:44.311090602 (15h3m6.055147463s) [upload=15489]: Upload finished. err: <nil> Aug 8 00:43:08.872038018 (15h3m30.616094879s) [upload=15496]: Starting upload Aug 8 00:43:08.872131319 (15h3m30.616188187s) [upload=15496]: Done starting upload Aug 8 00:43:08.872241899 (15h3m30.616298752s) [upload=15496]: Uploading segments... Aug 8 00:43:08.872314522 (15h3m30.616371385s) [upload=15496]: Got next segment. Inline: true Aug 8 00:43:08.872320103 (15h3m30.616376964s) [upload=15496]: Waiting for error group managing segments... Aug 8 00:43:08.872387129 (15h3m30.616443979s) [upload=15496]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:43:08.872392943 (15h3m30.616449792s) [upload=15496]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:43:08.872396750 (15h3m30.61645361s) [upload=15496]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:43:09.042725445 (15h3m30.786782296s) [upload=15496]: Done uploading segments... Aug 8 00:43:09.042744523 (15h3m30.786801373s) [upload=15496]: Upload finished. err: <nil> Aug 8 00:43:58.382659382 (15h4m20.126716247s) [upload=15511]: Starting upload Aug 8 00:43:58.382746751 (15h4m20.126803611s) [upload=15511]: Done starting upload Aug 8 00:43:58.382850091 (15h4m20.126906951s) [upload=15511]: Uploading segments... Aug 8 00:43:58.382921355 (15h4m20.126978226s) [upload=15511]: Got next segment. Inline: true Aug 8 00:43:58.382928916 (15h4m20.12698578s) [upload=15511]: Waiting for error group managing segments... Aug 8 00:43:58.383111923 (15h4m20.127168791s) [upload=15511]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:43:58.383121251 (15h4m20.127178114s) [upload=15511]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:43:58.383125081 (15h4m20.127181945s) [upload=15511]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:43:58.536263481 (15h4m20.280320332s) [upload=15511]: Done uploading segments... Aug 8 00:43:58.536281876 (15h4m20.28033873s) [upload=15511]: Upload finished. err: <nil> Aug 8 00:45:01.015818127 (15h5m22.759874977s) [upload=15529]: Starting upload Aug 8 00:45:01.015874109 (15h5m22.759930972s) [upload=15529]: Done starting upload Aug 8 00:45:01.015956182 (15h5m22.76001303s) [upload=15529]: Uploading segments... Aug 8 00:45:01.016043908 (15h5m22.760100776s) [upload=15529]: Got next segment. Inline: true Aug 8 00:45:01.016049919 (15h5m22.760106782s) [upload=15529]: Waiting for error group managing segments... Aug 8 00:45:01.016115031 (15h5m22.760171895s) [upload=15529]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:45:01.016122390 (15h5m22.760179257s) [upload=15529]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:45:01.016125863 (15h5m22.760182727s) [upload=15529]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:45:01.173239782 (15h5m22.91729665s) [upload=15529]: Done uploading segments... Aug 8 00:45:01.173279478 (15h5m22.917336328s) [upload=15529]: Upload finished. err: <nil> Duration: 648.336771ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 856.49202ms FAIL (2025-08-08 00:44:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 140355 (14h13m13.036197215s) [upload=14641]: Starting upload Aug 7 23:52:51.292216133 (14h13m13.036272985s) [upload=14641]: Done starting upload Aug 7 23:52:51.292307446 (14h13m13.036364307s) [upload=14641]: Uploading segments... Aug 7 23:52:51.292389087 (14h13m13.036445956s) [upload=14641]: Got next segment. Inline: true Aug 7 23:52:51.292394933 (14h13m13.036451793s) [upload=14641]: Waiting for error group managing segments... Aug 7 23:52:51.292456032 (14h13m13.036512883s) [upload=14641]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:51.292461787 (14h13m13.036518647s) [upload=14641]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:51.292467676 (14h13m13.036524537s) [upload=14641]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:51.436915828 (14h13m13.180972678s) [upload=14641]: Done uploading segments... Aug 7 23:52:51.436934458 (14h13m13.18099131s) [upload=14641]: Upload finished. err: <nil> Aug 7 23:53:10.946261153 (14h13m32.690318028s) [upload=14646]: Starting upload Aug 7 23:53:10.946350198 (14h13m32.690407065s) [upload=14646]: Done starting upload Aug 7 23:53:10.946435423 (14h13m32.690492276s) [upload=14646]: Uploading segments... Aug 7 23:53:10.946489255 (14h13m32.690546124s) [upload=14646]: Got next segment. Inline: true Aug 7 23:53:10.946497660 (14h13m32.690554519s) [upload=14646]: Waiting for error group managing segments... Aug 7 23:53:10.946542678 (14h13m32.690599529s) [upload=14646]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:10.946548267 (14h13m32.690605128s) [upload=14646]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:10.946553792 (14h13m32.690610652s) [upload=14646]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:14.554651436 (14h13m36.298708309s) [upload=14646]: Done uploading segments... Aug 7 23:53:14.554675183 (14h13m36.298732045s) [upload=14646]: Upload finished. err: <nil> Aug 7 23:53:53.584502029 (14h14m15.328558897s) [upload=14658]: Starting upload Aug 7 23:53:53.584599757 (14h14m15.328656616s) [upload=14658]: Done starting upload Aug 7 23:53:53.584702364 (14h14m15.328759218s) [upload=14658]: Uploading segments... Aug 7 23:53:53.584798610 (14h14m15.32885548s) [upload=14658]: Got next segment. Inline: true Aug 7 23:53:53.584806594 (14h14m15.328863461s) [upload=14658]: Waiting for error group managing segments... Aug 7 23:53:53.584875193 (14h14m15.328932062s) [upload=14658]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:53.584882324 (14h14m15.328939193s) [upload=14658]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:53.584901657 (14h14m15.328958528s) [upload=14658]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:23.318821682 (14h14m45.062878542s) [upload=14658]: Done uploading segments... Aug 7 23:54:23.318863033 (14h14m45.062919884s) [upload=14658]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:36.775209359 (14h14m58.519266209s) [upload=14671]: Starting upload Aug 7 23:54:36.775292235 (14h14m58.519349112s) [upload=14671]: Done starting upload Aug 7 23:54:36.775412031 (14h14m58.519468883s) [upload=14671]: Uploading segments... Aug 7 23:54:36.775480767 (14h14m58.519537636s) [upload=14671]: Got next segment. Inline: true Aug 7 23:54:36.775487320 (14h14m58.519544181s) [upload=14671]: Waiting for error group managing segments... Aug 7 23:54:36.775539752 (14h14m58.519596612s) [upload=14671]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:36.775545893 (14h14m58.519602753s) [upload=14671]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:36.775549619 (14h14m58.519606479s) [upload=14671]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:36.926276427 (14h14m58.670333277s) [upload=14671]: Done uploading segments... Aug 7 23:54:36.926295206 (14h14m58.670352057s) [upload=14671]: Upload finished. err: <nil> Aug 7 23:54:57.675347843 (14h15m19.419404694s) [upload=14676]: Starting upload Aug 7 23:54:57.675412352 (14h15m19.419469212s) [upload=14676]: Done starting upload Aug 7 23:54:57.675475670 (14h15m19.41953252s) [upload=14676]: Uploading segments... Aug 7 23:54:57.675516336 (14h15m19.419573205s) [upload=14676]: Got next segment. Inline: true Aug 7 23:54:57.675521071 (14h15m19.419577931s) [upload=14676]: Waiting for error group managing segments... Aug 7 23:54:57.675565512 (14h15m19.419622375s) [upload=14676]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:57.675574255 (14h15m19.419631202s) [upload=14676]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:57.675580694 (14h15m19.419637563s) [upload=14676]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:27.407552485 (14h15m49.151609344s) [upload=14676]: Done uploading segments... Aug 7 23:55:27.407592978 (14h15m49.151649829s) [upload=14676]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:56:06.851646232 (14h16m28.595703082s) [upload=14696]: Starting upload Aug 7 23:56:06.851717330 (14h16m28.595774191s) [upload=14696]: Done starting upload Aug 7 23:56:06.851786008 (14h16m28.59584286s) [upload=14696]: Uploading segments... Aug 7 23:56:06.851834232 (14h16m28.595891101s) [upload=14696]: Got next segment. Inline: true Aug 7 23:56:06.851839675 (14h16m28.595896535s) [upload=14696]: Waiting for error group managing segments... Aug 7 23:56:06.851896275 (14h16m28.595953125s) [upload=14696]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:06.851901659 (14h16m28.59595852s) [upload=14696]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:06.851905505 (14h16m28.595962364s) [upload=14696]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:07.006202008 (14h16m28.750258868s) [upload=14696]: Done uploading segments... Aug 7 23:56:07.006222000 (14h16m28.750278861s) [upload=14696]: Upload finished. err: <nil> Aug 7 23:56:59.020763574 (14h17m20.764820421s) [upload=14712]: Starting upload Aug 7 23:56:59.020858955 (14h17m20.764915806s) [upload=14712]: Done starting upload Aug 7 23:56:59.020935444 (14h17m20.764992295s) [upload=14712]: Uploading segments... Aug 7 23:56:59.021007501 (14h17m20.76506437s) [upload=14712]: Got next segment. Inline: true Aug 7 23:56:59.021013631 (14h17m20.765070491s) [upload=14712]: Waiting for error group managing segments... Aug 7 23:56:59.021070471 (14h17m20.76512733s) [upload=14712]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:59.021076966 (14h17m20.765133821s) [upload=14712]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:59.021081305 (14h17m20.76513816s) [upload=14712]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:59.154820708 (14h17m20.898877569s) [upload=14712]: Done uploading segments... Aug 7 23:56:59.154839835 (14h17m20.898896685s) [upload=14712]: Upload finished. err: <nil> Aug 7 23:58:10.963677657 (14h18m32.707734508s) [upload=14728]: Starting upload Aug 7 23:58:10.963770332 (14h18m32.70782718s) [upload=14728]: Done starting upload Aug 7 23:58:10.963851725 (14h18m32.707908586s) [upload=14728]: Uploading segments... Aug 7 23:58:10.963901559 (14h18m32.707958431s) [upload=14728]: Got next segment. Inline: true Aug 7 23:58:10.963907663 (14h18m32.707964525s) [upload=14728]: Waiting for error group managing segments... Aug 7 23:58:10.963960167 (14h18m32.708017032s) [upload=14728]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:10.963966034 (14h18m32.708022896s) [upload=14728]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:10.963969944 (14h18m32.708026806s) [upload=14728]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:11.106069715 (14h18m32.850126571s) [upload=14728]: Done uploading segments... Aug 7 23:58:11.106112099 (14h18m32.850168961s) [upload=14728]: Upload finished. err: <nil> Aug 7 23:59:19.078128070 (14h19m40.822184931s) [upload=14749]: Starting upload Aug 7 23:59:19.078217499 (14h19m40.822274359s) [upload=14749]: Done starting upload Aug 7 23:59:19.078291040 (14h19m40.822347895s) [upload=14749]: Uploading segments... Aug 7 23:59:19.078333863 (14h19m40.822390732s) [upload=14749]: Got next segment. Inline: true Aug 7 23:59:19.078339134 (14h19m40.822395984s) [upload=14749]: Waiting for error group managing segments... Aug 7 23:59:19.078391868 (14h19m40.822448717s) [upload=14749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:19.078397213 (14h19m40.822454064s) [upload=14749]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:19.078401054 (14h19m40.822457905s) [upload=14749]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:59:48.808547234 (14h20m10.552604081s) [upload=14749]: Done uploading segments... Aug 7 23:59:48.808568563 (14h20m10.552625409s) [upload=14749]: Upload finished. err: metaclient: context deadline exceeded Aug 8 00:00:20.896255037 (14h20m42.640311905s) [upload=14765]: Starting upload Aug 8 00:00:20.896381648 (14h20m42.640438517s) [upload=14765]: Done starting upload Aug 8 00:00:20.896532416 (14h20m42.640589282s) [upload=14765]: Uploading segments... Aug 8 00:00:20.896610121 (14h20m42.640666994s) [upload=14765]: Got next segment. Inline: true Aug 8 00:00:20.896618785 (14h20m42.640675649s) [upload=14765]: Waiting for error group managing segments... Aug 8 00:00:20.896682486 (14h20m42.640739601s) [upload=14765]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:00:20.896690306 (14h20m42.640747187s) [upload=14765]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:00:20.896694946 (14h20m42.640751814s) [upload=14765]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:50.607106781 (14h21m12.351163663s) [upload=14765]: Done uploading segments... Aug 8 00:00:50.607134684 (14h21m12.351191558s) [upload=14765]: Upload finished. err: metaclient: context deadline exceeded Duration: 856.541863ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 2.619133325s FAIL (2025-08-08 00:44:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.619174947s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.335260382s FAIL (2025-08-08 00:44:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== 6]: Uploading piece... Aug 7 23:57:10.893816491 (14h17m32.637873354s) [noise=true , address=45.91.116.12:4225 , piece_id=NTUY2VEJFWSFOAGWUPTJYNYNWL3XMK4C3FJUE6MSTMTS2TZJ7VUA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.894285524 (14h17m32.638342391s) [noise=true , address=104.36.82.16:4294 , piece_id=SIMVCA63FAX5EQZJO5TBOSGPVYLFOAICTABKMXS5SCQGNOUPFBIA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.895348932 (14h17m32.639405787s) [noise=true , address=77.90.31.14:4661 , piece_id=WFDXKM726LBL4MH4CJ4AQIUSXTQCQ3BJV34QBUGWPHNFFPZYGZKA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.895932061 (14h17m32.639988921s) [noise=true , address=104.36.82.10:4078 , piece_id=3DPLCXZLQWMZSYSSO3JPLZFP2VXJ5NKYO4FDVFYOWJLT43C7XSTQ, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.896545320 (14h17m32.640602184s) [noise=true , address=77.90.31.10:4518 , piece_id=LU6UTDDO6HUIGXV2QXCL45CIBFQMOQGS7CTQHBRKCBE2JJKLI6FA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.897117472 (14h17m32.641174341s) [noise=true , address=77.90.31.21:4877 , piece_id=SF7XDMNH7KGS26UE6MKUK3VLFZ6IOP6WZKMVO4H6SIAULM623GLA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.897621325 (14h17m32.641678192s) [noise=true , address=77.90.31.12:4589 , piece_id=5I5BMAQARX67CB2RRMCXG2U7KELJ6HSHRT3573EH3ZX6NJ257R4A, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.898101338 (14h17m32.642158204s) [noise=true , address=77.90.31.21:4878 , piece_id=XAXEYDOX54GUKMI22UKTA6BSTSY5WYATTITTGSRQ3BBYDGDCO5DQ, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.898693607 (14h17m32.642750474s) [noise=true , address=45.91.116.6:4975 , piece_id=VDEP57UE6G4CBU7DOYMMOIRXJUTTWPJKDJDGCGUOHBGVPAYCM6PA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.900512475 (14h17m32.644569335s) [noise=true , address=45.91.116.12:4227 , piece_id=J2FM3FAWY5HYXX3VFC6CSHUSWDQDNKUUB6NL6MWSF2ZG3BZFWBEA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.902337804 (14h17m32.646394669s) [noise=true , address=77.90.31.13:4626 , piece_id=PR6GY2Y7HDUORSLYZXU4IVLVA6RIMBFSRFJW4OTLXUZM7MJZ4KOQ, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.904215667 (14h17m32.648272537s) [noise=true , address=77.90.31.19:4841 , piece_id=Q3XFTIVGOSRQARSFONAKX26VAFTJF7THPIQV6USKI7N2LN3JBTLQ, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.906034481 (14h17m32.650091365s) [noise=true , address=104.36.82.5:4839 , piece_id=AV3ZBQ4W44IZEDAPHJGHKK5D5VRNU5XGOQGDWEARNGIKWRQAGYBQ, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.907857426 (14h17m32.651914295s) [noise=true , address=77.90.31.15:4698 , piece_id=WBU562MADMX3KSCRCD2H5R7MQN5IPFBLKYSKU47WMGYYBKZGPYJA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.908666188 (14h17m32.652723069s) [noise=true , address=104.36.82.14:4222 , piece_id=JFWGS6VCDTFOQYG32ZFTWNYLFGVVQPXD7K6TZXC4FR2SGEOUS7FA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:11.510238622 (14h17m33.254295484s) [upload=14716]: Next returned nil segment Aug 7 23:57:11.510261464 (14h17m33.254318324s) [upload=14716]: Waiting for error group managing segments... Aug 7 23:57:11.657094388 (14h17m33.401151236s) [noise=true , address=77.90.31.10:4518 , piece_id=LU6UTDDO6HUIGXV2QXCL45CIBFQMOQGS7CTQHBRKCBE2JJKLI6FA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.674408374 (14h17m33.418465234s) [noise=true , address=77.90.31.15:4698 , piece_id=WBU562MADMX3KSCRCD2H5R7MQN5IPFBLKYSKU47WMGYYBKZGPYJA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.681203586 (14h17m33.425260449s) [noise=true , address=77.90.31.12:4589 , piece_id=5I5BMAQARX67CB2RRMCXG2U7KELJ6HSHRT3573EH3ZX6NJ257R4A, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.682780850 (14h17m33.4268377s) [noise=true , address=77.90.31.21:4877 , piece_id=SF7XDMNH7KGS26UE6MKUK3VLFZ6IOP6WZKMVO4H6SIAULM623GLA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.684042315 (14h17m33.428099176s) [noise=true , address=104.36.82.16:4294 , piece_id=SIMVCA63FAX5EQZJO5TBOSGPVYLFOAICTABKMXS5SCQGNOUPFBIA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.719658547 (14h17m33.463715401s) [noise=true , address=45.91.116.12:4225 , piece_id=NTUY2VEJFWSFOAGWUPTJYNYNWL3XMK4C3FJUE6MSTMTS2TZJ7VUA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.730348391 (14h17m33.474405242s) [noise=true , address=77.90.31.21:4878 , piece_id=XAXEYDOX54GUKMI22UKTA6BSTSY5WYATTITTGSRQ3BBYDGDCO5DQ, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.748264399 (14h17m33.492321249s) [noise=true , address=77.90.31.19:4841 , piece_id=Q3XFTIVGOSRQARSFONAKX26VAFTJF7THPIQV6USKI7N2LN3JBTLQ, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.769764028 (14h17m33.513820878s) [noise=true , address=45.91.116.6:4975 , piece_id=VDEP57UE6G4CBU7DOYMMOIRXJUTTWPJKDJDGCGUOHBGVPAYCM6PA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.774177874 (14h17m33.518234733s) [noise=true , address=77.90.31.14:4661 , piece_id=WFDXKM726LBL4MH4CJ4AQIUSXTQCQ3BJV34QBUGWPHNFFPZYGZKA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.789864472 (14h17m33.533921326s) [noise=true , address=45.91.116.13:4153 , piece_id=WBFFQT5I6ZR5ZKUJRFULNOJZ455KIJMA5EEATDDOCVFUKRD67NFA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.799519597 (14h17m33.543576447s) [noise=true , address=45.91.116.12:4227 , piece_id=J2FM3FAWY5HYXX3VFC6CSHUSWDQDNKUUB6NL6MWSF2ZG3BZFWBEA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.800753821 (14h17m33.544810667s) [noise=true , address=77.90.31.13:4626 , piece_id=PR6GY2Y7HDUORSLYZXU4IVLVA6RIMBFSRFJW4OTLXUZM7MJZ4KOQ, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.800785983 (14h17m33.544842842s) [seg_pos={0 0}, upload=14716]: Segment reached optimal threshold of 13 pieces. Aug 7 23:57:11.801026052 (14h17m33.545082898s) [noise=true , address=104.36.82.14:4222 , piece_id=JFWGS6VCDTFOQYG32ZFTWNYLFGVVQPXD7K6TZXC4FR2SGEOUS7FA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1): protocol: expected piece hash; context canceled; EOF Aug 7 23:57:11.801098020 (14h17m33.545154867s) [noise=true , address=104.36.82.10:4078 , piece_id=3DPLCXZLQWMZSYSSO3JPLZFP2VXJ5NKYO4FDVFYOWJLT43C7XSTQ, seg_pos={0 0}, upload=14716]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Aug 7 23:57:11.801174944 (14h17m33.54523179s) [noise=true , address=104.36.82.5:4839 , piece_id=AV3ZBQ4W44IZEDAPHJGHKK5D5VRNU5XGOQGDWEARNGIKWRQAGYBQ, seg_pos={0 0}, upload=14716]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 139.59.144.22:48474->104.36.82.5:4839: use of closed network connection; write tcp 139.59.144.22:48474->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48474->104.36.82.5:4839: use of closed network connection Aug 7 23:57:11.801184926 (14h17m33.545241773s) [seg_pos={0 0}, upload=14716]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:57:11.801257096 (14h17m33.545313956s) [upload=14716]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:57:11.801261009 (14h17m33.545317869s) [upload=14716]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:11.942784613 (14h17m33.686841513s) [upload=14716]: Done uploading segments... Aug 7 23:57:11.942815424 (14h17m33.686872279s) [upload=14716]: Upload finished. err: <nil> Aug 7 23:58:20.561350310 (14h18m42.305407184s) [upload=14732]: Starting upload Aug 7 23:58:20.561483402 (14h18m42.30554027s) [upload=14732]: Done starting upload Aug 7 23:58:20.561650297 (14h18m42.305707159s) [upload=14732]: Uploading segments... Aug 7 23:58:20.562196341 (14h18m42.306253212s) [upload=14732]: Got next segment. Inline: false Aug 7 23:58:20.562334166 (14h18m42.306391031s) [upload=14732]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:20.562343255 (14h18m42.306400125s) [upload=14732]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:58:50.281195862 (14h19m12.025252712s) [upload=14732]: Next segment err: context deadline exceeded Aug 7 23:58:50.281383705 (14h19m12.025440555s) [upload=14732]: Done uploading segments... Aug 7 23:58:50.281405600 (14h19m12.02546245s) [upload=14732]: Upload finished. err: context deadline exceeded Aug 7 23:58:50.716777496 (14h19m12.460834346s) [upload=14742]: Starting upload Aug 7 23:58:50.716857905 (14h19m12.460914751s) [upload=14742]: Done starting upload Aug 7 23:58:50.716999972 (14h19m12.461056825s) [upload=14742]: Uploading segments... Aug 7 23:58:50.717186124 (14h19m12.461242976s) [upload=14742]: Got next segment. Inline: false Aug 7 23:58:50.717245601 (14h19m12.461302466s) [upload=14742]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:50.717252185 (14h19m12.461309053s) [upload=14742]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:59:20.436529435 (14h19m42.180586305s) [upload=14742]: Next segment err: context deadline exceeded Aug 7 23:59:20.436734882 (14h19m42.180791733s) [upload=14742]: Done uploading segments... Aug 7 23:59:20.436756200 (14h19m42.180813054s) [upload=14742]: Upload finished. err: context deadline exceeded Duration: 2.335287822s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 2.42284541s FAIL (2025-08-08 00:44:34 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.422875493s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 2 out of 60 is failed (threshold=3) [X.X.........................................................] duration: 946.002781ms FAIL (2025-08-08 00:44:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 0]: Starting upload Aug 7 23:53:17.153036870 (14h13m38.89709373s) [upload=14650]: Done starting upload Aug 7 23:53:17.153154678 (14h13m38.897211544s) [upload=14650]: Uploading segments... Aug 7 23:53:17.153233089 (14h13m38.897289957s) [upload=14650]: Got next segment. Inline: true Aug 7 23:53:17.153240256 (14h13m38.897297124s) [upload=14650]: Waiting for error group managing segments... Aug 7 23:53:17.153318892 (14h13m38.897375757s) [upload=14650]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:17.153325816 (14h13m38.89738268s) [upload=14650]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:17.153330453 (14h13m38.897387314s) [upload=14650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:16.875757982 (14h14m38.619814845s) [upload=14650]: Done uploading segments... Aug 7 23:54:16.875781849 (14h14m38.619838701s) [upload=14650]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:24.158448119 (14h14m45.90250498s) [upload=14667]: Starting upload Aug 7 23:54:24.158525946 (14h14m45.902582806s) [upload=14667]: Done starting upload Aug 7 23:54:24.158598972 (14h14m45.902655824s) [upload=14667]: Uploading segments... Aug 7 23:54:24.158654148 (14h14m45.902711017s) [upload=14667]: Got next segment. Inline: true Aug 7 23:54:24.158659995 (14h14m45.902716856s) [upload=14667]: Waiting for error group managing segments... Aug 7 23:54:24.158714896 (14h14m45.902771747s) [upload=14667]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:24.158720357 (14h14m45.902777209s) [upload=14667]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:24.158724105 (14h14m45.902780965s) [upload=14667]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:23.888400190 (14h15m45.63245704s) [upload=14667]: Done uploading segments... Aug 7 23:55:23.888419087 (14h15m45.632475938s) [upload=14667]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:24.297146309 (14h15m46.041203174s) [upload=14687]: Starting upload Aug 7 23:55:24.297232144 (14h15m46.04128901s) [upload=14687]: Done starting upload Aug 7 23:55:24.297315772 (14h15m46.041372619s) [upload=14687]: Uploading segments... Aug 7 23:55:24.297396238 (14h15m46.041453151s) [upload=14687]: Got next segment. Inline: true Aug 7 23:55:24.297403729 (14h15m46.04146059s) [upload=14687]: Waiting for error group managing segments... Aug 7 23:55:24.297444116 (14h15m46.041500976s) [upload=14687]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:24.297450971 (14h15m46.041507831s) [upload=14687]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:24.297453883 (14h15m46.041510749s) [upload=14687]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:24.439126204 (14h15m46.183183055s) [upload=14687]: Done uploading segments... Aug 7 23:55:24.439141121 (14h15m46.183197972s) [upload=14687]: Upload finished. err: <nil> Aug 7 23:56:03.068174127 (14h16m24.812230977s) [upload=14695]: Starting upload Aug 7 23:56:03.068258755 (14h16m24.812315617s) [upload=14695]: Done starting upload Aug 7 23:56:03.068368961 (14h16m24.812425812s) [upload=14695]: Uploading segments... Aug 7 23:56:03.068437851 (14h16m24.812494719s) [upload=14695]: Got next segment. Inline: true Aug 7 23:56:03.068445058 (14h16m24.812501927s) [upload=14695]: Waiting for error group managing segments... Aug 7 23:56:03.068516175 (14h16m24.812573049s) [upload=14695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:03.068523275 (14h16m24.812580137s) [upload=14695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:03.068527513 (14h16m24.812584488s) [upload=14695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:03.370378590 (14h16m25.114435452s) [upload=14695]: Done uploading segments... Aug 7 23:56:03.370401742 (14h16m25.114458656s) [upload=14695]: Upload finished. err: <nil> Aug 7 23:56:41.636734874 (14h17m3.380791742s) [upload=14707]: Starting upload Aug 7 23:56:41.636844337 (14h17m3.380901201s) [upload=14707]: Done starting upload Aug 7 23:56:41.636938361 (14h17m3.380995223s) [upload=14707]: Uploading segments... Aug 7 23:56:41.637004566 (14h17m3.381061436s) [upload=14707]: Got next segment. Inline: true Aug 7 23:56:41.637011720 (14h17m3.381068581s) [upload=14707]: Waiting for error group managing segments... Aug 7 23:56:41.637073380 (14h17m3.381130242s) [upload=14707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:41.637079684 (14h17m3.381136546s) [upload=14707]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:41.637083892 (14h17m3.381140753s) [upload=14707]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:41.369074470 (14h18m3.113131318s) [upload=14707]: Done uploading segments... Aug 7 23:57:41.369096313 (14h18m3.11315316s) [upload=14707]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:57:41.785857243 (14h18m3.529914104s) [upload=14726]: Starting upload Aug 7 23:57:41.785973295 (14h18m3.530030155s) [upload=14726]: Done starting upload Aug 7 23:57:41.786053418 (14h18m3.530110268s) [upload=14726]: Uploading segments... Aug 7 23:57:41.786115407 (14h18m3.530172273s) [upload=14726]: Got next segment. Inline: true Aug 7 23:57:41.786121268 (14h18m3.530178127s) [upload=14726]: Waiting for error group managing segments... Aug 7 23:57:41.786162041 (14h18m3.530218901s) [upload=14726]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:57:41.786167247 (14h18m3.530224108s) [upload=14726]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:57:41.786170583 (14h18m3.530227443s) [upload=14726]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:41.518215592 (14h19m3.262272452s) [upload=14726]: Done uploading segments... Aug 7 23:58:41.518237426 (14h19m3.262294275s) [upload=14726]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:58:49.080161447 (14h19m10.824218292s) [upload=14741]: Starting upload Aug 7 23:58:49.080236560 (14h19m10.824293407s) [upload=14741]: Done starting upload Aug 7 23:58:49.080313897 (14h19m10.824370744s) [upload=14741]: Uploading segments... Aug 7 23:58:49.080371825 (14h19m10.824428691s) [upload=14741]: Got next segment. Inline: true Aug 7 23:58:49.080376725 (14h19m10.824433586s) [upload=14741]: Waiting for error group managing segments... Aug 7 23:58:49.080418334 (14h19m10.824475197s) [upload=14741]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:49.080422560 (14h19m10.824479419s) [upload=14741]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:49.080438323 (14h19m10.824495186s) [upload=14741]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:49.220444289 (14h19m10.964501139s) [upload=14741]: Done uploading segments... Aug 7 23:58:49.220462660 (14h19m10.96451951s) [upload=14741]: Upload finished. err: <nil> Aug 7 23:59:26.271206280 (14h19m48.015263132s) [upload=14752]: Starting upload Aug 7 23:59:26.271300842 (14h19m48.015357706s) [upload=14752]: Done starting upload Aug 7 23:59:26.271409542 (14h19m48.015466389s) [upload=14752]: Uploading segments... Aug 7 23:59:26.271471969 (14h19m48.015528839s) [upload=14752]: Got next segment. Inline: true Aug 7 23:59:26.271478422 (14h19m48.015535286s) [upload=14752]: Waiting for error group managing segments... Aug 7 23:59:26.271524304 (14h19m48.015581168s) [upload=14752]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:26.271531803 (14h19m48.01558867s) [upload=14752]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:26.271535740 (14h19m48.015592604s) [upload=14752]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:26.006104454 (14h20m47.750161323s) [upload=14752]: Done uploading segments... Aug 8 00:00:26.006140966 (14h20m47.750197816s) [upload=14752]: Upload finished. err: metaclient: context deadline exceeded Aug 8 00:00:26.408809053 (14h20m48.152865927s) [upload=14766]: Starting upload Aug 8 00:00:26.408886561 (14h20m48.152943434s) [upload=14766]: Done starting upload Aug 8 00:00:26.408975679 (14h20m48.15303253s) [upload=14766]: Uploading segments... Aug 8 00:00:26.409028539 (14h20m48.153085476s) [upload=14766]: Got next segment. Inline: true Aug 8 00:00:26.409034850 (14h20m48.153091712s) [upload=14766]: Waiting for error group managing segments... Aug 8 00:00:26.409085190 (14h20m48.153142042s) [upload=14766]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:00:26.409090930 (14h20m48.153147793s) [upload=14766]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:00:26.409094742 (14h20m48.153151605s) [upload=14766]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:26.549853950 (14h20m48.2939108s) [upload=14766]: Done uploading segments... Aug 8 00:00:26.549878419 (14h20m48.293935265s) [upload=14766]: Upload finished. err: <nil> Aug 8 00:01:08.274172938 (14h21m30.018229799s) [upload=14780]: Starting upload Aug 8 00:01:08.274248443 (14h21m30.018305302s) [upload=14780]: Done starting upload Aug 8 00:01:08.274326739 (14h21m30.018383585s) [upload=14780]: Uploading segments... Aug 8 00:01:08.274380689 (14h21m30.018437549s) [upload=14780]: Got next segment. Inline: true Aug 8 00:01:08.274385704 (14h21m30.018442565s) [upload=14780]: Waiting for error group managing segments... Aug 8 00:01:08.274437335 (14h21m30.018494182s) [upload=14780]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:01:08.274442821 (14h21m30.018499667s) [upload=14780]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:01:08.274445609 (14h21m30.018502454s) [upload=14780]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:08.005343335 (14h22m29.749400197s) [upload=14780]: Done uploading segments... Aug 8 00:02:08.005365121 (14h22m29.749421982s) [upload=14780]: Upload finished. err: metaclient: context deadline exceeded Duration: 946.043219ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.X..............X...........................................] duration: 3.025958022s FAIL (2025-08-08 00:44:30 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== oad Aug 8 00:12:40.688890451 (14h33m2.432947302s) [upload=14976]: Uploading segments... Aug 8 00:12:40.689315108 (14h33m2.433371968s) [upload=14976]: Got next segment. Inline: false Aug 8 00:12:40.689365721 (14h33m2.433422571s) [upload=14976]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:12:40.689371637 (14h33m2.433428498s) [upload=14976]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:12:40.938712718 (14h33m2.682769583s) [seg_pos={0 0}, upload=14976]: Begin upload segment... Aug 8 00:12:40.938780540 (14h33m2.68283739s) [seg_pos={0 0}, upload=14976]: Done begin upload segment. Aug 8 00:12:40.938826511 (14h33m2.682883372s) [noise=true , address=104.36.82.11:4122 , piece_id=T4ATL4PZUGFLXWE47UGHDXX3ID2LEJKQRGQQMF2G4J42TET2JVYQ, seg_pos={0 0}, upload=14976]: Uploading piece... Aug 8 00:12:40.938837427 (14h33m2.682894292s) [noise=true , address=64.49.13.169:4715 , piece_id=TKX4G5GWLDNDPR66UADVNNWV2TAFGCVEMWZAU4GROJXFX3Q537TA, seg_pos={0 0}, upload=14976]: Uploading piece... Aug 8 00:12:40.939397428 (14h33m2.683454296s) [noise=true , address=104.36.82.44:4881 , piece_id=TS5LURERTPPJMH3KOK3ABNKEWSWOF7IUNNKP5DONPQTXKIF4S55Q, seg_pos={0 0}, upload=14976]: Uploading piece... Aug 8 00:12:40.939417193 (14h33m2.683474059s) [noise=true , address=104.36.82.5:4859 , piece_id=FHBE3H7NTIY3PHCICRNYDGY7MEILS27N4O5DLKQQWM5UXELX3EWQ, seg_pos={0 0}, upload=14976]: Uploading piece... Aug 8 00:12:40.939726153 (14h33m2.683783022s) [noise=true , address=104.36.82.45:4937 , piece_id=4Z6T6K7EP3LS3XYSTOARORNVPGKX6HYIHV33I665O5FI4OPGOXNA, seg_pos={0 0}, upload=14976]: Uploading piece... Aug 8 00:12:40.939786076 (14h33m2.683842943s) [noise=true , address=104.36.82.23:4453 , piece_id=V7D5MY7E63SEALXMVDJOS26NYQDQAFJ4SHUISCK6TPKDQKKI4Z3Q, seg_pos={0 0}, upload=14976]: Uploading piece... Aug 8 00:12:40.940057147 (14h33m2.68411401s) [noise=true , address=104.36.82.22:4537 , piece_id=3Q4MNZBN5UPYVL4BC673XQVPMLW3MYTY2CNLKFE7NW6ZWSFHAWPQ, seg_pos={0 0}, upload=14976]: Uploading piece... Aug 8 00:12:40.940208252 (14h33m2.684265105s) [noise=true , address=104.36.82.59:4313 , piece_id=LIWITE4IJVNHLO2DFFDI2M4D67UAHLM223EF6FAXGNGWIIMGH7FA, seg_pos={0 0}, upload=14976]: Uploading piece... Aug 8 00:12:40.940408469 (14h33m2.68446533s) [noise=true , address=64.49.13.154:4930 , piece_id=OGQJV2HMKOE6IPET6A6JRENBWMGJ6NWSGL7Q55653IB6QBAUARQA, seg_pos={0 0}, upload=14976]: Uploading piece... Aug 8 00:12:40.940742041 (14h33m2.684798902s) [noise=true , address=45.91.116.7:4034 , piece_id=XJQV5V55F3KLAF5MZZMWZN7HE6TRHQ2V6BDDMKOWMDMKEBAP6UZQ, seg_pos={0 0}, upload=14976]: Uploading piece... Aug 8 00:12:40.941021709 (14h33m2.685078579s) [noise=true , address=104.36.82.37:4671 , piece_id=HYA6VNL24EERU27SJLYO7TCV7DQIIH7Q4LXAEA5BXSHBTIXCHKDQ, seg_pos={0 0}, upload=14976]: Uploading piece... Aug 8 00:12:40.941299121 (14h33m2.685355986s) [noise=true , address=104.36.82.43:4855 , piece_id=AMNGI7UV2PCEQ2U4LAOXIFT4T3WXPD7YHLR77VR7MSSQGHESY5EA, seg_pos={0 0}, upload=14976]: Uploading piece... Aug 8 00:12:40.941588088 (14h33m2.685644951s) [noise=true , address=104.36.82.19:4401 , piece_id=NACQ6PBCDIHNJAQIFQUVYAPU2HY5C4ZDW6QMJBGY3E45TNWHFQNQ, seg_pos={0 0}, upload=14976]: Uploading piece... Aug 8 00:12:40.941637056 (14h33m2.685693925s) [noise=true , address=104.36.82.61:4404 , piece_id=F5YUVKGDN4MPHJKOLR5DRVABOU6326LGWAJXKL4RCWZI35TLMXKQ, seg_pos={0 0}, upload=14976]: Uploading piece... Aug 8 00:12:40.941950914 (14h33m2.686007788s) [noise=true , address=79.127.201.215:4351 , piece_id=DR37C77MS5YTW7DK7XOJQGMEXWOSUKOD76SZY7TXLIB5RLXLEVMQ, seg_pos={0 0}, upload=14976]: Uploading piece... Aug 8 00:12:40.942345807 (14h33m2.686402678s) [noise=true , address=104.36.82.55:4206 , piece_id=FEFQ3F2EFHMZI53KUIAGHESBATXL5S2EOAARLOHJ2YDSMGVLZRQQ, seg_pos={0 0}, upload=14976]: Uploading piece... Aug 8 00:12:40.942560802 (14h33m2.686617667s) [noise=true , address=104.36.82.12:4158 , piece_id=SCBLDCSKC6HTOUIUA43BA4452TPHEIVKCMOSVHR42YXVUWYUFPRA, seg_pos={0 0}, upload=14976]: Uploading piece... Aug 8 00:12:40.943202713 (14h33m2.687259576s) [noise=true , address=193.149.164.143:4693 , piece_id=P5YP7XDPKE2VYGTIJMAD6TV2LAPXJRN47E3PDPDDLVNI73VJGUBA, seg_pos={0 0}, upload=14976]: Uploading piece... Aug 8 00:12:41.525441834 (14h33m3.269498698s) [upload=14976]: Next returned nil segment Aug 8 00:12:41.525462664 (14h33m3.269519531s) [upload=14976]: Waiting for error group managing segments... Aug 8 00:12:41.699976519 (14h33m3.444033384s) [noise=true , address=104.36.82.19:4401 , piece_id=NACQ6PBCDIHNJAQIFQUVYAPU2HY5C4ZDW6QMJBGY3E45TNWHFQNQ, seg_pos={0 0}, upload=14976]: Done uploading piece. err: <nil> Aug 8 00:12:41.700944219 (14h33m3.445001137s) [noise=true , address=104.36.82.23:4453 , piece_id=V7D5MY7E63SEALXMVDJOS26NYQDQAFJ4SHUISCK6TPKDQKKI4Z3Q, seg_pos={0 0}, upload=14976]: Done uploading piece. err: <nil> Aug 8 00:12:41.702379298 (14h33m3.446436158s) [noise=true , address=104.36.82.37:4671 , piece_id=HYA6VNL24EERU27SJLYO7TCV7DQIIH7Q4LXAEA5BXSHBTIXCHKDQ, seg_pos={0 0}, upload=14976]: Done uploading piece. err: <nil> Aug 8 00:12:41.710467855 (14h33m3.454524716s) [noise=true , address=104.36.82.55:4206 , piece_id=FEFQ3F2EFHMZI53KUIAGHESBATXL5S2EOAARLOHJ2YDSMGVLZRQQ, seg_pos={0 0}, upload=14976]: Done uploading piece. err: <nil> Aug 8 00:12:41.716048399 (14h33m3.46010525s) [noise=true , address=104.36.82.22:4537 , piece_id=3Q4MNZBN5UPYVL4BC673XQVPMLW3MYTY2CNLKFE7NW6ZWSFHAWPQ, seg_pos={0 0}, upload=14976]: Done uploading piece. err: <nil> Aug 8 00:12:41.769601981 (14h33m3.513658832s) [noise=true , address=104.36.82.44:4881 , piece_id=TS5LURERTPPJMH3KOK3ABNKEWSWOF7IUNNKP5DONPQTXKIF4S55Q, seg_pos={0 0}, upload=14976]: Done uploading piece. err: <nil> Aug 8 00:12:41.774083597 (14h33m3.518140448s) [noise=true , address=104.36.82.5:4859 , piece_id=FHBE3H7NTIY3PHCICRNYDGY7MEILS27N4O5DLKQQWM5UXELX3EWQ, seg_pos={0 0}, upload=14976]: Done uploading piece. err: <nil> Aug 8 00:12:41.790244473 (14h33m3.534301324s) [noise=true , address=45.91.116.7:4034 , piece_id=XJQV5V55F3KLAF5MZZMWZN7HE6TRHQ2V6BDDMKOWMDMKEBAP6UZQ, seg_pos={0 0}, upload=14976]: Done uploading piece. err: <nil> Aug 8 00:12:41.790597987 (14h33m3.534654837s) [noise=true , address=104.36.82.61:4404 , piece_id=F5YUVKGDN4MPHJKOLR5DRVABOU6326LGWAJXKL4RCWZI35TLMXKQ, seg_pos={0 0}, upload=14976]: Done uploading piece. err: <nil> Aug 8 00:12:41.814650455 (14h33m3.558707307s) [noise=true , address=104.36.82.59:4313 , piece_id=LIWITE4IJVNHLO2DFFDI2M4D67UAHLM223EF6FAXGNGWIIMGH7FA, seg_pos={0 0}, upload=14976]: Done uploading piece. err: <nil> Aug 8 00:12:41.916929838 (14h33m3.660986699s) [noise=true , address=104.36.82.12:4158 , piece_id=SCBLDCSKC6HTOUIUA43BA4452TPHEIVKCMOSVHR42YXVUWYUFPRA, seg_pos={0 0}, upload=14976]: Done uploading piece. err: <nil> Aug 8 00:12:41.946542933 (14h33m3.690599791s) [noise=true , address=79.127.201.215:4351 , piece_id=DR37C77MS5YTW7DK7XOJQGMEXWOSUKOD76SZY7TXLIB5RLXLEVMQ, seg_pos={0 0}, upload=14976]: Done uploading piece. err: <nil> Aug 8 00:12:42.014532160 (14h33m3.75858901s) [noise=true , address=104.36.82.43:4855 , piece_id=AMNGI7UV2PCEQ2U4LAOXIFT4T3WXPD7YHLR77VR7MSSQGHESY5EA, seg_pos={0 0}, upload=14976]: Done uploading piece. err: <nil> Aug 8 00:12:42.157107757 (14h33m3.901164607s) [noise=true , address=104.36.82.11:4122 , piece_id=T4ATL4PZUGFLXWE47UGHDXX3ID2LEJKQRGQQMF2G4J42TET2JVYQ, seg_pos={0 0}, upload=14976]: Done uploading piece. err: <nil> Aug 8 00:12:42.157146638 (14h33m3.901203502s) [seg_pos={0 0}, upload=14976]: Segment reached optimal threshold of 14 pieces. Aug 8 00:12:42.157376876 (14h33m3.901433737s) [noise=true , address=64.49.13.169:4715 , piece_id=TKX4G5GWLDNDPR66UADVNNWV2TAFGCVEMWZAU4GROJXFX3Q537TA, seg_pos={0 0}, upload=14976]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHWknNPWJ79pDmU1vfv8TCqWKU9sxrP4upTRBNsPnc8LcLgzK): protocol: expected piece hash; context canceled; EOF Aug 8 00:12:42.157574941 (14h33m3.901631791s) [noise=true , address=193.149.164.143:4693 , piece_id=P5YP7XDPKE2VYGTIJMAD6TV2LAPXJRN47E3PDPDDLVNI73VJGUBA, seg_pos={0 0}, upload=14976]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: expected piece hash; context canceled; EOF Aug 8 00:12:42.157631621 (14h33m3.901688486s) [noise=true , address=64.49.13.154:4930 , piece_id=OGQJV2HMKOE6IPET6A6JRENBWMGJ6NWSGL7Q55653IB6QBAUARQA, seg_pos={0 0}, upload=14976]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kNK8y5et7MiiWNDT6iaC8YBAk3PX6WX1XSuU4QMSFBp3KymBb): protocol: expected piece hash; context canceled; EOF Aug 8 00:12:42.157680271 (14h33m3.901737121s) [noise=true , address=104.36.82.45:4937 , piece_id=4Z6T6K7EP3LS3XYSTOARORNVPGKX6HYIHV33I665O5FI4OPGOXNA, seg_pos={0 0}, upload=14976]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LKrFThdW3jJpG6dRZLUaxLXTa5Wy27yQ8PGUEqjvBWfFoRB94): protocol: write tcp 139.59.144.22:51352->104.36.82.45:4937: use of closed network connection; write tcp 139.59.144.22:51352->104.36.82.45:4937: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:51352->104.36.82.45:4937: use of closed network connection Aug 8 00:12:42.157698358 (14h33m3.901755219s) [seg_pos={0 0}, upload=14976]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 00:12:42.157816415 (14h33m3.901873289s) [upload=14976]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:12:42.157839617 (14h33m3.901896479s) [upload=14976]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:12:42.510566001 (14h33m4.254622851s) [upload=14976]: Done uploading segments... Aug 8 00:12:42.510584893 (14h33m4.254641743s) [upload=14976]: Upload finished. err: <nil> Duration: 3.025997196s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 2 out of 60 is failed (threshold=3) [X...X.......................................................] duration: 1.094494468s FAIL (2025-08-08 00:44:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.094550453s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 2.901250953s FAIL (2025-08-08 00:44:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.901275759s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 2 out of 60 is failed (threshold=3) [.XX.........................................................] duration: 855.451079ms FAIL (2025-08-08 00:44:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 140355 (14h13m13.036197215s) [upload=14641]: Starting upload Aug 7 23:52:51.292216133 (14h13m13.036272985s) [upload=14641]: Done starting upload Aug 7 23:52:51.292307446 (14h13m13.036364307s) [upload=14641]: Uploading segments... Aug 7 23:52:51.292389087 (14h13m13.036445956s) [upload=14641]: Got next segment. Inline: true Aug 7 23:52:51.292394933 (14h13m13.036451793s) [upload=14641]: Waiting for error group managing segments... Aug 7 23:52:51.292456032 (14h13m13.036512883s) [upload=14641]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:51.292461787 (14h13m13.036518647s) [upload=14641]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:51.292467676 (14h13m13.036524537s) [upload=14641]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:51.436915828 (14h13m13.180972678s) [upload=14641]: Done uploading segments... Aug 7 23:52:51.436934458 (14h13m13.18099131s) [upload=14641]: Upload finished. err: <nil> Aug 7 23:53:10.946261153 (14h13m32.690318028s) [upload=14646]: Starting upload Aug 7 23:53:10.946350198 (14h13m32.690407065s) [upload=14646]: Done starting upload Aug 7 23:53:10.946435423 (14h13m32.690492276s) [upload=14646]: Uploading segments... Aug 7 23:53:10.946489255 (14h13m32.690546124s) [upload=14646]: Got next segment. Inline: true Aug 7 23:53:10.946497660 (14h13m32.690554519s) [upload=14646]: Waiting for error group managing segments... Aug 7 23:53:10.946542678 (14h13m32.690599529s) [upload=14646]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:10.946548267 (14h13m32.690605128s) [upload=14646]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:10.946553792 (14h13m32.690610652s) [upload=14646]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:14.554651436 (14h13m36.298708309s) [upload=14646]: Done uploading segments... Aug 7 23:53:14.554675183 (14h13m36.298732045s) [upload=14646]: Upload finished. err: <nil> Aug 7 23:53:53.584502029 (14h14m15.328558897s) [upload=14658]: Starting upload Aug 7 23:53:53.584599757 (14h14m15.328656616s) [upload=14658]: Done starting upload Aug 7 23:53:53.584702364 (14h14m15.328759218s) [upload=14658]: Uploading segments... Aug 7 23:53:53.584798610 (14h14m15.32885548s) [upload=14658]: Got next segment. Inline: true Aug 7 23:53:53.584806594 (14h14m15.328863461s) [upload=14658]: Waiting for error group managing segments... Aug 7 23:53:53.584875193 (14h14m15.328932062s) [upload=14658]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:53.584882324 (14h14m15.328939193s) [upload=14658]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:53.584901657 (14h14m15.328958528s) [upload=14658]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:23.318821682 (14h14m45.062878542s) [upload=14658]: Done uploading segments... Aug 7 23:54:23.318863033 (14h14m45.062919884s) [upload=14658]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:36.775209359 (14h14m58.519266209s) [upload=14671]: Starting upload Aug 7 23:54:36.775292235 (14h14m58.519349112s) [upload=14671]: Done starting upload Aug 7 23:54:36.775412031 (14h14m58.519468883s) [upload=14671]: Uploading segments... Aug 7 23:54:36.775480767 (14h14m58.519537636s) [upload=14671]: Got next segment. Inline: true Aug 7 23:54:36.775487320 (14h14m58.519544181s) [upload=14671]: Waiting for error group managing segments... Aug 7 23:54:36.775539752 (14h14m58.519596612s) [upload=14671]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:36.775545893 (14h14m58.519602753s) [upload=14671]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:36.775549619 (14h14m58.519606479s) [upload=14671]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:36.926276427 (14h14m58.670333277s) [upload=14671]: Done uploading segments... Aug 7 23:54:36.926295206 (14h14m58.670352057s) [upload=14671]: Upload finished. err: <nil> Aug 7 23:54:57.675347843 (14h15m19.419404694s) [upload=14676]: Starting upload Aug 7 23:54:57.675412352 (14h15m19.419469212s) [upload=14676]: Done starting upload Aug 7 23:54:57.675475670 (14h15m19.41953252s) [upload=14676]: Uploading segments... Aug 7 23:54:57.675516336 (14h15m19.419573205s) [upload=14676]: Got next segment. Inline: true Aug 7 23:54:57.675521071 (14h15m19.419577931s) [upload=14676]: Waiting for error group managing segments... Aug 7 23:54:57.675565512 (14h15m19.419622375s) [upload=14676]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:57.675574255 (14h15m19.419631202s) [upload=14676]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:57.675580694 (14h15m19.419637563s) [upload=14676]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:27.407552485 (14h15m49.151609344s) [upload=14676]: Done uploading segments... Aug 7 23:55:27.407592978 (14h15m49.151649829s) [upload=14676]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:56:06.851646232 (14h16m28.595703082s) [upload=14696]: Starting upload Aug 7 23:56:06.851717330 (14h16m28.595774191s) [upload=14696]: Done starting upload Aug 7 23:56:06.851786008 (14h16m28.59584286s) [upload=14696]: Uploading segments... Aug 7 23:56:06.851834232 (14h16m28.595891101s) [upload=14696]: Got next segment. Inline: true Aug 7 23:56:06.851839675 (14h16m28.595896535s) [upload=14696]: Waiting for error group managing segments... Aug 7 23:56:06.851896275 (14h16m28.595953125s) [upload=14696]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:06.851901659 (14h16m28.59595852s) [upload=14696]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:06.851905505 (14h16m28.595962364s) [upload=14696]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:07.006202008 (14h16m28.750258868s) [upload=14696]: Done uploading segments... Aug 7 23:56:07.006222000 (14h16m28.750278861s) [upload=14696]: Upload finished. err: <nil> Aug 7 23:56:59.020763574 (14h17m20.764820421s) [upload=14712]: Starting upload Aug 7 23:56:59.020858955 (14h17m20.764915806s) [upload=14712]: Done starting upload Aug 7 23:56:59.020935444 (14h17m20.764992295s) [upload=14712]: Uploading segments... Aug 7 23:56:59.021007501 (14h17m20.76506437s) [upload=14712]: Got next segment. Inline: true Aug 7 23:56:59.021013631 (14h17m20.765070491s) [upload=14712]: Waiting for error group managing segments... Aug 7 23:56:59.021070471 (14h17m20.76512733s) [upload=14712]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:59.021076966 (14h17m20.765133821s) [upload=14712]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:59.021081305 (14h17m20.76513816s) [upload=14712]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:59.154820708 (14h17m20.898877569s) [upload=14712]: Done uploading segments... Aug 7 23:56:59.154839835 (14h17m20.898896685s) [upload=14712]: Upload finished. err: <nil> Aug 7 23:58:10.963677657 (14h18m32.707734508s) [upload=14728]: Starting upload Aug 7 23:58:10.963770332 (14h18m32.70782718s) [upload=14728]: Done starting upload Aug 7 23:58:10.963851725 (14h18m32.707908586s) [upload=14728]: Uploading segments... Aug 7 23:58:10.963901559 (14h18m32.707958431s) [upload=14728]: Got next segment. Inline: true Aug 7 23:58:10.963907663 (14h18m32.707964525s) [upload=14728]: Waiting for error group managing segments... Aug 7 23:58:10.963960167 (14h18m32.708017032s) [upload=14728]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:10.963966034 (14h18m32.708022896s) [upload=14728]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:10.963969944 (14h18m32.708026806s) [upload=14728]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:11.106069715 (14h18m32.850126571s) [upload=14728]: Done uploading segments... Aug 7 23:58:11.106112099 (14h18m32.850168961s) [upload=14728]: Upload finished. err: <nil> Aug 7 23:59:19.078128070 (14h19m40.822184931s) [upload=14749]: Starting upload Aug 7 23:59:19.078217499 (14h19m40.822274359s) [upload=14749]: Done starting upload Aug 7 23:59:19.078291040 (14h19m40.822347895s) [upload=14749]: Uploading segments... Aug 7 23:59:19.078333863 (14h19m40.822390732s) [upload=14749]: Got next segment. Inline: true Aug 7 23:59:19.078339134 (14h19m40.822395984s) [upload=14749]: Waiting for error group managing segments... Aug 7 23:59:19.078391868 (14h19m40.822448717s) [upload=14749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:19.078397213 (14h19m40.822454064s) [upload=14749]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:19.078401054 (14h19m40.822457905s) [upload=14749]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:59:48.808547234 (14h20m10.552604081s) [upload=14749]: Done uploading segments... Aug 7 23:59:48.808568563 (14h20m10.552625409s) [upload=14749]: Upload finished. err: metaclient: context deadline exceeded Aug 8 00:00:20.896255037 (14h20m42.640311905s) [upload=14765]: Starting upload Aug 8 00:00:20.896381648 (14h20m42.640438517s) [upload=14765]: Done starting upload Aug 8 00:00:20.896532416 (14h20m42.640589282s) [upload=14765]: Uploading segments... Aug 8 00:00:20.896610121 (14h20m42.640666994s) [upload=14765]: Got next segment. Inline: true Aug 8 00:00:20.896618785 (14h20m42.640675649s) [upload=14765]: Waiting for error group managing segments... Aug 8 00:00:20.896682486 (14h20m42.640739601s) [upload=14765]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:00:20.896690306 (14h20m42.640747187s) [upload=14765]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:00:20.896694946 (14h20m42.640751814s) [upload=14765]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:50.607106781 (14h21m12.351163663s) [upload=14765]: Done uploading segments... Aug 8 00:00:50.607134684 (14h21m12.351191558s) [upload=14765]: Upload finished. err: metaclient: context deadline exceeded Duration: 855.480528ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 2.546963076s FAIL (2025-08-08 00:44:04 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== 6]: Uploading piece... Aug 7 23:57:10.893816491 (14h17m32.637873354s) [noise=true , address=45.91.116.12:4225 , piece_id=NTUY2VEJFWSFOAGWUPTJYNYNWL3XMK4C3FJUE6MSTMTS2TZJ7VUA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.894285524 (14h17m32.638342391s) [noise=true , address=104.36.82.16:4294 , piece_id=SIMVCA63FAX5EQZJO5TBOSGPVYLFOAICTABKMXS5SCQGNOUPFBIA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.895348932 (14h17m32.639405787s) [noise=true , address=77.90.31.14:4661 , piece_id=WFDXKM726LBL4MH4CJ4AQIUSXTQCQ3BJV34QBUGWPHNFFPZYGZKA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.895932061 (14h17m32.639988921s) [noise=true , address=104.36.82.10:4078 , piece_id=3DPLCXZLQWMZSYSSO3JPLZFP2VXJ5NKYO4FDVFYOWJLT43C7XSTQ, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.896545320 (14h17m32.640602184s) [noise=true , address=77.90.31.10:4518 , piece_id=LU6UTDDO6HUIGXV2QXCL45CIBFQMOQGS7CTQHBRKCBE2JJKLI6FA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.897117472 (14h17m32.641174341s) [noise=true , address=77.90.31.21:4877 , piece_id=SF7XDMNH7KGS26UE6MKUK3VLFZ6IOP6WZKMVO4H6SIAULM623GLA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.897621325 (14h17m32.641678192s) [noise=true , address=77.90.31.12:4589 , piece_id=5I5BMAQARX67CB2RRMCXG2U7KELJ6HSHRT3573EH3ZX6NJ257R4A, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.898101338 (14h17m32.642158204s) [noise=true , address=77.90.31.21:4878 , piece_id=XAXEYDOX54GUKMI22UKTA6BSTSY5WYATTITTGSRQ3BBYDGDCO5DQ, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.898693607 (14h17m32.642750474s) [noise=true , address=45.91.116.6:4975 , piece_id=VDEP57UE6G4CBU7DOYMMOIRXJUTTWPJKDJDGCGUOHBGVPAYCM6PA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.900512475 (14h17m32.644569335s) [noise=true , address=45.91.116.12:4227 , piece_id=J2FM3FAWY5HYXX3VFC6CSHUSWDQDNKUUB6NL6MWSF2ZG3BZFWBEA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.902337804 (14h17m32.646394669s) [noise=true , address=77.90.31.13:4626 , piece_id=PR6GY2Y7HDUORSLYZXU4IVLVA6RIMBFSRFJW4OTLXUZM7MJZ4KOQ, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.904215667 (14h17m32.648272537s) [noise=true , address=77.90.31.19:4841 , piece_id=Q3XFTIVGOSRQARSFONAKX26VAFTJF7THPIQV6USKI7N2LN3JBTLQ, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.906034481 (14h17m32.650091365s) [noise=true , address=104.36.82.5:4839 , piece_id=AV3ZBQ4W44IZEDAPHJGHKK5D5VRNU5XGOQGDWEARNGIKWRQAGYBQ, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.907857426 (14h17m32.651914295s) [noise=true , address=77.90.31.15:4698 , piece_id=WBU562MADMX3KSCRCD2H5R7MQN5IPFBLKYSKU47WMGYYBKZGPYJA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.908666188 (14h17m32.652723069s) [noise=true , address=104.36.82.14:4222 , piece_id=JFWGS6VCDTFOQYG32ZFTWNYLFGVVQPXD7K6TZXC4FR2SGEOUS7FA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:11.510238622 (14h17m33.254295484s) [upload=14716]: Next returned nil segment Aug 7 23:57:11.510261464 (14h17m33.254318324s) [upload=14716]: Waiting for error group managing segments... Aug 7 23:57:11.657094388 (14h17m33.401151236s) [noise=true , address=77.90.31.10:4518 , piece_id=LU6UTDDO6HUIGXV2QXCL45CIBFQMOQGS7CTQHBRKCBE2JJKLI6FA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.674408374 (14h17m33.418465234s) [noise=true , address=77.90.31.15:4698 , piece_id=WBU562MADMX3KSCRCD2H5R7MQN5IPFBLKYSKU47WMGYYBKZGPYJA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.681203586 (14h17m33.425260449s) [noise=true , address=77.90.31.12:4589 , piece_id=5I5BMAQARX67CB2RRMCXG2U7KELJ6HSHRT3573EH3ZX6NJ257R4A, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.682780850 (14h17m33.4268377s) [noise=true , address=77.90.31.21:4877 , piece_id=SF7XDMNH7KGS26UE6MKUK3VLFZ6IOP6WZKMVO4H6SIAULM623GLA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.684042315 (14h17m33.428099176s) [noise=true , address=104.36.82.16:4294 , piece_id=SIMVCA63FAX5EQZJO5TBOSGPVYLFOAICTABKMXS5SCQGNOUPFBIA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.719658547 (14h17m33.463715401s) [noise=true , address=45.91.116.12:4225 , piece_id=NTUY2VEJFWSFOAGWUPTJYNYNWL3XMK4C3FJUE6MSTMTS2TZJ7VUA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.730348391 (14h17m33.474405242s) [noise=true , address=77.90.31.21:4878 , piece_id=XAXEYDOX54GUKMI22UKTA6BSTSY5WYATTITTGSRQ3BBYDGDCO5DQ, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.748264399 (14h17m33.492321249s) [noise=true , address=77.90.31.19:4841 , piece_id=Q3XFTIVGOSRQARSFONAKX26VAFTJF7THPIQV6USKI7N2LN3JBTLQ, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.769764028 (14h17m33.513820878s) [noise=true , address=45.91.116.6:4975 , piece_id=VDEP57UE6G4CBU7DOYMMOIRXJUTTWPJKDJDGCGUOHBGVPAYCM6PA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.774177874 (14h17m33.518234733s) [noise=true , address=77.90.31.14:4661 , piece_id=WFDXKM726LBL4MH4CJ4AQIUSXTQCQ3BJV34QBUGWPHNFFPZYGZKA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.789864472 (14h17m33.533921326s) [noise=true , address=45.91.116.13:4153 , piece_id=WBFFQT5I6ZR5ZKUJRFULNOJZ455KIJMA5EEATDDOCVFUKRD67NFA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.799519597 (14h17m33.543576447s) [noise=true , address=45.91.116.12:4227 , piece_id=J2FM3FAWY5HYXX3VFC6CSHUSWDQDNKUUB6NL6MWSF2ZG3BZFWBEA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.800753821 (14h17m33.544810667s) [noise=true , address=77.90.31.13:4626 , piece_id=PR6GY2Y7HDUORSLYZXU4IVLVA6RIMBFSRFJW4OTLXUZM7MJZ4KOQ, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.800785983 (14h17m33.544842842s) [seg_pos={0 0}, upload=14716]: Segment reached optimal threshold of 13 pieces. Aug 7 23:57:11.801026052 (14h17m33.545082898s) [noise=true , address=104.36.82.14:4222 , piece_id=JFWGS6VCDTFOQYG32ZFTWNYLFGVVQPXD7K6TZXC4FR2SGEOUS7FA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1): protocol: expected piece hash; context canceled; EOF Aug 7 23:57:11.801098020 (14h17m33.545154867s) [noise=true , address=104.36.82.10:4078 , piece_id=3DPLCXZLQWMZSYSSO3JPLZFP2VXJ5NKYO4FDVFYOWJLT43C7XSTQ, seg_pos={0 0}, upload=14716]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Aug 7 23:57:11.801174944 (14h17m33.54523179s) [noise=true , address=104.36.82.5:4839 , piece_id=AV3ZBQ4W44IZEDAPHJGHKK5D5VRNU5XGOQGDWEARNGIKWRQAGYBQ, seg_pos={0 0}, upload=14716]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 139.59.144.22:48474->104.36.82.5:4839: use of closed network connection; write tcp 139.59.144.22:48474->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48474->104.36.82.5:4839: use of closed network connection Aug 7 23:57:11.801184926 (14h17m33.545241773s) [seg_pos={0 0}, upload=14716]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:57:11.801257096 (14h17m33.545313956s) [upload=14716]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:57:11.801261009 (14h17m33.545317869s) [upload=14716]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:11.942784613 (14h17m33.686841513s) [upload=14716]: Done uploading segments... Aug 7 23:57:11.942815424 (14h17m33.686872279s) [upload=14716]: Upload finished. err: <nil> Aug 7 23:58:20.561350310 (14h18m42.305407184s) [upload=14732]: Starting upload Aug 7 23:58:20.561483402 (14h18m42.30554027s) [upload=14732]: Done starting upload Aug 7 23:58:20.561650297 (14h18m42.305707159s) [upload=14732]: Uploading segments... Aug 7 23:58:20.562196341 (14h18m42.306253212s) [upload=14732]: Got next segment. Inline: false Aug 7 23:58:20.562334166 (14h18m42.306391031s) [upload=14732]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:20.562343255 (14h18m42.306400125s) [upload=14732]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:58:50.281195862 (14h19m12.025252712s) [upload=14732]: Next segment err: context deadline exceeded Aug 7 23:58:50.281383705 (14h19m12.025440555s) [upload=14732]: Done uploading segments... Aug 7 23:58:50.281405600 (14h19m12.02546245s) [upload=14732]: Upload finished. err: context deadline exceeded Aug 7 23:58:50.716777496 (14h19m12.460834346s) [upload=14742]: Starting upload Aug 7 23:58:50.716857905 (14h19m12.460914751s) [upload=14742]: Done starting upload Aug 7 23:58:50.716999972 (14h19m12.461056825s) [upload=14742]: Uploading segments... Aug 7 23:58:50.717186124 (14h19m12.461242976s) [upload=14742]: Got next segment. Inline: false Aug 7 23:58:50.717245601 (14h19m12.461302466s) [upload=14742]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:50.717252185 (14h19m12.461309053s) [upload=14742]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:59:20.436529435 (14h19m42.180586305s) [upload=14742]: Next segment err: context deadline exceeded Aug 7 23:59:20.436734882 (14h19m42.180791733s) [upload=14742]: Done uploading segments... Aug 7 23:59:20.436756200 (14h19m42.180813054s) [upload=14742]: Upload finished. err: context deadline exceeded Duration: 2.546998225s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)