Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 15 out of 60 is failed (threshold=3) [...XXX.XXXXXX..XX.X...................................X...XX] duration: 30.001776179s FAIL (2025-08-07 23:54:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7 23:46:47.210142995 (14h7m8.954199841s) [upload=14538]: Upload finished. err: <nil> Aug 7 23:47:29.298549723 (14h7m51.042606575s) [upload=14552]: Starting upload Aug 7 23:47:29.298617898 (14h7m51.042674767s) [upload=14552]: Done starting upload Aug 7 23:47:29.298698678 (14h7m51.042755528s) [upload=14552]: Uploading segments... Aug 7 23:47:29.298753306 (14h7m51.042810175s) [upload=14552]: Got next segment. Inline: true Aug 7 23:47:29.298758297 (14h7m51.042815158s) [upload=14552]: Waiting for error group managing segments... Aug 7 23:47:29.298818439 (14h7m51.042875289s) [upload=14552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:47:29.298823844 (14h7m51.042880694s) [upload=14552]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:47:29.298827075 (14h7m51.042883925s) [upload=14552]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:47:29.457296385 (14h7m51.201353233s) [upload=14552]: Done uploading segments... Aug 7 23:47:29.457315555 (14h7m51.201372401s) [upload=14552]: Upload finished. err: <nil> Aug 7 23:48:42.140972885 (14h9m3.885029736s) [upload=14571]: Starting upload Aug 7 23:48:42.141036695 (14h9m3.885093558s) [upload=14571]: Done starting upload Aug 7 23:48:42.141117369 (14h9m3.88517422s) [upload=14571]: Uploading segments... Aug 7 23:48:42.141203219 (14h9m3.885260088s) [upload=14571]: Got next segment. Inline: true Aug 7 23:48:42.141210076 (14h9m3.885266937s) [upload=14571]: Waiting for error group managing segments... Aug 7 23:48:42.141288577 (14h9m3.885345438s) [upload=14571]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:48:42.141294246 (14h9m3.885351107s) [upload=14571]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:48:42.141297649 (14h9m3.885354509s) [upload=14571]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:48:42.292290080 (14h9m4.036346927s) [upload=14571]: Done uploading segments... Aug 7 23:48:42.292309333 (14h9m4.036366179s) [upload=14571]: Upload finished. err: <nil> Aug 7 23:49:47.788743937 (14h10m9.532800783s) [upload=14591]: Starting upload Aug 7 23:49:47.788822734 (14h10m9.5328796s) [upload=14591]: Done starting upload Aug 7 23:49:47.788889290 (14h10m9.532946138s) [upload=14591]: Uploading segments... Aug 7 23:49:47.788950938 (14h10m9.533007798s) [upload=14591]: Got next segment. Inline: true Aug 7 23:49:47.788955912 (14h10m9.533012775s) [upload=14591]: Waiting for error group managing segments... Aug 7 23:49:47.789005080 (14h10m9.533061926s) [upload=14591]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:49:47.789009334 (14h10m9.533066194s) [upload=14591]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:49:47.789014285 (14h10m9.53307113s) [upload=14591]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:49:47.932333015 (14h10m9.676389863s) [upload=14591]: Done uploading segments... Aug 7 23:49:47.932362673 (14h10m9.676419519s) [upload=14591]: Upload finished. err: <nil> Aug 7 23:50:23.531847225 (14h10m45.275904072s) [upload=14601]: Starting upload Aug 7 23:50:23.531926184 (14h10m45.275983047s) [upload=14601]: Done starting upload Aug 7 23:50:23.531993110 (14h10m45.27604996s) [upload=14601]: Uploading segments... Aug 7 23:50:23.532035359 (14h10m45.276092227s) [upload=14601]: Got next segment. Inline: true Aug 7 23:50:23.532041373 (14h10m45.276098223s) [upload=14601]: Waiting for error group managing segments... Aug 7 23:50:23.532083061 (14h10m45.276139912s) [upload=14601]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:23.532088436 (14h10m45.276145285s) [upload=14601]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:23.532091688 (14h10m45.276148539s) [upload=14601]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:23.673405160 (14h10m45.417462011s) [upload=14601]: Done uploading segments... Aug 7 23:50:23.673423409 (14h10m45.417480259s) [upload=14601]: Upload finished. err: <nil> Aug 7 23:51:32.204614635 (14h11m53.948671502s) [upload=14620]: Starting upload Aug 7 23:51:32.204700450 (14h11m53.948757313s) [upload=14620]: Done starting upload Aug 7 23:51:32.204781575 (14h11m53.948838427s) [upload=14620]: Uploading segments... Aug 7 23:51:32.204839741 (14h11m53.94889661s) [upload=14620]: Got next segment. Inline: true Aug 7 23:51:32.204847624 (14h11m53.948904474s) [upload=14620]: Waiting for error group managing segments... Aug 7 23:51:32.204902472 (14h11m53.948959327s) [upload=14620]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:51:32.204907775 (14h11m53.948964625s) [upload=14620]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:51:32.204911177 (14h11m53.948968027s) [upload=14620]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:01.937570352 (14h12m23.681627223s) [upload=14620]: Done uploading segments... Aug 7 23:52:01.937615027 (14h12m23.681671888s) [upload=14620]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:52:03.484443714 (14h12m25.228500575s) [upload=14625]: Starting upload Aug 7 23:52:03.484557174 (14h12m25.228614037s) [upload=14625]: Done starting upload Aug 7 23:52:03.484649660 (14h12m25.22870651s) [upload=14625]: Uploading segments... Aug 7 23:52:03.484705970 (14h12m25.22876283s) [upload=14625]: Got next segment. Inline: true Aug 7 23:52:03.484711540 (14h12m25.228768402s) [upload=14625]: Waiting for error group managing segments... Aug 7 23:52:03.484758149 (14h12m25.228815009s) [upload=14625]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:03.484763263 (14h12m25.228820122s) [upload=14625]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:03.484766746 (14h12m25.228823606s) [upload=14625]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:03.643631412 (14h12m25.387688267s) [upload=14625]: Done uploading segments... Aug 7 23:52:03.643670624 (14h12m25.387727482s) [upload=14625]: Upload finished. err: <nil> Aug 7 23:52:25.641252610 (14h12m47.385309456s) [upload=14634]: Starting upload Aug 7 23:52:25.641347708 (14h12m47.385404571s) [upload=14634]: Done starting upload Aug 7 23:52:25.641412738 (14h12m47.385469584s) [upload=14634]: Uploading segments... Aug 7 23:52:25.641487429 (14h12m47.385544276s) [upload=14634]: Got next segment. Inline: true Aug 7 23:52:25.641494756 (14h12m47.385551602s) [upload=14634]: Waiting for error group managing segments... Aug 7 23:52:25.641535502 (14h12m47.385592365s) [upload=14634]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:25.641556088 (14h12m47.385612953s) [upload=14634]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:25.641559364 (14h12m47.385616224s) [upload=14634]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:25.852337418 (14h12m47.596394279s) [upload=14634]: Done uploading segments... Aug 7 23:52:25.852355736 (14h12m47.596412584s) [upload=14634]: Upload finished. err: <nil> Aug 7 23:52:51.292140355 (14h13m13.036197215s) [upload=14641]: Starting upload Aug 7 23:52:51.292216133 (14h13m13.036272985s) [upload=14641]: Done starting upload Aug 7 23:52:51.292307446 (14h13m13.036364307s) [upload=14641]: Uploading segments... Aug 7 23:52:51.292389087 (14h13m13.036445956s) [upload=14641]: Got next segment. Inline: true Aug 7 23:52:51.292394933 (14h13m13.036451793s) [upload=14641]: Waiting for error group managing segments... Aug 7 23:52:51.292456032 (14h13m13.036512883s) [upload=14641]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:51.292461787 (14h13m13.036518647s) [upload=14641]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:51.292467676 (14h13m13.036524537s) [upload=14641]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:51.436915828 (14h13m13.180972678s) [upload=14641]: Done uploading segments... Aug 7 23:52:51.436934458 (14h13m13.18099131s) [upload=14641]: Upload finished. err: <nil> Aug 7 23:53:10.946261153 (14h13m32.690318028s) [upload=14646]: Starting upload Aug 7 23:53:10.946350198 (14h13m32.690407065s) [upload=14646]: Done starting upload Aug 7 23:53:10.946435423 (14h13m32.690492276s) [upload=14646]: Uploading segments... Aug 7 23:53:10.946489255 (14h13m32.690546124s) [upload=14646]: Got next segment. Inline: true Aug 7 23:53:10.946497660 (14h13m32.690554519s) [upload=14646]: Waiting for error group managing segments... Aug 7 23:53:10.946542678 (14h13m32.690599529s) [upload=14646]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:10.946548267 (14h13m32.690605128s) [upload=14646]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:10.946553792 (14h13m32.690610652s) [upload=14646]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:14.554651436 (14h13m36.298708309s) [upload=14646]: Done uploading segments... Aug 7 23:53:14.554675183 (14h13m36.298732045s) [upload=14646]: Upload finished. err: <nil> Aug 7 23:53:53.584502029 (14h14m15.328558897s) [upload=14658]: Starting upload Aug 7 23:53:53.584599757 (14h14m15.328656616s) [upload=14658]: Done starting upload Aug 7 23:53:53.584702364 (14h14m15.328759218s) [upload=14658]: Uploading segments... Aug 7 23:53:53.584798610 (14h14m15.32885548s) [upload=14658]: Got next segment. Inline: true Aug 7 23:53:53.584806594 (14h14m15.328863461s) [upload=14658]: Waiting for error group managing segments... Aug 7 23:53:53.584875193 (14h14m15.328932062s) [upload=14658]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:53.584882324 (14h14m15.328939193s) [upload=14658]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:53.584901657 (14h14m15.328958528s) [upload=14658]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:23.318821682 (14h14m45.062878542s) [upload=14658]: Done uploading segments... Aug 7 23:54:23.318863033 (14h14m45.062919884s) [upload=14658]: Upload finished. err: metaclient: context deadline exceeded Duration: 30.001820701s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 9 out of 60 is failed (threshold=3) [.....X.XXXXX.X............................................XX] duration: 1m0.001295774s FAIL (2025-08-07 23:54:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001330627s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 3.481831697s FAIL (2025-08-07 23:54:21 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== 294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:32:45.172127356 (13h53m6.916184241s) [upload=14294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:32:45.390149151 (13h53m7.134206011s) [seg_pos={0 0}, upload=14294]: Begin upload segment... Aug 7 23:32:45.390215910 (13h53m7.134272761s) [seg_pos={0 0}, upload=14294]: Done begin upload segment. Aug 7 23:32:45.390249193 (13h53m7.134306059s) [noise=true , address=104.36.82.19:4405 , piece_id=MHM3MWZJFTXSJT7PTOFSWHUYO3L6NIBUV6VU66XXZDH3MC3EFR2Q, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390272919 (13h53m7.134329788s) [noise=true , address=77.90.31.12:4593 , piece_id=C3RVQHFS4L7SKPFFIZ52MAQWGD7P4JNG3SNPCNY53SQUD72PZRUA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390658970 (13h53m7.13471583s) [noise=true , address=104.36.82.16:4321 , piece_id=YTF4RWV2SD6Z4UV4XRYPBRGPCTSUTC6TQKL3DHVKU65ALTNA4TEQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390852777 (13h53m7.134909659s) [noise=true , address=104.36.82.34:4566 , piece_id=3YEAUSXEL7WNZ3O7WM24Z2DKY2TDVKMRZSHMEPMNEL2LR7BY7DBQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390957991 (13h53m7.135014851s) [noise=true , address=104.36.82.47:4523 , piece_id=MJIMHSO66V7OWFFK2UALPKSI7CGGCTHNPZINDXMVIHDLENIW5LFA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.391483291 (13h53m7.135540162s) [noise=true , address=104.36.82.5:4861 , piece_id=GRDGO5LNCETQWRU3M6LACGRW5MNA2KFLXUBT5R2UH6SEGWP6RVRQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.391931094 (13h53m7.135987964s) [noise=true , address=104.36.82.10:4090 , piece_id=V6NPPVIYEH6WLIW7OMT3NDFPR7UDWIT46GW6AFCAWUJTBLC3O55Q, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392250266 (13h53m7.136307136s) [noise=true , address=104.36.82.62:4439 , piece_id=ILRD4TEFW4PSLV4EGPORG3FM6U2HB3WSL5KP3PN566FAOGWODLYQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392611689 (13h53m7.136668559s) [noise=true , address=104.36.82.54:4160 , piece_id=GDBORRYX5QBQGH7ZMMUUAV7AAR5UHZOTK5FTQUPC7YKZC2KD7I3A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392917410 (13h53m7.136974282s) [noise=true , address=104.36.82.23:4477 , piece_id=HAF6424ZCIBGSR4US5LN5ZWE5JMNMLFGUMQQ4YP2ARP7PQXBH5GQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393270536 (13h53m7.137327398s) [noise=true , address=104.36.82.44:4881 , piece_id=XXVSXG3ILFXG4IVMELHL45TRRHDFVVJMJ5LGOHU5EDHG6XXCDH6A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393582635 (13h53m7.137639498s) [noise=true , address=104.36.82.37:4665 , piece_id=RUKKGC4Y3FIVLZO2GRWJN3J6P36VWP5VSC2HCLEU3MZDDFXODJAQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393856926 (13h53m7.137913788s) [noise=true , address=104.36.82.11:4128 , piece_id=XRPULHI7BYDF2JOIASSN5QKGEAC6FJS2F5C22WIY7EKYILRHZ73A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394196364 (13h53m7.138253227s) [noise=true , address=104.36.82.38:4706 , piece_id=BIHIPQX6LRWBNFN4SDC3K3V22AMGL3RC5B4RVE7QXJFKB4XDN3BA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394484397 (13h53m7.138541257s) [noise=true , address=104.36.82.61:4381 , piece_id=TTZOSRP6YPL6P4C6PY5S4K7Z6CIZJDPX7NUNBCK73CWZPCA3HQHA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394593709 (13h53m7.138650578s) [noise=true , address=45.91.116.5:4300 , piece_id=MADJCANN2O35MKXT2PIR5PGMJ3PNTAQAJNNGOK6ISJNVAH2L7YIA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.395093013 (13h53m7.139149879s) [noise=true , address=104.36.82.22:4530 , piece_id=HMUUW5GYH6TRHYXP4YXBHD566FWQRILEHI53AVURA5QTQGG6USFA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.395447945 (13h53m7.139504805s) [noise=true , address=104.36.82.58:4293 , piece_id=I2ETFXYTHYWYJYSHTVOBQTO6ZPGNMA4D3NVKYDE2W34EH4SQE23A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.954469844 (13h53m7.698526694s) [upload=14294]: Next returned nil segment Aug 7 23:32:45.954487953 (13h53m7.69854487s) [upload=14294]: Waiting for error group managing segments... Aug 7 23:32:46.126888627 (13h53m7.870945474s) [noise=true , address=104.36.82.10:4090 , piece_id=V6NPPVIYEH6WLIW7OMT3NDFPR7UDWIT46GW6AFCAWUJTBLC3O55Q, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.132983844 (13h53m7.877040704s) [noise=true , address=104.36.82.16:4321 , piece_id=YTF4RWV2SD6Z4UV4XRYPBRGPCTSUTC6TQKL3DHVKU65ALTNA4TEQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.138277216 (13h53m7.882334067s) [noise=true , address=104.36.82.23:4477 , piece_id=HAF6424ZCIBGSR4US5LN5ZWE5JMNMLFGUMQQ4YP2ARP7PQXBH5GQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.142033326 (13h53m7.886090185s) [noise=true , address=104.36.82.34:4566 , piece_id=3YEAUSXEL7WNZ3O7WM24Z2DKY2TDVKMRZSHMEPMNEL2LR7BY7DBQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.151247206 (13h53m7.895304058s) [noise=true , address=104.36.82.22:4530 , piece_id=HMUUW5GYH6TRHYXP4YXBHD566FWQRILEHI53AVURA5QTQGG6USFA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.151981081 (13h53m7.896037932s) [noise=true , address=104.36.82.58:4293 , piece_id=I2ETFXYTHYWYJYSHTVOBQTO6ZPGNMA4D3NVKYDE2W34EH4SQE23A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.165022882 (13h53m7.909079743s) [noise=true , address=104.36.82.44:4881 , piece_id=XXVSXG3ILFXG4IVMELHL45TRRHDFVVJMJ5LGOHU5EDHG6XXCDH6A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.184418821 (13h53m7.928475667s) [noise=true , address=104.36.82.47:4523 , piece_id=MJIMHSO66V7OWFFK2UALPKSI7CGGCTHNPZINDXMVIHDLENIW5LFA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.184532669 (13h53m7.928589516s) [noise=true , address=45.91.116.5:4300 , piece_id=MADJCANN2O35MKXT2PIR5PGMJ3PNTAQAJNNGOK6ISJNVAH2L7YIA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.193375808 (13h53m7.937432656s) [noise=true , address=104.36.82.54:4160 , piece_id=GDBORRYX5QBQGH7ZMMUUAV7AAR5UHZOTK5FTQUPC7YKZC2KD7I3A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.204247501 (13h53m7.948304372s) [noise=true , address=104.36.82.38:4706 , piece_id=BIHIPQX6LRWBNFN4SDC3K3V22AMGL3RC5B4RVE7QXJFKB4XDN3BA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.235225782 (13h53m7.979282629s) [noise=true , address=104.36.82.62:4439 , piece_id=ILRD4TEFW4PSLV4EGPORG3FM6U2HB3WSL5KP3PN566FAOGWODLYQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.269129867 (13h53m8.013186721s) [noise=true , address=104.36.82.5:4861 , piece_id=GRDGO5LNCETQWRU3M6LACGRW5MNA2KFLXUBT5R2UH6SEGWP6RVRQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.278216912 (13h53m8.022273764s) [noise=true , address=104.36.82.19:4405 , piece_id=MHM3MWZJFTXSJT7PTOFSWHUYO3L6NIBUV6VU66XXZDH3MC3EFR2Q, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.278247847 (13h53m8.022304708s) [seg_pos={0 0}, upload=14294]: Segment reached optimal threshold of 14 pieces. Aug 7 23:32:46.278485612 (13h53m8.022542463s) [noise=true , address=104.36.82.11:4128 , piece_id=XRPULHI7BYDF2JOIASSN5QKGEAC6FJS2F5C22WIY7EKYILRHZ73A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wABkAtAjoYkKyNoAiaVCxSvY6WgvHvAYFPCjpNojtpiYiGtDS): protocol: expected piece hash; context canceled; EOF Aug 7 23:32:46.278511698 (13h53m8.022568566s) [noise=true , address=77.90.31.12:4593 , piece_id=C3RVQHFS4L7SKPFFIZ52MAQWGD7P4JNG3SNPCNY53SQUD72PZRUA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kSxoZb9nmG6oGNytpFTsoiGh3ZPmebVVe8TZxGzANaw7EfdeF): protocol: expected piece hash; context canceled; EOF Aug 7 23:32:46.278642807 (13h53m8.022699657s) [noise=true , address=104.36.82.61:4381 , piece_id=TTZOSRP6YPL6P4C6PY5S4K7Z6CIZJDPX7NUNBCK73CWZPCA3HQHA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zU35MWWaaC1npkP5qGcWBXAkfxGsd5UsdkM2nwX9x7wUGCDbY): protocol: write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection; write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection Aug 7 23:32:46.278650926 (13h53m8.022707786s) [noise=true , address=104.36.82.37:4665 , piece_id=RUKKGC4Y3FIVLZO2GRWJN3J6P36VWP5VSC2HCLEU3MZDDFXODJAQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jP9JrCmoQSdqrzXZxkNN1ihdFkaUMjGZss6Fh19zbL3YyWj5N): protocol: write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection; write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection Aug 7 23:32:46.278674050 (13h53m8.022730911s) [seg_pos={0 0}, upload=14294]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:32:46.278751154 (13h53m8.022808014s) [upload=14294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:32:46.278756168 (13h53m8.022813029s) [upload=14294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:32:46.498660020 (13h53m8.242716882s) [upload=14294]: Done uploading segments... Aug 7 23:32:46.498701263 (13h53m8.242758128s) [upload=14294]: Upload finished. err: <nil> Duration: 3.481908242s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 11 out of 60 is failed (threshold=3) [......XXXXXX.XX........................................X.XX.] duration: 2.624007647s FAIL (2025-08-07 23:54:19 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.624033326s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 7 out of 60 is failed (threshold=3) [..........X...X.XX.......................................XXX] duration: 1m0.00097252s FAIL (2025-08-07 23:54:18 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== ={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.400485298 (14h11m59.144542144s) [noise=true , address=104.36.82.61:4387 , piece_id=N3THT2DLNTVJK72OPSHHTBG34WP53NHR7L5VC6IU5EEXW7O7EPIA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.400710393 (14h11m59.144767254s) [noise=true , address=45.91.116.14:4218 , piece_id=4JCROTVISHYY7LBCTQ637ZROGDEDIJUH222CFSKVIYDBJR7BTK3Q, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.400941379 (14h11m59.14499824s) [noise=true , address=104.36.82.23:4453 , piece_id=ZKDR6X6V3GMABOUC66VFXH33QVIEC3PQSMLLVJMMOBQIFNQVOKZA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.401251698 (14h11m59.145308559s) [noise=true , address=104.36.82.42:4808 , piece_id=HBNT2DIZC4L66S7FLKXCK5XL4JDRVXTYZD6NPYYHUBC72ZHRDFKA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.403864642 (14h11m59.147921505s) [noise=true , address=104.36.82.12:4165 , piece_id=XRWHBRGQHCBTIBJUYQMW337V3ZDR2GP67ARSQVLIX5KALUD27AWA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.404219088 (14h11m59.148275951s) [noise=true , address=104.36.82.41:4765 , piece_id=K2M5MCWY2WSDKPTBPCEL5GQ3FDFQYCFUOBW5CNJUVB72ALWM3DRA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.404507731 (14h11m59.148564592s) [noise=true , address=104.36.82.45:4909 , piece_id=CEPK5VMIAWXINNVXZIDAUQMIMO42RGHBSMC5PFECCZ7TM4BZIUUA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.404771250 (14h11m59.148828112s) [noise=true , address=79.127.201.215:4349 , piece_id=NMWPQT2OMODFL4ZPUJMYMAXPQ27KIQOUXSGKJREJGV65ZLZIBY7Q, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.405722883 (14h11m59.149779746s) [noise=true , address=104.36.82.37:4674 , piece_id=CXTRIGLDCWLTVOB3TUPG6S7VAGWBNBGPTNTWZOGJFXJA3CYQQ3EA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.405966619 (14h11m59.150023478s) [noise=true , address=104.36.82.9:4039 , piece_id=TL3QGWAW4ZHOULIJRERYSPBHH5YAA4SRYQ3SMPCZ5T6O7E6SVNUQ, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.406068528 (14h11m59.150125388s) [noise=true , address=104.36.82.62:4434 , piece_id=CQPE5HJRY2LXTHJWWMYBNDHVOR5B5MBM5DGQQ7MRWXJR6M7IKOBA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.406404119 (14h11m59.150460966s) [noise=true , address=104.36.82.60:4352 , piece_id=JZ2HOZNEYEHOFN5JT67EAVLY7EQGP3B5ENHUFRI6NJCH276P72QQ, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.406657446 (14h11m59.150714311s) [noise=true , address=104.36.82.58:4272 , piece_id=5UBUOONWVORSDNUG2D3WW7VYRK4N22D6Y3Y3GNBMHF2ECZUXDAKQ, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.983166607 (14h11m59.727223467s) [upload=14617]: Next returned nil segment Aug 7 23:51:37.983195615 (14h11m59.727252481s) [upload=14617]: Waiting for error group managing segments... Aug 7 23:51:38.130379502 (14h11m59.874436364s) [noise=true , address=104.36.82.53:4136 , piece_id=IOQFF6W5IC5AUDFV76ZVTKWUAIOVNNXK6WSCCINXXTZLFRIJWHWQ, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.151652145 (14h11m59.895708992s) [noise=true , address=104.36.82.41:4765 , piece_id=K2M5MCWY2WSDKPTBPCEL5GQ3FDFQYCFUOBW5CNJUVB72ALWM3DRA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.152381420 (14h11m59.896438274s) [noise=true , address=104.36.82.58:4272 , piece_id=5UBUOONWVORSDNUG2D3WW7VYRK4N22D6Y3Y3GNBMHF2ECZUXDAKQ, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.153368721 (14h11m59.897425572s) [noise=true , address=104.36.82.55:4205 , piece_id=HI7WZRDI3BHRFWQYMDR4Z4IHJUAYOA4B3ECS65MQKQZN2YDEVHZQ, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.166206728 (14h11m59.910263574s) [noise=true , address=104.36.82.60:4352 , piece_id=JZ2HOZNEYEHOFN5JT67EAVLY7EQGP3B5ENHUFRI6NJCH276P72QQ, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.178216859 (14h11m59.922273709s) [noise=true , address=104.36.82.42:4808 , piece_id=HBNT2DIZC4L66S7FLKXCK5XL4JDRVXTYZD6NPYYHUBC72ZHRDFKA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.191290896 (14h11m59.935347742s) [noise=true , address=45.91.116.14:4218 , piece_id=4JCROTVISHYY7LBCTQ637ZROGDEDIJUH222CFSKVIYDBJR7BTK3Q, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.192130895 (14h11m59.936187748s) [noise=true , address=104.36.82.61:4387 , piece_id=N3THT2DLNTVJK72OPSHHTBG34WP53NHR7L5VC6IU5EEXW7O7EPIA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.206383044 (14h11m59.950439898s) [noise=true , address=104.36.82.12:4165 , piece_id=XRWHBRGQHCBTIBJUYQMW337V3ZDR2GP67ARSQVLIX5KALUD27AWA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.214301438 (14h11m59.958358284s) [noise=true , address=104.36.82.9:4039 , piece_id=TL3QGWAW4ZHOULIJRERYSPBHH5YAA4SRYQ3SMPCZ5T6O7E6SVNUQ, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.313332841 (14h12m0.057389691s) [noise=true , address=104.36.82.62:4434 , piece_id=CQPE5HJRY2LXTHJWWMYBNDHVOR5B5MBM5DGQQ7MRWXJR6M7IKOBA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.369163404 (14h12m0.113220261s) [noise=true , address=104.36.82.23:4453 , piece_id=ZKDR6X6V3GMABOUC66VFXH33QVIEC3PQSMLLVJMMOBQIFNQVOKZA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.468307450 (14h12m0.21236431s) [noise=true , address=104.36.82.37:4674 , piece_id=CXTRIGLDCWLTVOB3TUPG6S7VAGWBNBGPTNTWZOGJFXJA3CYQQ3EA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.539243271 (14h12m0.283300147s) [noise=true , address=79.127.201.215:4349 , piece_id=NMWPQT2OMODFL4ZPUJMYMAXPQ27KIQOUXSGKJREJGV65ZLZIBY7Q, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.539285855 (14h12m0.283342716s) [seg_pos={0 0}, upload=14617]: Segment reached optimal threshold of 14 pieces. Aug 7 23:51:38.539484535 (14h12m0.283541382s) [noise=true , address=104.36.82.45:4909 , piece_id=CEPK5VMIAWXINNVXZIDAUQMIMO42RGHBSMC5PFECCZ7TM4BZIUUA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14Pqp5cbhmd44QnikAuD5TgsKFDR3gB2qtLuVSroaWVv4U2cDh): protocol: expected piece hash; context canceled; EOF Aug 7 23:51:38.539517293 (14h12m0.283574153s) [noise=true , address=64.49.13.138:4488 , piece_id=C3DZT3DLUGAFQTRQARWSBWVJ4I77QWHJGIQZTVX4W6LU5HABXAZA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mQqzHsTJvR5wHpEGXDixk3NcRuBKxY2cL82eJMHidcxFiPCZb): protocol: expected piece hash; context canceled; EOF Aug 7 23:51:38.539590397 (14h12m0.283647259s) [noise=true , address=104.36.82.43:4868 , piece_id=VP5ASYQLM6PAWHV4DMBVHIRSDX7WPMXORUCQQHLXQ7UE5WUNGY7Q, seg_pos={0 0}, upload=14617]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124LJtYshPwu1uZJwVyjrjUMk3htyVnp9ntnZWSUb7X3bNkwU6t): protocol: expected piece hash; context canceled; EOF Aug 7 23:51:38.539646799 (14h12m0.28370366s) [noise=true , address=64.49.13.169:4707 , piece_id=Y5MWKGWFH3LP2T42UHWOLNJTHV3SCKK2ZA7HRCBVHVDIG2UQEN7Q, seg_pos={0 0}, upload=14617]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: expected piece hash; context canceled; EOF Aug 7 23:51:38.539673304 (14h12m0.283730165s) [seg_pos={0 0}, upload=14617]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:51:38.539768774 (14h12m0.283825642s) [upload=14617]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:51:38.539775704 (14h12m0.283832576s) [upload=14617]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:17.809583811 (14h12m39.553640671s) [upload=14617]: Done uploading segments... Aug 7 23:52:17.809611159 (14h12m39.55366802s) [upload=14617]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:52:18.222005824 (14h12m39.966062686s) [upload=14632]: Starting upload Aug 7 23:52:18.222080177 (14h12m39.966137038s) [upload=14632]: Done starting upload Aug 7 23:52:18.222201874 (14h12m39.966258723s) [upload=14632]: Uploading segments... Aug 7 23:52:18.222371001 (14h12m39.966427854s) [upload=14632]: Got next segment. Inline: false Aug 7 23:52:18.222437250 (14h12m39.966494102s) [upload=14632]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:18.222442846 (14h12m39.966499706s) [upload=14632]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:53:17.949653879 (14h13m39.693710745s) [upload=14632]: Next segment err: context deadline exceeded Aug 7 23:53:17.949857332 (14h13m39.693914187s) [upload=14632]: Done uploading segments... Aug 7 23:53:17.949878048 (14h13m39.6939349s) [upload=14632]: Upload finished. err: context deadline exceeded Aug 7 23:53:18.377163292 (14h13m40.121220142s) [upload=14651]: Starting upload Aug 7 23:53:18.377258643 (14h13m40.121315509s) [upload=14651]: Done starting upload Aug 7 23:53:18.377377813 (14h13m40.121434664s) [upload=14651]: Uploading segments... Aug 7 23:53:18.377644949 (14h13m40.121701821s) [upload=14651]: Got next segment. Inline: false Aug 7 23:53:18.377703340 (14h13m40.121760202s) [upload=14651]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:18.377710506 (14h13m40.12176737s) [upload=14651]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:54:18.110025843 (14h14m39.854082712s) [upload=14651]: Next segment err: context deadline exceeded Aug 7 23:54:18.110107720 (14h14m39.854164579s) [upload=14651]: Done uploading segments... Aug 7 23:54:18.110139157 (14h14m39.854196008s) [upload=14651]: Upload finished. err: context deadline exceeded Duration: 1m0.000990386s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.......XXXXXX.XX........................................X.XX] duration: 30.001472118s FAIL (2025-08-07 23:54:16 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 30.001503527s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 8 out of 60 is failed (threshold=3) [..XX..XXXXX................................................X] duration: 1m0.001222255s FAIL (2025-08-07 23:54:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 13061s) [upload=14531]: Done uploading segments... Aug 7 23:46:13.567267763 (14h6m35.311324613s) [upload=14531]: Upload finished. err: <nil> Aug 7 23:47:11.601739186 (14h7m33.345796037s) [upload=14546]: Starting upload Aug 7 23:47:11.601823244 (14h7m33.345880108s) [upload=14546]: Done starting upload Aug 7 23:47:11.601961578 (14h7m33.346018429s) [upload=14546]: Uploading segments... Aug 7 23:47:11.602021419 (14h7m33.346078279s) [upload=14546]: Got next segment. Inline: true Aug 7 23:47:11.602027549 (14h7m33.3460844s) [upload=14546]: Waiting for error group managing segments... Aug 7 23:47:11.602074321 (14h7m33.34613117s) [upload=14546]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:47:11.602079767 (14h7m33.346136617s) [upload=14546]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:47:11.602094800 (14h7m33.34615166s) [upload=14546]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:47:11.744532495 (14h7m33.488589355s) [upload=14546]: Done uploading segments... Aug 7 23:47:11.744570899 (14h7m33.488627759s) [upload=14546]: Upload finished. err: <nil> Aug 7 23:47:36.646321654 (14h7m58.390378505s) [upload=14555]: Starting upload Aug 7 23:47:36.646397959 (14h7m58.390454819s) [upload=14555]: Done starting upload Aug 7 23:47:36.646478843 (14h7m58.390535692s) [upload=14555]: Uploading segments... Aug 7 23:47:36.646531314 (14h7m58.390588184s) [upload=14555]: Got next segment. Inline: true Aug 7 23:47:36.646537170 (14h7m58.390594035s) [upload=14555]: Waiting for error group managing segments... Aug 7 23:47:36.646608407 (14h7m58.390665271s) [upload=14555]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:47:36.646614964 (14h7m58.390671828s) [upload=14555]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:47:36.646619338 (14h7m58.3906762s) [upload=14555]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:47:36.788927126 (14h7m58.532983976s) [upload=14555]: Done uploading segments... Aug 7 23:47:36.788947833 (14h7m58.533004684s) [upload=14555]: Upload finished. err: <nil> Aug 7 23:48:14.273939802 (14h8m36.017996675s) [upload=14564]: Starting upload Aug 7 23:48:14.274034437 (14h8m36.018091288s) [upload=14564]: Done starting upload Aug 7 23:48:14.274143056 (14h8m36.018199915s) [upload=14564]: Uploading segments... Aug 7 23:48:14.274205362 (14h8m36.01826223s) [upload=14564]: Got next segment. Inline: true Aug 7 23:48:14.274210960 (14h8m36.018267824s) [upload=14564]: Waiting for error group managing segments... Aug 7 23:48:14.274266836 (14h8m36.018323686s) [upload=14564]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:48:14.274272451 (14h8m36.018329314s) [upload=14564]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:48:14.274276048 (14h8m36.018332908s) [upload=14564]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:48:14.410395079 (14h8m36.154451958s) [upload=14564]: Done uploading segments... Aug 7 23:48:14.410429515 (14h8m36.154486376s) [upload=14564]: Upload finished. err: <nil> Aug 7 23:48:40.673955862 (14h9m2.418012725s) [upload=14570]: Starting upload Aug 7 23:48:40.674027551 (14h9m2.418084403s) [upload=14570]: Done starting upload Aug 7 23:48:40.674101278 (14h9m2.418158129s) [upload=14570]: Uploading segments... Aug 7 23:48:40.674137993 (14h9m2.418194862s) [upload=14570]: Got next segment. Inline: true Aug 7 23:48:40.674142901 (14h9m2.418199761s) [upload=14570]: Waiting for error group managing segments... Aug 7 23:48:40.674183743 (14h9m2.418240593s) [upload=14570]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:48:40.674188816 (14h9m2.418245676s) [upload=14570]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:48:40.674192280 (14h9m2.41824914s) [upload=14570]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:48:40.820583815 (14h9m2.564640664s) [upload=14570]: Done uploading segments... Aug 7 23:48:40.820602074 (14h9m2.564658926s) [upload=14570]: Upload finished. err: <nil> Aug 7 23:49:25.777587368 (14h9m47.521644216s) [upload=14582]: Starting upload Aug 7 23:49:25.777649686 (14h9m47.521706547s) [upload=14582]: Done starting upload Aug 7 23:49:25.777703819 (14h9m47.521760666s) [upload=14582]: Uploading segments... Aug 7 23:49:25.777747152 (14h9m47.521804022s) [upload=14582]: Got next segment. Inline: true Aug 7 23:49:25.777751825 (14h9m47.521808685s) [upload=14582]: Waiting for error group managing segments... Aug 7 23:49:25.777787950 (14h9m47.521844813s) [upload=14582]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:49:25.777792452 (14h9m47.521849312s) [upload=14582]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:49:25.777795064 (14h9m47.521851924s) [upload=14582]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:49:25.918566855 (14h9m47.662623707s) [upload=14582]: Done uploading segments... Aug 7 23:49:25.918602989 (14h9m47.66265984s) [upload=14582]: Upload finished. err: <nil> Aug 7 23:50:24.097841430 (14h10m45.841898277s) [upload=14602]: Starting upload Aug 7 23:50:24.097929459 (14h10m45.841986306s) [upload=14602]: Done starting upload Aug 7 23:50:24.097991265 (14h10m45.842048125s) [upload=14602]: Uploading segments... Aug 7 23:50:24.098029893 (14h10m45.842086762s) [upload=14602]: Got next segment. Inline: true Aug 7 23:50:24.098034701 (14h10m45.84209156s) [upload=14602]: Waiting for error group managing segments... Aug 7 23:50:24.098071586 (14h10m45.842128432s) [upload=14602]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:24.098076409 (14h10m45.842133268s) [upload=14602]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:24.098079284 (14h10m45.842136144s) [upload=14602]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:24.238728801 (14h10m45.982785662s) [upload=14602]: Done uploading segments... Aug 7 23:50:24.238753595 (14h10m45.982810442s) [upload=14602]: Upload finished. err: <nil> Aug 7 23:51:03.108197922 (14h11m24.852254772s) [upload=14612]: Starting upload Aug 7 23:51:03.108300837 (14h11m24.852357707s) [upload=14612]: Done starting upload Aug 7 23:51:03.108453294 (14h11m24.852510146s) [upload=14612]: Uploading segments... Aug 7 23:51:03.108972593 (14h11m24.853029491s) [upload=14612]: Got next segment. Inline: true Aug 7 23:51:03.108988857 (14h11m24.85304573s) [upload=14612]: Waiting for error group managing segments... Aug 7 23:51:03.109073931 (14h11m24.853130786s) [upload=14612]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:51:03.109080082 (14h11m24.853136933s) [upload=14612]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:51:03.109083894 (14h11m24.853140746s) [upload=14612]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:03.446817008 (14h11m25.190873858s) [upload=14612]: Done uploading segments... Aug 7 23:51:03.446842582 (14h11m25.190899436s) [upload=14612]: Upload finished. err: <nil> Aug 7 23:52:09.742356648 (14h12m31.486413495s) [upload=14628]: Starting upload Aug 7 23:52:09.742413553 (14h12m31.486470415s) [upload=14628]: Done starting upload Aug 7 23:52:09.742476702 (14h12m31.486533548s) [upload=14628]: Uploading segments... Aug 7 23:52:09.742511425 (14h12m31.486568294s) [upload=14628]: Got next segment. Inline: true Aug 7 23:52:09.742515305 (14h12m31.486572152s) [upload=14628]: Waiting for error group managing segments... Aug 7 23:52:09.742550700 (14h12m31.486607545s) [upload=14628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:09.742555041 (14h12m31.486611889s) [upload=14628]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:09.742557771 (14h12m31.486614632s) [upload=14628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:09.895389395 (14h12m31.639446255s) [upload=14628]: Done uploading segments... Aug 7 23:52:09.895421188 (14h12m31.639478039s) [upload=14628]: Upload finished. err: <nil> Aug 7 23:52:28.792030311 (14h12m50.536087163s) [upload=14635]: Starting upload Aug 7 23:52:28.792092644 (14h12m50.536149508s) [upload=14635]: Done starting upload Aug 7 23:52:28.792152087 (14h12m50.53620894s) [upload=14635]: Uploading segments... Aug 7 23:52:28.792190462 (14h12m50.536247332s) [upload=14635]: Got next segment. Inline: true Aug 7 23:52:28.792195687 (14h12m50.536252548s) [upload=14635]: Waiting for error group managing segments... Aug 7 23:52:28.792239958 (14h12m50.536296808s) [upload=14635]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:28.792245830 (14h12m50.536302691s) [upload=14635]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:28.792249165 (14h12m50.536306026s) [upload=14635]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:28.947698658 (14h12m50.691755509s) [upload=14635]: Done uploading segments... Aug 7 23:52:28.947717721 (14h12m50.691774571s) [upload=14635]: Upload finished. err: <nil> Aug 7 23:53:17.152926358 (14h13m38.896983229s) [upload=14650]: Starting upload Aug 7 23:53:17.153036870 (14h13m38.89709373s) [upload=14650]: Done starting upload Aug 7 23:53:17.153154678 (14h13m38.897211544s) [upload=14650]: Uploading segments... Aug 7 23:53:17.153233089 (14h13m38.897289957s) [upload=14650]: Got next segment. Inline: true Aug 7 23:53:17.153240256 (14h13m38.897297124s) [upload=14650]: Waiting for error group managing segments... Aug 7 23:53:17.153318892 (14h13m38.897375757s) [upload=14650]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:17.153325816 (14h13m38.89738268s) [upload=14650]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:17.153330453 (14h13m38.897387314s) [upload=14650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:16.875757982 (14h14m38.619814845s) [upload=14650]: Done uploading segments... Aug 7 23:54:16.875781849 (14h14m38.619838701s) [upload=14650]: Upload finished. err: metaclient: context deadline exceeded Duration: 1m0.001246102s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 332.848432ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 23:54:09 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 332.932195ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 860.408695ms FAIL (2025-08-07 23:54:03 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== pload=14121]: Done uploading segments... Aug 7 23:22:28.505845197 (13h42m50.249902044s) [upload=14121]: Upload finished. err: <nil> Aug 7 23:23:10.377483605 (13h43m32.121540458s) [upload=14132]: Starting upload Aug 7 23:23:10.377535635 (13h43m32.121592495s) [upload=14132]: Done starting upload Aug 7 23:23:10.377603797 (13h43m32.121660647s) [upload=14132]: Uploading segments... Aug 7 23:23:10.377638935 (13h43m32.121695804s) [upload=14132]: Got next segment. Inline: true Aug 7 23:23:10.377643594 (13h43m32.121700454s) [upload=14132]: Waiting for error group managing segments... Aug 7 23:23:10.377688252 (13h43m32.121745104s) [upload=14132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:23:10.377693427 (13h43m32.121750286s) [upload=14132]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:23:10.377696805 (13h43m32.121753666s) [upload=14132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:23:10.562260724 (13h43m32.306317573s) [upload=14132]: Done uploading segments... Aug 7 23:23:10.562279228 (13h43m32.306336078s) [upload=14132]: Upload finished. err: <nil> Aug 7 23:24:07.896404858 (13h44m29.640461721s) [upload=14147]: Starting upload Aug 7 23:24:07.896490216 (13h44m29.640547079s) [upload=14147]: Done starting upload Aug 7 23:24:07.896580488 (13h44m29.640637339s) [upload=14147]: Uploading segments... Aug 7 23:24:07.896655006 (13h44m29.640711874s) [upload=14147]: Got next segment. Inline: true Aug 7 23:24:07.896660179 (13h44m29.640717039s) [upload=14147]: Waiting for error group managing segments... Aug 7 23:24:07.896714840 (13h44m29.6407717s) [upload=14147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:24:07.896719399 (13h44m29.64077626s) [upload=14147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:24:07.896722726 (13h44m29.640779586s) [upload=14147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:24:08.048898130 (13h44m29.792954991s) [upload=14147]: Done uploading segments... Aug 7 23:24:08.048952526 (13h44m29.793009386s) [upload=14147]: Upload finished. err: <nil> Aug 7 23:24:53.794754379 (13h45m15.53881125s) [upload=14162]: Starting upload Aug 7 23:24:53.794842317 (13h45m15.538899177s) [upload=14162]: Done starting upload Aug 7 23:24:53.794935964 (13h45m15.538992814s) [upload=14162]: Uploading segments... Aug 7 23:24:53.795003752 (13h45m15.539060623s) [upload=14162]: Got next segment. Inline: true Aug 7 23:24:53.795010814 (13h45m15.539067676s) [upload=14162]: Waiting for error group managing segments... Aug 7 23:24:53.795066607 (13h45m15.539123488s) [upload=14162]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:24:53.795158449 (13h45m15.5392153s) [upload=14162]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:24:53.795163166 (13h45m15.539220026s) [upload=14162]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:24:53.946994982 (13h45m15.691051833s) [upload=14162]: Done uploading segments... Aug 7 23:24:53.947013862 (13h45m15.691070711s) [upload=14162]: Upload finished. err: <nil> Aug 7 23:25:40.029856966 (13h46m1.773913836s) [upload=14177]: Starting upload Aug 7 23:25:40.029986938 (13h46m1.774043809s) [upload=14177]: Done starting upload Aug 7 23:25:40.030076796 (13h46m1.774133642s) [upload=14177]: Uploading segments... Aug 7 23:25:40.030132567 (13h46m1.774189438s) [upload=14177]: Got next segment. Inline: true Aug 7 23:25:40.030139885 (13h46m1.774196757s) [upload=14177]: Waiting for error group managing segments... Aug 7 23:25:40.030197588 (13h46m1.774254461s) [upload=14177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:25:40.030204146 (13h46m1.774261016s) [upload=14177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:25:40.030207984 (13h46m1.77426485s) [upload=14177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:25:40.184198742 (13h46m1.928255592s) [upload=14177]: Done uploading segments... Aug 7 23:25:40.184219017 (13h46m1.928275867s) [upload=14177]: Upload finished. err: <nil> Aug 7 23:26:54.625649110 (13h47m16.369705971s) [upload=14198]: Starting upload Aug 7 23:26:54.625738563 (13h47m16.369795424s) [upload=14198]: Done starting upload Aug 7 23:26:54.625842691 (13h47m16.369899539s) [upload=14198]: Uploading segments... Aug 7 23:26:54.626016839 (13h47m16.37007371s) [upload=14198]: Got next segment. Inline: true Aug 7 23:26:54.626052112 (13h47m16.370108973s) [upload=14198]: Waiting for error group managing segments... Aug 7 23:26:54.626129780 (13h47m16.370186645s) [upload=14198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:26:54.626137392 (13h47m16.370194264s) [upload=14198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:26:54.626143182 (13h47m16.370200048s) [upload=14198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:26:54.775804171 (13h47m16.519861021s) [upload=14198]: Done uploading segments... Aug 7 23:26:54.775824249 (13h47m16.519881102s) [upload=14198]: Upload finished. err: <nil> Aug 7 23:27:32.263746490 (13h47m54.007803361s) [upload=14208]: Starting upload Aug 7 23:27:32.263833655 (13h47m54.007890501s) [upload=14208]: Done starting upload Aug 7 23:27:32.263976536 (13h47m54.008033385s) [upload=14208]: Uploading segments... Aug 7 23:27:32.264070926 (13h47m54.008127796s) [upload=14208]: Got next segment. Inline: true Aug 7 23:27:32.264080356 (13h47m54.00813733s) [upload=14208]: Waiting for error group managing segments... Aug 7 23:27:32.264154657 (13h47m54.008211526s) [upload=14208]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:27:32.264163587 (13h47m54.008220454s) [upload=14208]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:27:32.264172017 (13h47m54.008228914s) [upload=14208]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:27:32.490188559 (13h47m54.23424541s) [upload=14208]: Done uploading segments... Aug 7 23:27:32.490207326 (13h47m54.234264172s) [upload=14208]: Upload finished. err: <nil> Aug 7 23:27:57.927940477 (13h48m19.671997323s) [upload=14217]: Starting upload Aug 7 23:27:57.928016423 (13h48m19.672073283s) [upload=14217]: Done starting upload Aug 7 23:27:57.928101290 (13h48m19.672158142s) [upload=14217]: Uploading segments... Aug 7 23:27:57.928155692 (13h48m19.672212566s) [upload=14217]: Got next segment. Inline: true Aug 7 23:27:57.928162009 (13h48m19.672218869s) [upload=14217]: Waiting for error group managing segments... Aug 7 23:27:57.928213294 (13h48m19.672270145s) [upload=14217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:27:57.928219212 (13h48m19.672276074s) [upload=14217]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:27:57.928223521 (13h48m19.672280383s) [upload=14217]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:27:58.087544005 (13h48m19.831600874s) [upload=14217]: Done uploading segments... Aug 7 23:27:58.087570226 (13h48m19.831627094s) [upload=14217]: Upload finished. err: <nil> Aug 7 23:29:12.765200935 (13h49m34.509257795s) [upload=14237]: Starting upload Aug 7 23:29:12.765253106 (13h49m34.509309977s) [upload=14237]: Done starting upload Aug 7 23:29:12.765324641 (13h49m34.509381499s) [upload=14237]: Uploading segments... Aug 7 23:29:12.765374187 (13h49m34.509431056s) [upload=14237]: Got next segment. Inline: true Aug 7 23:29:12.765378947 (13h49m34.509435807s) [upload=14237]: Waiting for error group managing segments... Aug 7 23:29:12.765430859 (13h49m34.50948772s) [upload=14237]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:29:12.765438106 (13h49m34.50949497s) [upload=14237]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:29:12.765443127 (13h49m34.509499988s) [upload=14237]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:29:12.926437674 (13h49m34.670494535s) [upload=14237]: Done uploading segments... Aug 7 23:29:12.926457418 (13h49m34.670514268s) [upload=14237]: Upload finished. err: <nil> Aug 7 23:29:42.225760316 (13h50m3.969817177s) [upload=14244]: Starting upload Aug 7 23:29:42.225855063 (13h50m3.969911929s) [upload=14244]: Done starting upload Aug 7 23:29:42.225962489 (13h50m3.970019355s) [upload=14244]: Uploading segments... Aug 7 23:29:42.226078484 (13h50m3.970135331s) [upload=14244]: Got next segment. Inline: true Aug 7 23:29:42.226083996 (13h50m3.970140842s) [upload=14244]: Waiting for error group managing segments... Aug 7 23:29:42.226143067 (13h50m3.970199914s) [upload=14244]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:29:42.226148511 (13h50m3.970205371s) [upload=14244]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:29:42.226153006 (13h50m3.970209865s) [upload=14244]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:29:42.401822947 (13h50m4.14587981s) [upload=14244]: Done uploading segments... Aug 7 23:29:42.401847556 (13h50m4.145904405s) [upload=14244]: Upload finished. err: <nil> Aug 7 23:30:31.529840301 (13h50m53.273897165s) [upload=14254]: Starting upload Aug 7 23:30:31.530002531 (13h50m53.274059378s) [upload=14254]: Done starting upload Aug 7 23:30:31.530201544 (13h50m53.274258391s) [upload=14254]: Uploading segments... Aug 7 23:30:31.530298120 (13h50m53.274354985s) [upload=14254]: Got next segment. Inline: true Aug 7 23:30:31.530303457 (13h50m53.274360317s) [upload=14254]: Waiting for error group managing segments... Aug 7 23:30:31.530353721 (13h50m53.274410567s) [upload=14254]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:30:31.530358927 (13h50m53.274415788s) [upload=14254]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:30:31.530363758 (13h50m53.274420619s) [upload=14254]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:30:31.696350967 (13h50m53.440407828s) [upload=14254]: Done uploading segments... Aug 7 23:30:31.696369460 (13h50m53.440426307s) [upload=14254]: Upload finished. err: <nil> Duration: 860.440005ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 11 out of 60 is failed (threshold=3) [....XXXXX.XX.XX.X.........................................X.] duration: 873.432887ms FAIL (2025-08-07 23:53:59 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 873.464374ms core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....X.XX..X................................................] duration: 2.754569641s FAIL (2025-08-07 23:53:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.754599732s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 10 out of 60 is failed (threshold=3) [........XXXXXX.XX........................................X.X] duration: 30.000631195s FAIL (2025-08-07 23:53:46 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 30.000653207s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [.....XXXXXXXXXX...........X............................XX.X.] duration: 2.440800609s FAIL (2025-08-07 23:53:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== g 7 23:52:06.811973313 (14h12m28.556030174s) [upload=14627]: Done starting upload Aug 7 23:52:06.812083421 (14h12m28.556140271s) [upload=14627]: Uploading segments... Aug 7 23:52:06.812257002 (14h12m28.556313863s) [upload=14627]: Got next segment. Inline: false Aug 7 23:52:06.812301439 (14h12m28.55635829s) [upload=14627]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:06.812307735 (14h12m28.556364595s) [upload=14627]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:52:06.922936130 (14h12m28.666992991s) [seg_pos={0 0}, upload=14627]: Begin upload segment... Aug 7 23:52:06.922998276 (14h12m28.667055137s) [seg_pos={0 0}, upload=14627]: Done begin upload segment. Aug 7 23:52:06.923158618 (14h12m28.667215481s) [noise=true , address=104.36.82.10:4077 , piece_id=ZQC75CI4UIW34VEFEF244HOVYW3MHYBDIG7DIIFV5TCVISVOZQKQ, seg_pos={0 0}, upload=14627]: Uploading piece... Aug 7 23:52:06.923173122 (14h12m28.667229982s) [noise=true , address=45.91.116.5:4297 , piece_id=FWOCFXYRESTVS7TRKU7TJSO7PY3WKMQUR5NXH2M3TAEE5OWGKNCQ, seg_pos={0 0}, upload=14627]: Uploading piece... Aug 7 23:52:06.923253658 (14h12m28.667310517s) [noise=true , address=77.90.31.15:4699 , piece_id=OYD2XFHFSJBXJYQKPNEFEJJGLIW7KYJLAERPQDNAY5NBIUJJTXBQ, seg_pos={0 0}, upload=14627]: Uploading piece... Aug 7 23:52:06.923528322 (14h12m28.667585183s) [noise=true , address=45.91.116.6:4974 , piece_id=7KMWWVS624W7GVSR6ARHY4RQ7KFJLEQ2NOZTP7UAC7PXEVRGGREA, seg_pos={0 0}, upload=14627]: Uploading piece... Aug 7 23:52:06.923756600 (14h12m28.66781347s) [noise=true , address=104.36.82.2:4729 , piece_id=BQXAZEYWDHOIFXDXW3OUGPYK7RW6NAKNYPRTCNLVVDB4PH64QAVA, seg_pos={0 0}, upload=14627]: Uploading piece... Aug 7 23:52:06.923788647 (14h12m28.667845513s) [noise=true , address=45.91.116.4:4940 , piece_id=IWS6FRSCHNLSDAFISCYMIV5LM3OVERU27FJHYU3KGGKJZAHZF5WQ, seg_pos={0 0}, upload=14627]: Uploading piece... Aug 7 23:52:06.924231963 (14h12m28.668288828s) [noise=true , address=104.36.82.7:4910 , piece_id=34GAU3RFDGQA3EQFIVDJYKPH6Y4BB5UTGGO6HJKKSIESWCMIGMAQ, seg_pos={0 0}, upload=14627]: Uploading piece... Aug 7 23:52:06.924685993 (14h12m28.668742858s) [noise=true , address=104.36.82.13:4185 , piece_id=LSAULRBDUSHO6GQNSUSERGDAJ4KJKWCZNJZ2ZH7KAGOE3FP7STUQ, seg_pos={0 0}, upload=14627]: Uploading piece... Aug 7 23:52:06.924979041 (14h12m28.669035902s) [noise=true , address=104.36.82.14:4223 , piece_id=2X23HJIHMDOTJFW2KSUDQCCPOLFRR2YVLO5GP2HOF6BPQMOH6OLA, seg_pos={0 0}, upload=14627]: Uploading piece... Aug 7 23:52:06.925246280 (14h12m28.669303142s) [noise=true , address=104.36.82.12:4150 , piece_id=MGZSW3YOC4KLJEE6MXNLDO5L5XAFGCNKHCPX22VZTNM7XA33AZ2A, seg_pos={0 0}, upload=14627]: Uploading piece... Aug 7 23:52:06.925527713 (14h12m28.669584573s) [noise=true , address=77.90.31.18:4806 , piece_id=EVVXK2VDTIKJBQZE6QOEXWKOU3ZMWQZFUOMLVCU6NXUAYH7YONJQ, seg_pos={0 0}, upload=14627]: Uploading piece... Aug 7 23:52:06.925827355 (14h12m28.669884216s) [noise=true , address=45.91.116.14:4191 , piece_id=42ZVMTDP5G3SKBL3ILW5SMSD7HPWYJ5EPA64S3Q3FFL4WBWA3NYA, seg_pos={0 0}, upload=14627]: Uploading piece... Aug 7 23:52:06.926044129 (14h12m28.670100992s) [noise=true , address=45.91.116.5:4299 , piece_id=GKNZNJKKVDGRP3JIXFMCJTPOQC5JOVS2CUL5HA5R64EBB2WICNXQ, seg_pos={0 0}, upload=14627]: Uploading piece... Aug 7 23:52:06.926384675 (14h12m28.670441543s) [noise=true , address=77.90.31.11:4555 , piece_id=HTMEN2TFNYR6SX5A4HA25W3PT65EYHTMQ62TLUBHDQEXPIIKXYPQ, seg_pos={0 0}, upload=14627]: Uploading piece... Aug 7 23:52:06.926664564 (14h12m28.670721424s) [noise=true , address=77.90.31.13:4627 , piece_id=VH42KWREONRGCKMXBH5A2B67Q5B4VNOWHTY5Y2GAGSH6PTT6TIWQ, seg_pos={0 0}, upload=14627]: Uploading piece... Aug 7 23:52:06.926965510 (14h12m28.671022377s) [noise=true , address=45.91.116.14:4190 , piece_id=VHSJB7CNUIQUWFXC66NTPNGMWXGBQP5OCVRMZKHTCEE5OPRKCJBQ, seg_pos={0 0}, upload=14627]: Uploading piece... Aug 7 23:52:07.591896131 (14h12m29.335952981s) [upload=14627]: Next returned nil segment Aug 7 23:52:07.591911929 (14h12m29.33596878s) [upload=14627]: Waiting for error group managing segments... Aug 7 23:52:07.701736559 (14h12m29.445793406s) [noise=true , address=104.36.82.2:4729 , piece_id=BQXAZEYWDHOIFXDXW3OUGPYK7RW6NAKNYPRTCNLVVDB4PH64QAVA, seg_pos={0 0}, upload=14627]: Done uploading piece. err: <nil> Aug 7 23:52:07.717615872 (14h12m29.461672724s) [noise=true , address=104.36.82.12:4150 , piece_id=MGZSW3YOC4KLJEE6MXNLDO5L5XAFGCNKHCPX22VZTNM7XA33AZ2A, seg_pos={0 0}, upload=14627]: Done uploading piece. err: <nil> Aug 7 23:52:07.718025677 (14h12m29.462082537s) [noise=true , address=104.36.82.14:4223 , piece_id=2X23HJIHMDOTJFW2KSUDQCCPOLFRR2YVLO5GP2HOF6BPQMOH6OLA, seg_pos={0 0}, upload=14627]: Done uploading piece. err: <nil> Aug 7 23:52:07.728916644 (14h12m29.472973506s) [noise=true , address=104.36.82.7:4910 , piece_id=34GAU3RFDGQA3EQFIVDJYKPH6Y4BB5UTGGO6HJKKSIESWCMIGMAQ, seg_pos={0 0}, upload=14627]: Done uploading piece. err: <nil> Aug 7 23:52:07.730391318 (14h12m29.474448165s) [noise=true , address=77.90.31.18:4806 , piece_id=EVVXK2VDTIKJBQZE6QOEXWKOU3ZMWQZFUOMLVCU6NXUAYH7YONJQ, seg_pos={0 0}, upload=14627]: Done uploading piece. err: <nil> Aug 7 23:52:07.743509212 (14h12m29.487566062s) [noise=true , address=104.36.82.10:4077 , piece_id=ZQC75CI4UIW34VEFEF244HOVYW3MHYBDIG7DIIFV5TCVISVOZQKQ, seg_pos={0 0}, upload=14627]: Done uploading piece. err: <nil> Aug 7 23:52:07.809677669 (14h12m29.553734518s) [noise=true , address=77.90.31.15:4699 , piece_id=OYD2XFHFSJBXJYQKPNEFEJJGLIW7KYJLAERPQDNAY5NBIUJJTXBQ, seg_pos={0 0}, upload=14627]: Done uploading piece. err: <nil> Aug 7 23:52:07.811147179 (14h12m29.555204028s) [noise=true , address=77.90.31.13:4627 , piece_id=VH42KWREONRGCKMXBH5A2B67Q5B4VNOWHTY5Y2GAGSH6PTT6TIWQ, seg_pos={0 0}, upload=14627]: Done uploading piece. err: <nil> Aug 7 23:52:07.817302636 (14h12m29.56135949s) [noise=true , address=45.91.116.5:4297 , piece_id=FWOCFXYRESTVS7TRKU7TJSO7PY3WKMQUR5NXH2M3TAEE5OWGKNCQ, seg_pos={0 0}, upload=14627]: Done uploading piece. err: <nil> Aug 7 23:52:07.819957158 (14h12m29.564014008s) [noise=true , address=45.91.116.4:4940 , piece_id=IWS6FRSCHNLSDAFISCYMIV5LM3OVERU27FJHYU3KGGKJZAHZF5WQ, seg_pos={0 0}, upload=14627]: Done uploading piece. err: <nil> Aug 7 23:52:07.874140462 (14h12m29.618197308s) [noise=true , address=77.90.31.11:4555 , piece_id=HTMEN2TFNYR6SX5A4HA25W3PT65EYHTMQ62TLUBHDQEXPIIKXYPQ, seg_pos={0 0}, upload=14627]: Done uploading piece. err: <nil> Aug 7 23:52:07.888812964 (14h12m29.632869815s) [noise=true , address=45.91.116.14:4191 , piece_id=42ZVMTDP5G3SKBL3ILW5SMSD7HPWYJ5EPA64S3Q3FFL4WBWA3NYA, seg_pos={0 0}, upload=14627]: Done uploading piece. err: <nil> Aug 7 23:52:07.904458238 (14h12m29.648515099s) [noise=true , address=45.91.116.6:4974 , piece_id=7KMWWVS624W7GVSR6ARHY4RQ7KFJLEQ2NOZTP7UAC7PXEVRGGREA, seg_pos={0 0}, upload=14627]: Done uploading piece. err: <nil> Aug 7 23:52:07.904516512 (14h12m29.648573374s) [seg_pos={0 0}, upload=14627]: Segment reached optimal threshold of 13 pieces. Aug 7 23:52:07.904863779 (14h12m29.648920642s) [noise=true , address=104.36.82.13:4185 , piece_id=LSAULRBDUSHO6GQNSUSERGDAJ4KJKWCZNJZ2ZH7KAGOE3FP7STUQ, seg_pos={0 0}, upload=14627]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: write tcp 139.59.144.22:58536->104.36.82.13:4185: use of closed network connection; write tcp 139.59.144.22:58536->104.36.82.13:4185: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58536->104.36.82.13:4185: use of closed network connection Aug 7 23:52:07.905012358 (14h12m29.649069217s) [noise=true , address=45.91.116.5:4299 , piece_id=GKNZNJKKVDGRP3JIXFMCJTPOQC5JOVS2CUL5HA5R64EBB2WICNXQ, seg_pos={0 0}, upload=14627]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: expected piece hash; context canceled; EOF Aug 7 23:52:07.905250111 (14h12m29.649306973s) [noise=true , address=45.91.116.14:4190 , piece_id=VHSJB7CNUIQUWFXC66NTPNGMWXGBQP5OCVRMZKHTCEE5OPRKCJBQ, seg_pos={0 0}, upload=14627]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF Aug 7 23:52:07.905278216 (14h12m29.649335083s) [seg_pos={0 0}, upload=14627]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:52:07.905420431 (14h12m29.649477299s) [upload=14627]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:52:07.905427413 (14h12m29.649484283s) [upload=14627]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:08.050540094 (14h12m29.794596945s) [upload=14627]: Done uploading segments... Aug 7 23:52:08.050560106 (14h12m29.794616959s) [upload=14627]: Upload finished. err: <nil> Aug 7 23:53:13.437055603 (14h13m35.181112449s) [upload=14648]: Starting upload Aug 7 23:53:13.437115756 (14h13m35.181172619s) [upload=14648]: Done starting upload Aug 7 23:53:13.437176696 (14h13m35.181233545s) [upload=14648]: Uploading segments... Aug 7 23:53:13.437358088 (14h13m35.181414934s) [upload=14648]: Got next segment. Inline: false Aug 7 23:53:13.437394976 (14h13m35.181451837s) [upload=14648]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:13.437399508 (14h13m35.181456367s) [upload=14648]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:53:43.167542607 (14h14m4.911599467s) [upload=14648]: Next segment err: context deadline exceeded Aug 7 23:53:43.167705717 (14h14m4.911762564s) [upload=14648]: Done uploading segments... Aug 7 23:53:43.167737922 (14h14m4.911794768s) [upload=14648]: Upload finished. err: context deadline exceeded Duration: 2.440825083s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [......XXXXXXXXXX...........X............................XX.X] duration: 30.000569717s FAIL (2025-08-07 23:53:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== g 7 23:52:06.811973313 (14h12m28.556030174s) [upload=14627]: Done starting upload Aug 7 23:52:06.812083421 (14h12m28.556140271s) [upload=14627]: Uploading segments... Aug 7 23:52:06.812257002 (14h12m28.556313863s) [upload=14627]: Got next segment. Inline: false Aug 7 23:52:06.812301439 (14h12m28.55635829s) [upload=14627]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:06.812307735 (14h12m28.556364595s) [upload=14627]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:52:06.922936130 (14h12m28.666992991s) [seg_pos={0 0}, upload=14627]: Begin upload segment... Aug 7 23:52:06.922998276 (14h12m28.667055137s) [seg_pos={0 0}, upload=14627]: Done begin upload segment. Aug 7 23:52:06.923158618 (14h12m28.667215481s) [noise=true , address=104.36.82.10:4077 , piece_id=ZQC75CI4UIW34VEFEF244HOVYW3MHYBDIG7DIIFV5TCVISVOZQKQ, seg_pos={0 0}, upload=14627]: Uploading piece... Aug 7 23:52:06.923173122 (14h12m28.667229982s) [noise=true , address=45.91.116.5:4297 , piece_id=FWOCFXYRESTVS7TRKU7TJSO7PY3WKMQUR5NXH2M3TAEE5OWGKNCQ, seg_pos={0 0}, upload=14627]: Uploading piece... Aug 7 23:52:06.923253658 (14h12m28.667310517s) [noise=true , address=77.90.31.15:4699 , piece_id=OYD2XFHFSJBXJYQKPNEFEJJGLIW7KYJLAERPQDNAY5NBIUJJTXBQ, seg_pos={0 0}, upload=14627]: Uploading piece... Aug 7 23:52:06.923528322 (14h12m28.667585183s) [noise=true , address=45.91.116.6:4974 , piece_id=7KMWWVS624W7GVSR6ARHY4RQ7KFJLEQ2NOZTP7UAC7PXEVRGGREA, seg_pos={0 0}, upload=14627]: Uploading piece... Aug 7 23:52:06.923756600 (14h12m28.66781347s) [noise=true , address=104.36.82.2:4729 , piece_id=BQXAZEYWDHOIFXDXW3OUGPYK7RW6NAKNYPRTCNLVVDB4PH64QAVA, seg_pos={0 0}, upload=14627]: Uploading piece... Aug 7 23:52:06.923788647 (14h12m28.667845513s) [noise=true , address=45.91.116.4:4940 , piece_id=IWS6FRSCHNLSDAFISCYMIV5LM3OVERU27FJHYU3KGGKJZAHZF5WQ, seg_pos={0 0}, upload=14627]: Uploading piece... Aug 7 23:52:06.924231963 (14h12m28.668288828s) [noise=true , address=104.36.82.7:4910 , piece_id=34GAU3RFDGQA3EQFIVDJYKPH6Y4BB5UTGGO6HJKKSIESWCMIGMAQ, seg_pos={0 0}, upload=14627]: Uploading piece... Aug 7 23:52:06.924685993 (14h12m28.668742858s) [noise=true , address=104.36.82.13:4185 , piece_id=LSAULRBDUSHO6GQNSUSERGDAJ4KJKWCZNJZ2ZH7KAGOE3FP7STUQ, seg_pos={0 0}, upload=14627]: Uploading piece... Aug 7 23:52:06.924979041 (14h12m28.669035902s) [noise=true , address=104.36.82.14:4223 , piece_id=2X23HJIHMDOTJFW2KSUDQCCPOLFRR2YVLO5GP2HOF6BPQMOH6OLA, seg_pos={0 0}, upload=14627]: Uploading piece... Aug 7 23:52:06.925246280 (14h12m28.669303142s) [noise=true , address=104.36.82.12:4150 , piece_id=MGZSW3YOC4KLJEE6MXNLDO5L5XAFGCNKHCPX22VZTNM7XA33AZ2A, seg_pos={0 0}, upload=14627]: Uploading piece... Aug 7 23:52:06.925527713 (14h12m28.669584573s) [noise=true , address=77.90.31.18:4806 , piece_id=EVVXK2VDTIKJBQZE6QOEXWKOU3ZMWQZFUOMLVCU6NXUAYH7YONJQ, seg_pos={0 0}, upload=14627]: Uploading piece... Aug 7 23:52:06.925827355 (14h12m28.669884216s) [noise=true , address=45.91.116.14:4191 , piece_id=42ZVMTDP5G3SKBL3ILW5SMSD7HPWYJ5EPA64S3Q3FFL4WBWA3NYA, seg_pos={0 0}, upload=14627]: Uploading piece... Aug 7 23:52:06.926044129 (14h12m28.670100992s) [noise=true , address=45.91.116.5:4299 , piece_id=GKNZNJKKVDGRP3JIXFMCJTPOQC5JOVS2CUL5HA5R64EBB2WICNXQ, seg_pos={0 0}, upload=14627]: Uploading piece... Aug 7 23:52:06.926384675 (14h12m28.670441543s) [noise=true , address=77.90.31.11:4555 , piece_id=HTMEN2TFNYR6SX5A4HA25W3PT65EYHTMQ62TLUBHDQEXPIIKXYPQ, seg_pos={0 0}, upload=14627]: Uploading piece... Aug 7 23:52:06.926664564 (14h12m28.670721424s) [noise=true , address=77.90.31.13:4627 , piece_id=VH42KWREONRGCKMXBH5A2B67Q5B4VNOWHTY5Y2GAGSH6PTT6TIWQ, seg_pos={0 0}, upload=14627]: Uploading piece... Aug 7 23:52:06.926965510 (14h12m28.671022377s) [noise=true , address=45.91.116.14:4190 , piece_id=VHSJB7CNUIQUWFXC66NTPNGMWXGBQP5OCVRMZKHTCEE5OPRKCJBQ, seg_pos={0 0}, upload=14627]: Uploading piece... Aug 7 23:52:07.591896131 (14h12m29.335952981s) [upload=14627]: Next returned nil segment Aug 7 23:52:07.591911929 (14h12m29.33596878s) [upload=14627]: Waiting for error group managing segments... Aug 7 23:52:07.701736559 (14h12m29.445793406s) [noise=true , address=104.36.82.2:4729 , piece_id=BQXAZEYWDHOIFXDXW3OUGPYK7RW6NAKNYPRTCNLVVDB4PH64QAVA, seg_pos={0 0}, upload=14627]: Done uploading piece. err: <nil> Aug 7 23:52:07.717615872 (14h12m29.461672724s) [noise=true , address=104.36.82.12:4150 , piece_id=MGZSW3YOC4KLJEE6MXNLDO5L5XAFGCNKHCPX22VZTNM7XA33AZ2A, seg_pos={0 0}, upload=14627]: Done uploading piece. err: <nil> Aug 7 23:52:07.718025677 (14h12m29.462082537s) [noise=true , address=104.36.82.14:4223 , piece_id=2X23HJIHMDOTJFW2KSUDQCCPOLFRR2YVLO5GP2HOF6BPQMOH6OLA, seg_pos={0 0}, upload=14627]: Done uploading piece. err: <nil> Aug 7 23:52:07.728916644 (14h12m29.472973506s) [noise=true , address=104.36.82.7:4910 , piece_id=34GAU3RFDGQA3EQFIVDJYKPH6Y4BB5UTGGO6HJKKSIESWCMIGMAQ, seg_pos={0 0}, upload=14627]: Done uploading piece. err: <nil> Aug 7 23:52:07.730391318 (14h12m29.474448165s) [noise=true , address=77.90.31.18:4806 , piece_id=EVVXK2VDTIKJBQZE6QOEXWKOU3ZMWQZFUOMLVCU6NXUAYH7YONJQ, seg_pos={0 0}, upload=14627]: Done uploading piece. err: <nil> Aug 7 23:52:07.743509212 (14h12m29.487566062s) [noise=true , address=104.36.82.10:4077 , piece_id=ZQC75CI4UIW34VEFEF244HOVYW3MHYBDIG7DIIFV5TCVISVOZQKQ, seg_pos={0 0}, upload=14627]: Done uploading piece. err: <nil> Aug 7 23:52:07.809677669 (14h12m29.553734518s) [noise=true , address=77.90.31.15:4699 , piece_id=OYD2XFHFSJBXJYQKPNEFEJJGLIW7KYJLAERPQDNAY5NBIUJJTXBQ, seg_pos={0 0}, upload=14627]: Done uploading piece. err: <nil> Aug 7 23:52:07.811147179 (14h12m29.555204028s) [noise=true , address=77.90.31.13:4627 , piece_id=VH42KWREONRGCKMXBH5A2B67Q5B4VNOWHTY5Y2GAGSH6PTT6TIWQ, seg_pos={0 0}, upload=14627]: Done uploading piece. err: <nil> Aug 7 23:52:07.817302636 (14h12m29.56135949s) [noise=true , address=45.91.116.5:4297 , piece_id=FWOCFXYRESTVS7TRKU7TJSO7PY3WKMQUR5NXH2M3TAEE5OWGKNCQ, seg_pos={0 0}, upload=14627]: Done uploading piece. err: <nil> Aug 7 23:52:07.819957158 (14h12m29.564014008s) [noise=true , address=45.91.116.4:4940 , piece_id=IWS6FRSCHNLSDAFISCYMIV5LM3OVERU27FJHYU3KGGKJZAHZF5WQ, seg_pos={0 0}, upload=14627]: Done uploading piece. err: <nil> Aug 7 23:52:07.874140462 (14h12m29.618197308s) [noise=true , address=77.90.31.11:4555 , piece_id=HTMEN2TFNYR6SX5A4HA25W3PT65EYHTMQ62TLUBHDQEXPIIKXYPQ, seg_pos={0 0}, upload=14627]: Done uploading piece. err: <nil> Aug 7 23:52:07.888812964 (14h12m29.632869815s) [noise=true , address=45.91.116.14:4191 , piece_id=42ZVMTDP5G3SKBL3ILW5SMSD7HPWYJ5EPA64S3Q3FFL4WBWA3NYA, seg_pos={0 0}, upload=14627]: Done uploading piece. err: <nil> Aug 7 23:52:07.904458238 (14h12m29.648515099s) [noise=true , address=45.91.116.6:4974 , piece_id=7KMWWVS624W7GVSR6ARHY4RQ7KFJLEQ2NOZTP7UAC7PXEVRGGREA, seg_pos={0 0}, upload=14627]: Done uploading piece. err: <nil> Aug 7 23:52:07.904516512 (14h12m29.648573374s) [seg_pos={0 0}, upload=14627]: Segment reached optimal threshold of 13 pieces. Aug 7 23:52:07.904863779 (14h12m29.648920642s) [noise=true , address=104.36.82.13:4185 , piece_id=LSAULRBDUSHO6GQNSUSERGDAJ4KJKWCZNJZ2ZH7KAGOE3FP7STUQ, seg_pos={0 0}, upload=14627]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: write tcp 139.59.144.22:58536->104.36.82.13:4185: use of closed network connection; write tcp 139.59.144.22:58536->104.36.82.13:4185: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58536->104.36.82.13:4185: use of closed network connection Aug 7 23:52:07.905012358 (14h12m29.649069217s) [noise=true , address=45.91.116.5:4299 , piece_id=GKNZNJKKVDGRP3JIXFMCJTPOQC5JOVS2CUL5HA5R64EBB2WICNXQ, seg_pos={0 0}, upload=14627]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: expected piece hash; context canceled; EOF Aug 7 23:52:07.905250111 (14h12m29.649306973s) [noise=true , address=45.91.116.14:4190 , piece_id=VHSJB7CNUIQUWFXC66NTPNGMWXGBQP5OCVRMZKHTCEE5OPRKCJBQ, seg_pos={0 0}, upload=14627]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF Aug 7 23:52:07.905278216 (14h12m29.649335083s) [seg_pos={0 0}, upload=14627]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:52:07.905420431 (14h12m29.649477299s) [upload=14627]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:52:07.905427413 (14h12m29.649484283s) [upload=14627]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:08.050540094 (14h12m29.794596945s) [upload=14627]: Done uploading segments... Aug 7 23:52:08.050560106 (14h12m29.794616959s) [upload=14627]: Upload finished. err: <nil> Aug 7 23:53:13.437055603 (14h13m35.181112449s) [upload=14648]: Starting upload Aug 7 23:53:13.437115756 (14h13m35.181172619s) [upload=14648]: Done starting upload Aug 7 23:53:13.437176696 (14h13m35.181233545s) [upload=14648]: Uploading segments... Aug 7 23:53:13.437358088 (14h13m35.181414934s) [upload=14648]: Got next segment. Inline: false Aug 7 23:53:13.437394976 (14h13m35.181451837s) [upload=14648]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:13.437399508 (14h13m35.181456367s) [upload=14648]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:53:43.167542607 (14h14m4.911599467s) [upload=14648]: Next segment err: context deadline exceeded Aug 7 23:53:43.167705717 (14h14m4.911762564s) [upload=14648]: Done uploading segments... Aug 7 23:53:43.167737922 (14h14m4.911794768s) [upload=14648]: Upload finished. err: context deadline exceeded Duration: 30.000586721s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 14 out of 60 is failed (threshold=3) [....XXX.XXXXXX..XX.X...................................X...X] duration: 30.000897165s FAIL (2025-08-07 23:53:40 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== s) [upload=14528]: Done uploading segments... Aug 7 23:46:09.155664178 (14h6m30.899721028s) [upload=14528]: Upload finished. err: <nil> Aug 7 23:46:47.072912780 (14h7m8.816969628s) [upload=14538]: Starting upload Aug 7 23:46:47.072990499 (14h7m8.817047359s) [upload=14538]: Done starting upload Aug 7 23:46:47.073066127 (14h7m8.817122973s) [upload=14538]: Uploading segments... Aug 7 23:46:47.073122515 (14h7m8.817179381s) [upload=14538]: Got next segment. Inline: true Aug 7 23:46:47.073127584 (14h7m8.817184447s) [upload=14538]: Waiting for error group managing segments... Aug 7 23:46:47.073168839 (14h7m8.817225702s) [upload=14538]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:46:47.073173325 (14h7m8.817230185s) [upload=14538]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:46:47.073176118 (14h7m8.817232978s) [upload=14538]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:46:47.210119395 (14h7m8.954176244s) [upload=14538]: Done uploading segments... Aug 7 23:46:47.210142995 (14h7m8.954199841s) [upload=14538]: Upload finished. err: <nil> Aug 7 23:47:29.298549723 (14h7m51.042606575s) [upload=14552]: Starting upload Aug 7 23:47:29.298617898 (14h7m51.042674767s) [upload=14552]: Done starting upload Aug 7 23:47:29.298698678 (14h7m51.042755528s) [upload=14552]: Uploading segments... Aug 7 23:47:29.298753306 (14h7m51.042810175s) [upload=14552]: Got next segment. Inline: true Aug 7 23:47:29.298758297 (14h7m51.042815158s) [upload=14552]: Waiting for error group managing segments... Aug 7 23:47:29.298818439 (14h7m51.042875289s) [upload=14552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:47:29.298823844 (14h7m51.042880694s) [upload=14552]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:47:29.298827075 (14h7m51.042883925s) [upload=14552]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:47:29.457296385 (14h7m51.201353233s) [upload=14552]: Done uploading segments... Aug 7 23:47:29.457315555 (14h7m51.201372401s) [upload=14552]: Upload finished. err: <nil> Aug 7 23:48:42.140972885 (14h9m3.885029736s) [upload=14571]: Starting upload Aug 7 23:48:42.141036695 (14h9m3.885093558s) [upload=14571]: Done starting upload Aug 7 23:48:42.141117369 (14h9m3.88517422s) [upload=14571]: Uploading segments... Aug 7 23:48:42.141203219 (14h9m3.885260088s) [upload=14571]: Got next segment. Inline: true Aug 7 23:48:42.141210076 (14h9m3.885266937s) [upload=14571]: Waiting for error group managing segments... Aug 7 23:48:42.141288577 (14h9m3.885345438s) [upload=14571]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:48:42.141294246 (14h9m3.885351107s) [upload=14571]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:48:42.141297649 (14h9m3.885354509s) [upload=14571]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:48:42.292290080 (14h9m4.036346927s) [upload=14571]: Done uploading segments... Aug 7 23:48:42.292309333 (14h9m4.036366179s) [upload=14571]: Upload finished. err: <nil> Aug 7 23:49:47.788743937 (14h10m9.532800783s) [upload=14591]: Starting upload Aug 7 23:49:47.788822734 (14h10m9.5328796s) [upload=14591]: Done starting upload Aug 7 23:49:47.788889290 (14h10m9.532946138s) [upload=14591]: Uploading segments... Aug 7 23:49:47.788950938 (14h10m9.533007798s) [upload=14591]: Got next segment. Inline: true Aug 7 23:49:47.788955912 (14h10m9.533012775s) [upload=14591]: Waiting for error group managing segments... Aug 7 23:49:47.789005080 (14h10m9.533061926s) [upload=14591]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:49:47.789009334 (14h10m9.533066194s) [upload=14591]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:49:47.789014285 (14h10m9.53307113s) [upload=14591]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:49:47.932333015 (14h10m9.676389863s) [upload=14591]: Done uploading segments... Aug 7 23:49:47.932362673 (14h10m9.676419519s) [upload=14591]: Upload finished. err: <nil> Aug 7 23:50:23.531847225 (14h10m45.275904072s) [upload=14601]: Starting upload Aug 7 23:50:23.531926184 (14h10m45.275983047s) [upload=14601]: Done starting upload Aug 7 23:50:23.531993110 (14h10m45.27604996s) [upload=14601]: Uploading segments... Aug 7 23:50:23.532035359 (14h10m45.276092227s) [upload=14601]: Got next segment. Inline: true Aug 7 23:50:23.532041373 (14h10m45.276098223s) [upload=14601]: Waiting for error group managing segments... Aug 7 23:50:23.532083061 (14h10m45.276139912s) [upload=14601]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:23.532088436 (14h10m45.276145285s) [upload=14601]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:23.532091688 (14h10m45.276148539s) [upload=14601]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:23.673405160 (14h10m45.417462011s) [upload=14601]: Done uploading segments... Aug 7 23:50:23.673423409 (14h10m45.417480259s) [upload=14601]: Upload finished. err: <nil> Aug 7 23:51:32.204614635 (14h11m53.948671502s) [upload=14620]: Starting upload Aug 7 23:51:32.204700450 (14h11m53.948757313s) [upload=14620]: Done starting upload Aug 7 23:51:32.204781575 (14h11m53.948838427s) [upload=14620]: Uploading segments... Aug 7 23:51:32.204839741 (14h11m53.94889661s) [upload=14620]: Got next segment. Inline: true Aug 7 23:51:32.204847624 (14h11m53.948904474s) [upload=14620]: Waiting for error group managing segments... Aug 7 23:51:32.204902472 (14h11m53.948959327s) [upload=14620]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:51:32.204907775 (14h11m53.948964625s) [upload=14620]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:51:32.204911177 (14h11m53.948968027s) [upload=14620]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:01.937570352 (14h12m23.681627223s) [upload=14620]: Done uploading segments... Aug 7 23:52:01.937615027 (14h12m23.681671888s) [upload=14620]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:52:03.484443714 (14h12m25.228500575s) [upload=14625]: Starting upload Aug 7 23:52:03.484557174 (14h12m25.228614037s) [upload=14625]: Done starting upload Aug 7 23:52:03.484649660 (14h12m25.22870651s) [upload=14625]: Uploading segments... Aug 7 23:52:03.484705970 (14h12m25.22876283s) [upload=14625]: Got next segment. Inline: true Aug 7 23:52:03.484711540 (14h12m25.228768402s) [upload=14625]: Waiting for error group managing segments... Aug 7 23:52:03.484758149 (14h12m25.228815009s) [upload=14625]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:03.484763263 (14h12m25.228820122s) [upload=14625]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:03.484766746 (14h12m25.228823606s) [upload=14625]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:03.643631412 (14h12m25.387688267s) [upload=14625]: Done uploading segments... Aug 7 23:52:03.643670624 (14h12m25.387727482s) [upload=14625]: Upload finished. err: <nil> Aug 7 23:52:25.641252610 (14h12m47.385309456s) [upload=14634]: Starting upload Aug 7 23:52:25.641347708 (14h12m47.385404571s) [upload=14634]: Done starting upload Aug 7 23:52:25.641412738 (14h12m47.385469584s) [upload=14634]: Uploading segments... Aug 7 23:52:25.641487429 (14h12m47.385544276s) [upload=14634]: Got next segment. Inline: true Aug 7 23:52:25.641494756 (14h12m47.385551602s) [upload=14634]: Waiting for error group managing segments... Aug 7 23:52:25.641535502 (14h12m47.385592365s) [upload=14634]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:25.641556088 (14h12m47.385612953s) [upload=14634]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:25.641559364 (14h12m47.385616224s) [upload=14634]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:25.852337418 (14h12m47.596394279s) [upload=14634]: Done uploading segments... Aug 7 23:52:25.852355736 (14h12m47.596412584s) [upload=14634]: Upload finished. err: <nil> Aug 7 23:52:51.292140355 (14h13m13.036197215s) [upload=14641]: Starting upload Aug 7 23:52:51.292216133 (14h13m13.036272985s) [upload=14641]: Done starting upload Aug 7 23:52:51.292307446 (14h13m13.036364307s) [upload=14641]: Uploading segments... Aug 7 23:52:51.292389087 (14h13m13.036445956s) [upload=14641]: Got next segment. Inline: true Aug 7 23:52:51.292394933 (14h13m13.036451793s) [upload=14641]: Waiting for error group managing segments... Aug 7 23:52:51.292456032 (14h13m13.036512883s) [upload=14641]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:51.292461787 (14h13m13.036518647s) [upload=14641]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:51.292467676 (14h13m13.036524537s) [upload=14641]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:51.436915828 (14h13m13.180972678s) [upload=14641]: Done uploading segments... Aug 7 23:52:51.436934458 (14h13m13.18099131s) [upload=14641]: Upload finished. err: <nil> Aug 7 23:53:10.946261153 (14h13m32.690318028s) [upload=14646]: Starting upload Aug 7 23:53:10.946350198 (14h13m32.690407065s) [upload=14646]: Done starting upload Aug 7 23:53:10.946435423 (14h13m32.690492276s) [upload=14646]: Uploading segments... Aug 7 23:53:10.946489255 (14h13m32.690546124s) [upload=14646]: Got next segment. Inline: true Aug 7 23:53:10.946497660 (14h13m32.690554519s) [upload=14646]: Waiting for error group managing segments... Aug 7 23:53:10.946542678 (14h13m32.690599529s) [upload=14646]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:10.946548267 (14h13m32.690605128s) [upload=14646]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:10.946553792 (14h13m32.690610652s) [upload=14646]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:14.554651436 (14h13m36.298708309s) [upload=14646]: Done uploading segments... Aug 7 23:53:14.554675183 (14h13m36.298732045s) [upload=14646]: Upload finished. err: <nil> Duration: 30.000920367s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 6 out of 60 is failed (threshold=3) [.........XXX...XX.X.........................................] duration: 1.040113414s FAIL (2025-08-07 23:53:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.04013854s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 8 out of 60 is failed (threshold=3) [......X.XXXXX.X............................................X] duration: 1m0.000585653s FAIL (2025-08-07 23:53:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000607916s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 311.111302ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 23:53:20 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 311.206441ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 11 out of 60 is failed (threshold=3) [.....XXXXX.XX.XX.X.........................................X] duration: 30.001371532s FAIL (2025-08-07 23:53:18 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.001404052s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [...........X...X.XX.......................................XX] duration: 1m0.000707232s FAIL (2025-08-07 23:53:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== iece_id=HI7WZRDI3BHRFWQYMDR4Z4IHJUAYOA4B3ECS65MQKQZN2YDEVHZQ, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.399392520 (14h11m59.143449371s) [noise=true , address=64.49.13.169:4707 , piece_id=Y5MWKGWFH3LP2T42UHWOLNJTHV3SCKK2ZA7HRCBVHVDIG2UQEN7Q, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.399686408 (14h11m59.143743258s) [noise=true , address=64.49.13.138:4488 , piece_id=C3DZT3DLUGAFQTRQARWSBWVJ4I77QWHJGIQZTVX4W6LU5HABXAZA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.399996250 (14h11m59.14405311s) [noise=true , address=104.36.82.43:4868 , piece_id=VP5ASYQLM6PAWHV4DMBVHIRSDX7WPMXORUCQQHLXQ7UE5WUNGY7Q, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.400264269 (14h11m59.144321115s) [noise=true , address=104.36.82.53:4136 , piece_id=IOQFF6W5IC5AUDFV76ZVTKWUAIOVNNXK6WSCCINXXTZLFRIJWHWQ, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.400485298 (14h11m59.144542144s) [noise=true , address=104.36.82.61:4387 , piece_id=N3THT2DLNTVJK72OPSHHTBG34WP53NHR7L5VC6IU5EEXW7O7EPIA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.400710393 (14h11m59.144767254s) [noise=true , address=45.91.116.14:4218 , piece_id=4JCROTVISHYY7LBCTQ637ZROGDEDIJUH222CFSKVIYDBJR7BTK3Q, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.400941379 (14h11m59.14499824s) [noise=true , address=104.36.82.23:4453 , piece_id=ZKDR6X6V3GMABOUC66VFXH33QVIEC3PQSMLLVJMMOBQIFNQVOKZA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.401251698 (14h11m59.145308559s) [noise=true , address=104.36.82.42:4808 , piece_id=HBNT2DIZC4L66S7FLKXCK5XL4JDRVXTYZD6NPYYHUBC72ZHRDFKA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.403864642 (14h11m59.147921505s) [noise=true , address=104.36.82.12:4165 , piece_id=XRWHBRGQHCBTIBJUYQMW337V3ZDR2GP67ARSQVLIX5KALUD27AWA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.404219088 (14h11m59.148275951s) [noise=true , address=104.36.82.41:4765 , piece_id=K2M5MCWY2WSDKPTBPCEL5GQ3FDFQYCFUOBW5CNJUVB72ALWM3DRA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.404507731 (14h11m59.148564592s) [noise=true , address=104.36.82.45:4909 , piece_id=CEPK5VMIAWXINNVXZIDAUQMIMO42RGHBSMC5PFECCZ7TM4BZIUUA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.404771250 (14h11m59.148828112s) [noise=true , address=79.127.201.215:4349 , piece_id=NMWPQT2OMODFL4ZPUJMYMAXPQ27KIQOUXSGKJREJGV65ZLZIBY7Q, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.405722883 (14h11m59.149779746s) [noise=true , address=104.36.82.37:4674 , piece_id=CXTRIGLDCWLTVOB3TUPG6S7VAGWBNBGPTNTWZOGJFXJA3CYQQ3EA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.405966619 (14h11m59.150023478s) [noise=true , address=104.36.82.9:4039 , piece_id=TL3QGWAW4ZHOULIJRERYSPBHH5YAA4SRYQ3SMPCZ5T6O7E6SVNUQ, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.406068528 (14h11m59.150125388s) [noise=true , address=104.36.82.62:4434 , piece_id=CQPE5HJRY2LXTHJWWMYBNDHVOR5B5MBM5DGQQ7MRWXJR6M7IKOBA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.406404119 (14h11m59.150460966s) [noise=true , address=104.36.82.60:4352 , piece_id=JZ2HOZNEYEHOFN5JT67EAVLY7EQGP3B5ENHUFRI6NJCH276P72QQ, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.406657446 (14h11m59.150714311s) [noise=true , address=104.36.82.58:4272 , piece_id=5UBUOONWVORSDNUG2D3WW7VYRK4N22D6Y3Y3GNBMHF2ECZUXDAKQ, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.983166607 (14h11m59.727223467s) [upload=14617]: Next returned nil segment Aug 7 23:51:37.983195615 (14h11m59.727252481s) [upload=14617]: Waiting for error group managing segments... Aug 7 23:51:38.130379502 (14h11m59.874436364s) [noise=true , address=104.36.82.53:4136 , piece_id=IOQFF6W5IC5AUDFV76ZVTKWUAIOVNNXK6WSCCINXXTZLFRIJWHWQ, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.151652145 (14h11m59.895708992s) [noise=true , address=104.36.82.41:4765 , piece_id=K2M5MCWY2WSDKPTBPCEL5GQ3FDFQYCFUOBW5CNJUVB72ALWM3DRA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.152381420 (14h11m59.896438274s) [noise=true , address=104.36.82.58:4272 , piece_id=5UBUOONWVORSDNUG2D3WW7VYRK4N22D6Y3Y3GNBMHF2ECZUXDAKQ, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.153368721 (14h11m59.897425572s) [noise=true , address=104.36.82.55:4205 , piece_id=HI7WZRDI3BHRFWQYMDR4Z4IHJUAYOA4B3ECS65MQKQZN2YDEVHZQ, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.166206728 (14h11m59.910263574s) [noise=true , address=104.36.82.60:4352 , piece_id=JZ2HOZNEYEHOFN5JT67EAVLY7EQGP3B5ENHUFRI6NJCH276P72QQ, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.178216859 (14h11m59.922273709s) [noise=true , address=104.36.82.42:4808 , piece_id=HBNT2DIZC4L66S7FLKXCK5XL4JDRVXTYZD6NPYYHUBC72ZHRDFKA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.191290896 (14h11m59.935347742s) [noise=true , address=45.91.116.14:4218 , piece_id=4JCROTVISHYY7LBCTQ637ZROGDEDIJUH222CFSKVIYDBJR7BTK3Q, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.192130895 (14h11m59.936187748s) [noise=true , address=104.36.82.61:4387 , piece_id=N3THT2DLNTVJK72OPSHHTBG34WP53NHR7L5VC6IU5EEXW7O7EPIA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.206383044 (14h11m59.950439898s) [noise=true , address=104.36.82.12:4165 , piece_id=XRWHBRGQHCBTIBJUYQMW337V3ZDR2GP67ARSQVLIX5KALUD27AWA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.214301438 (14h11m59.958358284s) [noise=true , address=104.36.82.9:4039 , piece_id=TL3QGWAW4ZHOULIJRERYSPBHH5YAA4SRYQ3SMPCZ5T6O7E6SVNUQ, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.313332841 (14h12m0.057389691s) [noise=true , address=104.36.82.62:4434 , piece_id=CQPE5HJRY2LXTHJWWMYBNDHVOR5B5MBM5DGQQ7MRWXJR6M7IKOBA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.369163404 (14h12m0.113220261s) [noise=true , address=104.36.82.23:4453 , piece_id=ZKDR6X6V3GMABOUC66VFXH33QVIEC3PQSMLLVJMMOBQIFNQVOKZA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.468307450 (14h12m0.21236431s) [noise=true , address=104.36.82.37:4674 , piece_id=CXTRIGLDCWLTVOB3TUPG6S7VAGWBNBGPTNTWZOGJFXJA3CYQQ3EA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.539243271 (14h12m0.283300147s) [noise=true , address=79.127.201.215:4349 , piece_id=NMWPQT2OMODFL4ZPUJMYMAXPQ27KIQOUXSGKJREJGV65ZLZIBY7Q, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.539285855 (14h12m0.283342716s) [seg_pos={0 0}, upload=14617]: Segment reached optimal threshold of 14 pieces. Aug 7 23:51:38.539484535 (14h12m0.283541382s) [noise=true , address=104.36.82.45:4909 , piece_id=CEPK5VMIAWXINNVXZIDAUQMIMO42RGHBSMC5PFECCZ7TM4BZIUUA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14Pqp5cbhmd44QnikAuD5TgsKFDR3gB2qtLuVSroaWVv4U2cDh): protocol: expected piece hash; context canceled; EOF Aug 7 23:51:38.539517293 (14h12m0.283574153s) [noise=true , address=64.49.13.138:4488 , piece_id=C3DZT3DLUGAFQTRQARWSBWVJ4I77QWHJGIQZTVX4W6LU5HABXAZA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mQqzHsTJvR5wHpEGXDixk3NcRuBKxY2cL82eJMHidcxFiPCZb): protocol: expected piece hash; context canceled; EOF Aug 7 23:51:38.539590397 (14h12m0.283647259s) [noise=true , address=104.36.82.43:4868 , piece_id=VP5ASYQLM6PAWHV4DMBVHIRSDX7WPMXORUCQQHLXQ7UE5WUNGY7Q, seg_pos={0 0}, upload=14617]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124LJtYshPwu1uZJwVyjrjUMk3htyVnp9ntnZWSUb7X3bNkwU6t): protocol: expected piece hash; context canceled; EOF Aug 7 23:51:38.539646799 (14h12m0.28370366s) [noise=true , address=64.49.13.169:4707 , piece_id=Y5MWKGWFH3LP2T42UHWOLNJTHV3SCKK2ZA7HRCBVHVDIG2UQEN7Q, seg_pos={0 0}, upload=14617]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: expected piece hash; context canceled; EOF Aug 7 23:51:38.539673304 (14h12m0.283730165s) [seg_pos={0 0}, upload=14617]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:51:38.539768774 (14h12m0.283825642s) [upload=14617]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:51:38.539775704 (14h12m0.283832576s) [upload=14617]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:17.809583811 (14h12m39.553640671s) [upload=14617]: Done uploading segments... Aug 7 23:52:17.809611159 (14h12m39.55366802s) [upload=14617]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:52:18.222005824 (14h12m39.966062686s) [upload=14632]: Starting upload Aug 7 23:52:18.222080177 (14h12m39.966137038s) [upload=14632]: Done starting upload Aug 7 23:52:18.222201874 (14h12m39.966258723s) [upload=14632]: Uploading segments... Aug 7 23:52:18.222371001 (14h12m39.966427854s) [upload=14632]: Got next segment. Inline: false Aug 7 23:52:18.222437250 (14h12m39.966494102s) [upload=14632]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:18.222442846 (14h12m39.966499706s) [upload=14632]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:53:17.949653879 (14h13m39.693710745s) [upload=14632]: Next segment err: context deadline exceeded Aug 7 23:53:17.949857332 (14h13m39.693914187s) [upload=14632]: Done uploading segments... Aug 7 23:53:17.949878048 (14h13m39.6939349s) [upload=14632]: Upload finished. err: context deadline exceeded Duration: 1m0.000721345s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 2.855460509s FAIL (2025-08-07 23:53:14 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== 294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:32:45.172127356 (13h53m6.916184241s) [upload=14294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:32:45.390149151 (13h53m7.134206011s) [seg_pos={0 0}, upload=14294]: Begin upload segment... Aug 7 23:32:45.390215910 (13h53m7.134272761s) [seg_pos={0 0}, upload=14294]: Done begin upload segment. Aug 7 23:32:45.390249193 (13h53m7.134306059s) [noise=true , address=104.36.82.19:4405 , piece_id=MHM3MWZJFTXSJT7PTOFSWHUYO3L6NIBUV6VU66XXZDH3MC3EFR2Q, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390272919 (13h53m7.134329788s) [noise=true , address=77.90.31.12:4593 , piece_id=C3RVQHFS4L7SKPFFIZ52MAQWGD7P4JNG3SNPCNY53SQUD72PZRUA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390658970 (13h53m7.13471583s) [noise=true , address=104.36.82.16:4321 , piece_id=YTF4RWV2SD6Z4UV4XRYPBRGPCTSUTC6TQKL3DHVKU65ALTNA4TEQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390852777 (13h53m7.134909659s) [noise=true , address=104.36.82.34:4566 , piece_id=3YEAUSXEL7WNZ3O7WM24Z2DKY2TDVKMRZSHMEPMNEL2LR7BY7DBQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390957991 (13h53m7.135014851s) [noise=true , address=104.36.82.47:4523 , piece_id=MJIMHSO66V7OWFFK2UALPKSI7CGGCTHNPZINDXMVIHDLENIW5LFA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.391483291 (13h53m7.135540162s) [noise=true , address=104.36.82.5:4861 , piece_id=GRDGO5LNCETQWRU3M6LACGRW5MNA2KFLXUBT5R2UH6SEGWP6RVRQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.391931094 (13h53m7.135987964s) [noise=true , address=104.36.82.10:4090 , piece_id=V6NPPVIYEH6WLIW7OMT3NDFPR7UDWIT46GW6AFCAWUJTBLC3O55Q, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392250266 (13h53m7.136307136s) [noise=true , address=104.36.82.62:4439 , piece_id=ILRD4TEFW4PSLV4EGPORG3FM6U2HB3WSL5KP3PN566FAOGWODLYQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392611689 (13h53m7.136668559s) [noise=true , address=104.36.82.54:4160 , piece_id=GDBORRYX5QBQGH7ZMMUUAV7AAR5UHZOTK5FTQUPC7YKZC2KD7I3A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392917410 (13h53m7.136974282s) [noise=true , address=104.36.82.23:4477 , piece_id=HAF6424ZCIBGSR4US5LN5ZWE5JMNMLFGUMQQ4YP2ARP7PQXBH5GQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393270536 (13h53m7.137327398s) [noise=true , address=104.36.82.44:4881 , piece_id=XXVSXG3ILFXG4IVMELHL45TRRHDFVVJMJ5LGOHU5EDHG6XXCDH6A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393582635 (13h53m7.137639498s) [noise=true , address=104.36.82.37:4665 , piece_id=RUKKGC4Y3FIVLZO2GRWJN3J6P36VWP5VSC2HCLEU3MZDDFXODJAQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393856926 (13h53m7.137913788s) [noise=true , address=104.36.82.11:4128 , piece_id=XRPULHI7BYDF2JOIASSN5QKGEAC6FJS2F5C22WIY7EKYILRHZ73A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394196364 (13h53m7.138253227s) [noise=true , address=104.36.82.38:4706 , piece_id=BIHIPQX6LRWBNFN4SDC3K3V22AMGL3RC5B4RVE7QXJFKB4XDN3BA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394484397 (13h53m7.138541257s) [noise=true , address=104.36.82.61:4381 , piece_id=TTZOSRP6YPL6P4C6PY5S4K7Z6CIZJDPX7NUNBCK73CWZPCA3HQHA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394593709 (13h53m7.138650578s) [noise=true , address=45.91.116.5:4300 , piece_id=MADJCANN2O35MKXT2PIR5PGMJ3PNTAQAJNNGOK6ISJNVAH2L7YIA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.395093013 (13h53m7.139149879s) [noise=true , address=104.36.82.22:4530 , piece_id=HMUUW5GYH6TRHYXP4YXBHD566FWQRILEHI53AVURA5QTQGG6USFA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.395447945 (13h53m7.139504805s) [noise=true , address=104.36.82.58:4293 , piece_id=I2ETFXYTHYWYJYSHTVOBQTO6ZPGNMA4D3NVKYDE2W34EH4SQE23A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.954469844 (13h53m7.698526694s) [upload=14294]: Next returned nil segment Aug 7 23:32:45.954487953 (13h53m7.69854487s) [upload=14294]: Waiting for error group managing segments... Aug 7 23:32:46.126888627 (13h53m7.870945474s) [noise=true , address=104.36.82.10:4090 , piece_id=V6NPPVIYEH6WLIW7OMT3NDFPR7UDWIT46GW6AFCAWUJTBLC3O55Q, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.132983844 (13h53m7.877040704s) [noise=true , address=104.36.82.16:4321 , piece_id=YTF4RWV2SD6Z4UV4XRYPBRGPCTSUTC6TQKL3DHVKU65ALTNA4TEQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.138277216 (13h53m7.882334067s) [noise=true , address=104.36.82.23:4477 , piece_id=HAF6424ZCIBGSR4US5LN5ZWE5JMNMLFGUMQQ4YP2ARP7PQXBH5GQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.142033326 (13h53m7.886090185s) [noise=true , address=104.36.82.34:4566 , piece_id=3YEAUSXEL7WNZ3O7WM24Z2DKY2TDVKMRZSHMEPMNEL2LR7BY7DBQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.151247206 (13h53m7.895304058s) [noise=true , address=104.36.82.22:4530 , piece_id=HMUUW5GYH6TRHYXP4YXBHD566FWQRILEHI53AVURA5QTQGG6USFA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.151981081 (13h53m7.896037932s) [noise=true , address=104.36.82.58:4293 , piece_id=I2ETFXYTHYWYJYSHTVOBQTO6ZPGNMA4D3NVKYDE2W34EH4SQE23A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.165022882 (13h53m7.909079743s) [noise=true , address=104.36.82.44:4881 , piece_id=XXVSXG3ILFXG4IVMELHL45TRRHDFVVJMJ5LGOHU5EDHG6XXCDH6A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.184418821 (13h53m7.928475667s) [noise=true , address=104.36.82.47:4523 , piece_id=MJIMHSO66V7OWFFK2UALPKSI7CGGCTHNPZINDXMVIHDLENIW5LFA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.184532669 (13h53m7.928589516s) [noise=true , address=45.91.116.5:4300 , piece_id=MADJCANN2O35MKXT2PIR5PGMJ3PNTAQAJNNGOK6ISJNVAH2L7YIA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.193375808 (13h53m7.937432656s) [noise=true , address=104.36.82.54:4160 , piece_id=GDBORRYX5QBQGH7ZMMUUAV7AAR5UHZOTK5FTQUPC7YKZC2KD7I3A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.204247501 (13h53m7.948304372s) [noise=true , address=104.36.82.38:4706 , piece_id=BIHIPQX6LRWBNFN4SDC3K3V22AMGL3RC5B4RVE7QXJFKB4XDN3BA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.235225782 (13h53m7.979282629s) [noise=true , address=104.36.82.62:4439 , piece_id=ILRD4TEFW4PSLV4EGPORG3FM6U2HB3WSL5KP3PN566FAOGWODLYQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.269129867 (13h53m8.013186721s) [noise=true , address=104.36.82.5:4861 , piece_id=GRDGO5LNCETQWRU3M6LACGRW5MNA2KFLXUBT5R2UH6SEGWP6RVRQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.278216912 (13h53m8.022273764s) [noise=true , address=104.36.82.19:4405 , piece_id=MHM3MWZJFTXSJT7PTOFSWHUYO3L6NIBUV6VU66XXZDH3MC3EFR2Q, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.278247847 (13h53m8.022304708s) [seg_pos={0 0}, upload=14294]: Segment reached optimal threshold of 14 pieces. Aug 7 23:32:46.278485612 (13h53m8.022542463s) [noise=true , address=104.36.82.11:4128 , piece_id=XRPULHI7BYDF2JOIASSN5QKGEAC6FJS2F5C22WIY7EKYILRHZ73A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wABkAtAjoYkKyNoAiaVCxSvY6WgvHvAYFPCjpNojtpiYiGtDS): protocol: expected piece hash; context canceled; EOF Aug 7 23:32:46.278511698 (13h53m8.022568566s) [noise=true , address=77.90.31.12:4593 , piece_id=C3RVQHFS4L7SKPFFIZ52MAQWGD7P4JNG3SNPCNY53SQUD72PZRUA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kSxoZb9nmG6oGNytpFTsoiGh3ZPmebVVe8TZxGzANaw7EfdeF): protocol: expected piece hash; context canceled; EOF Aug 7 23:32:46.278642807 (13h53m8.022699657s) [noise=true , address=104.36.82.61:4381 , piece_id=TTZOSRP6YPL6P4C6PY5S4K7Z6CIZJDPX7NUNBCK73CWZPCA3HQHA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zU35MWWaaC1npkP5qGcWBXAkfxGsd5UsdkM2nwX9x7wUGCDbY): protocol: write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection; write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection Aug 7 23:32:46.278650926 (13h53m8.022707786s) [noise=true , address=104.36.82.37:4665 , piece_id=RUKKGC4Y3FIVLZO2GRWJN3J6P36VWP5VSC2HCLEU3MZDDFXODJAQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jP9JrCmoQSdqrzXZxkNN1ihdFkaUMjGZss6Fh19zbL3YyWj5N): protocol: write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection; write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection Aug 7 23:32:46.278674050 (13h53m8.022730911s) [seg_pos={0 0}, upload=14294]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:32:46.278751154 (13h53m8.022808014s) [upload=14294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:32:46.278756168 (13h53m8.022813029s) [upload=14294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:32:46.498660020 (13h53m8.242716882s) [upload=14294]: Done uploading segments... Aug 7 23:32:46.498701263 (13h53m8.242758128s) [upload=14294]: Upload finished. err: <nil> Duration: 2.855497242s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 961.020882ms FAIL (2025-08-07 23:53:07 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== pload=14121]: Done uploading segments... Aug 7 23:22:28.505845197 (13h42m50.249902044s) [upload=14121]: Upload finished. err: <nil> Aug 7 23:23:10.377483605 (13h43m32.121540458s) [upload=14132]: Starting upload Aug 7 23:23:10.377535635 (13h43m32.121592495s) [upload=14132]: Done starting upload Aug 7 23:23:10.377603797 (13h43m32.121660647s) [upload=14132]: Uploading segments... Aug 7 23:23:10.377638935 (13h43m32.121695804s) [upload=14132]: Got next segment. Inline: true Aug 7 23:23:10.377643594 (13h43m32.121700454s) [upload=14132]: Waiting for error group managing segments... Aug 7 23:23:10.377688252 (13h43m32.121745104s) [upload=14132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:23:10.377693427 (13h43m32.121750286s) [upload=14132]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:23:10.377696805 (13h43m32.121753666s) [upload=14132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:23:10.562260724 (13h43m32.306317573s) [upload=14132]: Done uploading segments... Aug 7 23:23:10.562279228 (13h43m32.306336078s) [upload=14132]: Upload finished. err: <nil> Aug 7 23:24:07.896404858 (13h44m29.640461721s) [upload=14147]: Starting upload Aug 7 23:24:07.896490216 (13h44m29.640547079s) [upload=14147]: Done starting upload Aug 7 23:24:07.896580488 (13h44m29.640637339s) [upload=14147]: Uploading segments... Aug 7 23:24:07.896655006 (13h44m29.640711874s) [upload=14147]: Got next segment. Inline: true Aug 7 23:24:07.896660179 (13h44m29.640717039s) [upload=14147]: Waiting for error group managing segments... Aug 7 23:24:07.896714840 (13h44m29.6407717s) [upload=14147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:24:07.896719399 (13h44m29.64077626s) [upload=14147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:24:07.896722726 (13h44m29.640779586s) [upload=14147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:24:08.048898130 (13h44m29.792954991s) [upload=14147]: Done uploading segments... Aug 7 23:24:08.048952526 (13h44m29.793009386s) [upload=14147]: Upload finished. err: <nil> Aug 7 23:24:53.794754379 (13h45m15.53881125s) [upload=14162]: Starting upload Aug 7 23:24:53.794842317 (13h45m15.538899177s) [upload=14162]: Done starting upload Aug 7 23:24:53.794935964 (13h45m15.538992814s) [upload=14162]: Uploading segments... Aug 7 23:24:53.795003752 (13h45m15.539060623s) [upload=14162]: Got next segment. Inline: true Aug 7 23:24:53.795010814 (13h45m15.539067676s) [upload=14162]: Waiting for error group managing segments... Aug 7 23:24:53.795066607 (13h45m15.539123488s) [upload=14162]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:24:53.795158449 (13h45m15.5392153s) [upload=14162]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:24:53.795163166 (13h45m15.539220026s) [upload=14162]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:24:53.946994982 (13h45m15.691051833s) [upload=14162]: Done uploading segments... Aug 7 23:24:53.947013862 (13h45m15.691070711s) [upload=14162]: Upload finished. err: <nil> Aug 7 23:25:40.029856966 (13h46m1.773913836s) [upload=14177]: Starting upload Aug 7 23:25:40.029986938 (13h46m1.774043809s) [upload=14177]: Done starting upload Aug 7 23:25:40.030076796 (13h46m1.774133642s) [upload=14177]: Uploading segments... Aug 7 23:25:40.030132567 (13h46m1.774189438s) [upload=14177]: Got next segment. Inline: true Aug 7 23:25:40.030139885 (13h46m1.774196757s) [upload=14177]: Waiting for error group managing segments... Aug 7 23:25:40.030197588 (13h46m1.774254461s) [upload=14177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:25:40.030204146 (13h46m1.774261016s) [upload=14177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:25:40.030207984 (13h46m1.77426485s) [upload=14177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:25:40.184198742 (13h46m1.928255592s) [upload=14177]: Done uploading segments... Aug 7 23:25:40.184219017 (13h46m1.928275867s) [upload=14177]: Upload finished. err: <nil> Aug 7 23:26:54.625649110 (13h47m16.369705971s) [upload=14198]: Starting upload Aug 7 23:26:54.625738563 (13h47m16.369795424s) [upload=14198]: Done starting upload Aug 7 23:26:54.625842691 (13h47m16.369899539s) [upload=14198]: Uploading segments... Aug 7 23:26:54.626016839 (13h47m16.37007371s) [upload=14198]: Got next segment. Inline: true Aug 7 23:26:54.626052112 (13h47m16.370108973s) [upload=14198]: Waiting for error group managing segments... Aug 7 23:26:54.626129780 (13h47m16.370186645s) [upload=14198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:26:54.626137392 (13h47m16.370194264s) [upload=14198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:26:54.626143182 (13h47m16.370200048s) [upload=14198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:26:54.775804171 (13h47m16.519861021s) [upload=14198]: Done uploading segments... Aug 7 23:26:54.775824249 (13h47m16.519881102s) [upload=14198]: Upload finished. err: <nil> Aug 7 23:27:32.263746490 (13h47m54.007803361s) [upload=14208]: Starting upload Aug 7 23:27:32.263833655 (13h47m54.007890501s) [upload=14208]: Done starting upload Aug 7 23:27:32.263976536 (13h47m54.008033385s) [upload=14208]: Uploading segments... Aug 7 23:27:32.264070926 (13h47m54.008127796s) [upload=14208]: Got next segment. Inline: true Aug 7 23:27:32.264080356 (13h47m54.00813733s) [upload=14208]: Waiting for error group managing segments... Aug 7 23:27:32.264154657 (13h47m54.008211526s) [upload=14208]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:27:32.264163587 (13h47m54.008220454s) [upload=14208]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:27:32.264172017 (13h47m54.008228914s) [upload=14208]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:27:32.490188559 (13h47m54.23424541s) [upload=14208]: Done uploading segments... Aug 7 23:27:32.490207326 (13h47m54.234264172s) [upload=14208]: Upload finished. err: <nil> Aug 7 23:27:57.927940477 (13h48m19.671997323s) [upload=14217]: Starting upload Aug 7 23:27:57.928016423 (13h48m19.672073283s) [upload=14217]: Done starting upload Aug 7 23:27:57.928101290 (13h48m19.672158142s) [upload=14217]: Uploading segments... Aug 7 23:27:57.928155692 (13h48m19.672212566s) [upload=14217]: Got next segment. Inline: true Aug 7 23:27:57.928162009 (13h48m19.672218869s) [upload=14217]: Waiting for error group managing segments... Aug 7 23:27:57.928213294 (13h48m19.672270145s) [upload=14217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:27:57.928219212 (13h48m19.672276074s) [upload=14217]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:27:57.928223521 (13h48m19.672280383s) [upload=14217]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:27:58.087544005 (13h48m19.831600874s) [upload=14217]: Done uploading segments... Aug 7 23:27:58.087570226 (13h48m19.831627094s) [upload=14217]: Upload finished. err: <nil> Aug 7 23:29:12.765200935 (13h49m34.509257795s) [upload=14237]: Starting upload Aug 7 23:29:12.765253106 (13h49m34.509309977s) [upload=14237]: Done starting upload Aug 7 23:29:12.765324641 (13h49m34.509381499s) [upload=14237]: Uploading segments... Aug 7 23:29:12.765374187 (13h49m34.509431056s) [upload=14237]: Got next segment. Inline: true Aug 7 23:29:12.765378947 (13h49m34.509435807s) [upload=14237]: Waiting for error group managing segments... Aug 7 23:29:12.765430859 (13h49m34.50948772s) [upload=14237]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:29:12.765438106 (13h49m34.50949497s) [upload=14237]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:29:12.765443127 (13h49m34.509499988s) [upload=14237]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:29:12.926437674 (13h49m34.670494535s) [upload=14237]: Done uploading segments... Aug 7 23:29:12.926457418 (13h49m34.670514268s) [upload=14237]: Upload finished. err: <nil> Aug 7 23:29:42.225760316 (13h50m3.969817177s) [upload=14244]: Starting upload Aug 7 23:29:42.225855063 (13h50m3.969911929s) [upload=14244]: Done starting upload Aug 7 23:29:42.225962489 (13h50m3.970019355s) [upload=14244]: Uploading segments... Aug 7 23:29:42.226078484 (13h50m3.970135331s) [upload=14244]: Got next segment. Inline: true Aug 7 23:29:42.226083996 (13h50m3.970140842s) [upload=14244]: Waiting for error group managing segments... Aug 7 23:29:42.226143067 (13h50m3.970199914s) [upload=14244]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:29:42.226148511 (13h50m3.970205371s) [upload=14244]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:29:42.226153006 (13h50m3.970209865s) [upload=14244]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:29:42.401822947 (13h50m4.14587981s) [upload=14244]: Done uploading segments... Aug 7 23:29:42.401847556 (13h50m4.145904405s) [upload=14244]: Upload finished. err: <nil> Aug 7 23:30:31.529840301 (13h50m53.273897165s) [upload=14254]: Starting upload Aug 7 23:30:31.530002531 (13h50m53.274059378s) [upload=14254]: Done starting upload Aug 7 23:30:31.530201544 (13h50m53.274258391s) [upload=14254]: Uploading segments... Aug 7 23:30:31.530298120 (13h50m53.274354985s) [upload=14254]: Got next segment. Inline: true Aug 7 23:30:31.530303457 (13h50m53.274360317s) [upload=14254]: Waiting for error group managing segments... Aug 7 23:30:31.530353721 (13h50m53.274410567s) [upload=14254]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:30:31.530358927 (13h50m53.274415788s) [upload=14254]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:30:31.530363758 (13h50m53.274420619s) [upload=14254]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:30:31.696350967 (13h50m53.440407828s) [upload=14254]: Done uploading segments... Aug 7 23:30:31.696369460 (13h50m53.440426307s) [upload=14254]: Upload finished. err: <nil> Duration: 961.061856ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 13 out of 60 is failed (threshold=3) [.....XXX.XXXXXX..XX.X...................................X...] duration: 862.165642ms FAIL (2025-08-07 23:52:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7 23:42:52.368849329 (14h3m14.112906191s) [upload=14469]: Done uploading segments... Aug 7 23:42:52.368869169 (14h3m14.112926018s) [upload=14469]: Upload finished. err: <nil> Aug 7 23:44:01.454335967 (14h4m23.198392817s) [upload=14497]: Starting upload Aug 7 23:44:01.454411480 (14h4m23.198468329s) [upload=14497]: Done starting upload Aug 7 23:44:01.454478346 (14h4m23.198535196s) [upload=14497]: Uploading segments... Aug 7 23:44:01.454558538 (14h4m23.198615407s) [upload=14497]: Got next segment. Inline: true Aug 7 23:44:01.454565269 (14h4m23.19862212s) [upload=14497]: Waiting for error group managing segments... Aug 7 23:44:01.454634028 (14h4m23.198690879s) [upload=14497]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:44:01.454640175 (14h4m23.198697042s) [upload=14497]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:44:01.454643973 (14h4m23.198700834s) [upload=14497]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:44:01.608945362 (14h4m23.353002226s) [upload=14497]: Done uploading segments... Aug 7 23:44:01.608968737 (14h4m23.353025586s) [upload=14497]: Upload finished. err: <nil> Aug 7 23:44:17.431921627 (14h4m39.175978475s) [upload=14500]: Starting upload Aug 7 23:44:17.431986941 (14h4m39.176043788s) [upload=14500]: Done starting upload Aug 7 23:44:17.432052901 (14h4m39.176109747s) [upload=14500]: Uploading segments... Aug 7 23:44:17.432152156 (14h4m39.176209002s) [upload=14500]: Got next segment. Inline: true Aug 7 23:44:17.432157449 (14h4m39.176214316s) [upload=14500]: Waiting for error group managing segments... Aug 7 23:44:17.432211045 (14h4m39.176267891s) [upload=14500]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:44:17.432215731 (14h4m39.176272598s) [upload=14500]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:44:17.432218553 (14h4m39.176275416s) [upload=14500]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:44:17.574187435 (14h4m39.318244286s) [upload=14500]: Done uploading segments... Aug 7 23:44:17.574205206 (14h4m39.31826206s) [upload=14500]: Upload finished. err: <nil> Aug 7 23:45:13.608437918 (14h5m35.352494778s) [upload=14514]: Starting upload Aug 7 23:45:13.608496969 (14h5m35.352553819s) [upload=14514]: Done starting upload Aug 7 23:45:13.608551220 (14h5m35.352608068s) [upload=14514]: Uploading segments... Aug 7 23:45:13.608624509 (14h5m35.352681379s) [upload=14514]: Got next segment. Inline: true Aug 7 23:45:13.608630286 (14h5m35.352687147s) [upload=14514]: Waiting for error group managing segments... Aug 7 23:45:13.608683973 (14h5m35.352740835s) [upload=14514]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:45:13.608690269 (14h5m35.352747129s) [upload=14514]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:45:13.608692795 (14h5m35.352749655s) [upload=14514]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:45:13.754427790 (14h5m35.498484651s) [upload=14514]: Done uploading segments... Aug 7 23:45:13.754446774 (14h5m35.498503628s) [upload=14514]: Upload finished. err: <nil> Aug 7 23:46:09.008386295 (14h6m30.752443146s) [upload=14528]: Starting upload Aug 7 23:46:09.008449478 (14h6m30.752506341s) [upload=14528]: Done starting upload Aug 7 23:46:09.008515325 (14h6m30.752572176s) [upload=14528]: Uploading segments... Aug 7 23:46:09.008597175 (14h6m30.75265404s) [upload=14528]: Got next segment. Inline: true Aug 7 23:46:09.008603295 (14h6m30.752660155s) [upload=14528]: Waiting for error group managing segments... Aug 7 23:46:09.008650523 (14h6m30.752707384s) [upload=14528]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:46:09.008656863 (14h6m30.752713712s) [upload=14528]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:46:09.008662476 (14h6m30.75271934s) [upload=14528]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:46:09.155644852 (14h6m30.899701703s) [upload=14528]: Done uploading segments... Aug 7 23:46:09.155664178 (14h6m30.899721028s) [upload=14528]: Upload finished. err: <nil> Aug 7 23:46:47.072912780 (14h7m8.816969628s) [upload=14538]: Starting upload Aug 7 23:46:47.072990499 (14h7m8.817047359s) [upload=14538]: Done starting upload Aug 7 23:46:47.073066127 (14h7m8.817122973s) [upload=14538]: Uploading segments... Aug 7 23:46:47.073122515 (14h7m8.817179381s) [upload=14538]: Got next segment. Inline: true Aug 7 23:46:47.073127584 (14h7m8.817184447s) [upload=14538]: Waiting for error group managing segments... Aug 7 23:46:47.073168839 (14h7m8.817225702s) [upload=14538]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:46:47.073173325 (14h7m8.817230185s) [upload=14538]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:46:47.073176118 (14h7m8.817232978s) [upload=14538]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:46:47.210119395 (14h7m8.954176244s) [upload=14538]: Done uploading segments... Aug 7 23:46:47.210142995 (14h7m8.954199841s) [upload=14538]: Upload finished. err: <nil> Aug 7 23:47:29.298549723 (14h7m51.042606575s) [upload=14552]: Starting upload Aug 7 23:47:29.298617898 (14h7m51.042674767s) [upload=14552]: Done starting upload Aug 7 23:47:29.298698678 (14h7m51.042755528s) [upload=14552]: Uploading segments... Aug 7 23:47:29.298753306 (14h7m51.042810175s) [upload=14552]: Got next segment. Inline: true Aug 7 23:47:29.298758297 (14h7m51.042815158s) [upload=14552]: Waiting for error group managing segments... Aug 7 23:47:29.298818439 (14h7m51.042875289s) [upload=14552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:47:29.298823844 (14h7m51.042880694s) [upload=14552]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:47:29.298827075 (14h7m51.042883925s) [upload=14552]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:47:29.457296385 (14h7m51.201353233s) [upload=14552]: Done uploading segments... Aug 7 23:47:29.457315555 (14h7m51.201372401s) [upload=14552]: Upload finished. err: <nil> Aug 7 23:48:42.140972885 (14h9m3.885029736s) [upload=14571]: Starting upload Aug 7 23:48:42.141036695 (14h9m3.885093558s) [upload=14571]: Done starting upload Aug 7 23:48:42.141117369 (14h9m3.88517422s) [upload=14571]: Uploading segments... Aug 7 23:48:42.141203219 (14h9m3.885260088s) [upload=14571]: Got next segment. Inline: true Aug 7 23:48:42.141210076 (14h9m3.885266937s) [upload=14571]: Waiting for error group managing segments... Aug 7 23:48:42.141288577 (14h9m3.885345438s) [upload=14571]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:48:42.141294246 (14h9m3.885351107s) [upload=14571]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:48:42.141297649 (14h9m3.885354509s) [upload=14571]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:48:42.292290080 (14h9m4.036346927s) [upload=14571]: Done uploading segments... Aug 7 23:48:42.292309333 (14h9m4.036366179s) [upload=14571]: Upload finished. err: <nil> Aug 7 23:49:47.788743937 (14h10m9.532800783s) [upload=14591]: Starting upload Aug 7 23:49:47.788822734 (14h10m9.5328796s) [upload=14591]: Done starting upload Aug 7 23:49:47.788889290 (14h10m9.532946138s) [upload=14591]: Uploading segments... Aug 7 23:49:47.788950938 (14h10m9.533007798s) [upload=14591]: Got next segment. Inline: true Aug 7 23:49:47.788955912 (14h10m9.533012775s) [upload=14591]: Waiting for error group managing segments... Aug 7 23:49:47.789005080 (14h10m9.533061926s) [upload=14591]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:49:47.789009334 (14h10m9.533066194s) [upload=14591]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:49:47.789014285 (14h10m9.53307113s) [upload=14591]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:49:47.932333015 (14h10m9.676389863s) [upload=14591]: Done uploading segments... Aug 7 23:49:47.932362673 (14h10m9.676419519s) [upload=14591]: Upload finished. err: <nil> Aug 7 23:50:23.531847225 (14h10m45.275904072s) [upload=14601]: Starting upload Aug 7 23:50:23.531926184 (14h10m45.275983047s) [upload=14601]: Done starting upload Aug 7 23:50:23.531993110 (14h10m45.27604996s) [upload=14601]: Uploading segments... Aug 7 23:50:23.532035359 (14h10m45.276092227s) [upload=14601]: Got next segment. Inline: true Aug 7 23:50:23.532041373 (14h10m45.276098223s) [upload=14601]: Waiting for error group managing segments... Aug 7 23:50:23.532083061 (14h10m45.276139912s) [upload=14601]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:23.532088436 (14h10m45.276145285s) [upload=14601]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:23.532091688 (14h10m45.276148539s) [upload=14601]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:23.673405160 (14h10m45.417462011s) [upload=14601]: Done uploading segments... Aug 7 23:50:23.673423409 (14h10m45.417480259s) [upload=14601]: Upload finished. err: <nil> Aug 7 23:51:32.204614635 (14h11m53.948671502s) [upload=14620]: Starting upload Aug 7 23:51:32.204700450 (14h11m53.948757313s) [upload=14620]: Done starting upload Aug 7 23:51:32.204781575 (14h11m53.948838427s) [upload=14620]: Uploading segments... Aug 7 23:51:32.204839741 (14h11m53.94889661s) [upload=14620]: Got next segment. Inline: true Aug 7 23:51:32.204847624 (14h11m53.948904474s) [upload=14620]: Waiting for error group managing segments... Aug 7 23:51:32.204902472 (14h11m53.948959327s) [upload=14620]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:51:32.204907775 (14h11m53.948964625s) [upload=14620]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:51:32.204911177 (14h11m53.948968027s) [upload=14620]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:01.937570352 (14h12m23.681627223s) [upload=14620]: Done uploading segments... Aug 7 23:52:01.937615027 (14h12m23.681671888s) [upload=14620]: Upload finished. err: metaclient: context deadline exceeded Duration: 862.205998ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 6 out of 60 is failed (threshold=3) [..........XXX...XX.X........................................] duration: 1.138213068s FAIL (2025-08-07 23:52:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.138242573s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [......X.XX..X...............................................] duration: 2.92363881s FAIL (2025-08-07 23:52:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.923678778s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 9 out of 60 is failed (threshold=3) [.........XXXXXX.XX........................................X.] duration: 2.214374998s FAIL (2025-08-07 23:52:42 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.214408087s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 7 out of 60 is failed (threshold=3) [...XX..XXXXX................................................] duration: 878.384646ms FAIL (2025-08-07 23:52:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== :10:03.085332650 (13h30m24.829389513s) [upload=13904]: Done starting upload Aug 7 23:10:03.085414170 (13h30m24.829471025s) [upload=13904]: Uploading segments... Aug 7 23:10:03.085475435 (13h30m24.829532308s) [upload=13904]: Got next segment. Inline: true Aug 7 23:10:03.085480843 (13h30m24.829537706s) [upload=13904]: Waiting for error group managing segments... Aug 7 23:10:03.085544347 (13h30m24.829601211s) [upload=13904]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:03.085552551 (13h30m24.829609424s) [upload=13904]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:03.085555974 (13h30m24.829612844s) [upload=13904]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:10:03.247308964 (13h30m24.991365825s) [upload=13904]: Done uploading segments... Aug 7 23:10:03.247396260 (13h30m24.991453126s) [upload=13904]: Upload finished. err: <nil> Aug 7 23:10:40.522399484 (13h31m2.266456348s) [upload=13915]: Starting upload Aug 7 23:10:40.522487646 (13h31m2.266544496s) [upload=13915]: Done starting upload Aug 7 23:10:40.522571041 (13h31m2.266627902s) [upload=13915]: Uploading segments... Aug 7 23:10:40.522619425 (13h31m2.266676294s) [upload=13915]: Got next segment. Inline: true Aug 7 23:10:40.522626307 (13h31m2.266683183s) [upload=13915]: Waiting for error group managing segments... Aug 7 23:10:40.522684106 (13h31m2.266740971s) [upload=13915]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:40.522691484 (13h31m2.26674835s) [upload=13915]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:40.522696090 (13h31m2.266752961s) [upload=13915]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:40.254283342 (13h32m1.998340192s) [upload=13915]: Done uploading segments... Aug 7 23:11:40.254304548 (13h32m1.998361398s) [upload=13915]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:11:40.667902567 (13h32m2.411959413s) [upload=13933]: Starting upload Aug 7 23:11:40.667961940 (13h32m2.412018788s) [upload=13933]: Done starting upload Aug 7 23:11:40.668020524 (13h32m2.41207737s) [upload=13933]: Uploading segments... Aug 7 23:11:40.668075247 (13h32m2.412132113s) [upload=13933]: Got next segment. Inline: true Aug 7 23:11:40.668080664 (13h32m2.412137525s) [upload=13933]: Waiting for error group managing segments... Aug 7 23:11:40.668121190 (13h32m2.412178036s) [upload=13933]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:40.668125756 (13h32m2.412182616s) [upload=13933]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:40.668128431 (13h32m2.412185291s) [upload=13933]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:40.394243254 (13h33m2.138300118s) [upload=13933]: Done uploading segments... Aug 7 23:12:40.394270270 (13h33m2.138327132s) [upload=13933]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:12:40.798630406 (13h33m2.542687267s) [upload=13953]: Starting upload Aug 7 23:12:40.798725809 (13h33m2.54278266s) [upload=13953]: Done starting upload Aug 7 23:12:40.798811946 (13h33m2.542868796s) [upload=13953]: Uploading segments... Aug 7 23:12:40.798867069 (13h33m2.542923932s) [upload=13953]: Got next segment. Inline: true Aug 7 23:12:40.798873108 (13h33m2.542929959s) [upload=13953]: Waiting for error group managing segments... Aug 7 23:12:40.798918573 (13h33m2.542975422s) [upload=13953]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:40.798924603 (13h33m2.542981463s) [upload=13953]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:40.798939855 (13h33m2.542996716s) [upload=13953]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:40.942390421 (13h33m2.686447272s) [upload=13953]: Done uploading segments... Aug 7 23:12:40.942410212 (13h33m2.686467062s) [upload=13953]: Upload finished. err: <nil> Aug 7 23:13:18.137108974 (13h33m39.881165825s) [upload=13967]: Starting upload Aug 7 23:13:18.137180408 (13h33m39.881237292s) [upload=13967]: Done starting upload Aug 7 23:13:18.137258925 (13h33m39.881315786s) [upload=13967]: Uploading segments... Aug 7 23:13:18.137309635 (13h33m39.881366505s) [upload=13967]: Got next segment. Inline: true Aug 7 23:13:18.137314511 (13h33m39.881371371s) [upload=13967]: Waiting for error group managing segments... Aug 7 23:13:18.137374772 (13h33m39.881431633s) [upload=13967]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:18.137380412 (13h33m39.881437273s) [upload=13967]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:18.137383733 (13h33m39.881440593s) [upload=13967]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:18.301480519 (13h33m40.045537379s) [upload=13967]: Done uploading segments... Aug 7 23:13:18.301498918 (13h33m40.045555767s) [upload=13967]: Upload finished. err: <nil> Aug 7 23:14:13.169029066 (13h34m34.913085913s) [upload=13980]: Starting upload Aug 7 23:14:13.169095254 (13h34m34.913152101s) [upload=13980]: Done starting upload Aug 7 23:14:13.169153338 (13h34m34.913210184s) [upload=13980]: Uploading segments... Aug 7 23:14:13.169225662 (13h34m34.913282508s) [upload=13980]: Got next segment. Inline: true Aug 7 23:14:13.169230885 (13h34m34.913287744s) [upload=13980]: Waiting for error group managing segments... Aug 7 23:14:13.169272521 (13h34m34.913329381s) [upload=13980]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:13.169288952 (13h34m34.913345813s) [upload=13980]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:13.169291623 (13h34m34.913348483s) [upload=13980]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:12.895960606 (13h35m34.640017467s) [upload=13980]: Done uploading segments... Aug 7 23:15:12.895983608 (13h35m34.640040456s) [upload=13980]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:15:13.308146067 (13h35m35.052202918s) [upload=13997]: Starting upload Aug 7 23:15:13.308219271 (13h35m35.052276134s) [upload=13997]: Done starting upload Aug 7 23:15:13.308297319 (13h35m35.052354171s) [upload=13997]: Uploading segments... Aug 7 23:15:13.308360203 (13h35m35.052417072s) [upload=13997]: Got next segment. Inline: true Aug 7 23:15:13.308365548 (13h35m35.052422408s) [upload=13997]: Waiting for error group managing segments... Aug 7 23:15:13.308409794 (13h35m35.052466657s) [upload=13997]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:15:13.308426102 (13h35m35.052482963s) [upload=13997]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:15:13.308429725 (13h35m35.052486585s) [upload=13997]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:13.040589970 (13h36m34.784646843s) [upload=13997]: Done uploading segments... Aug 7 23:16:13.040614428 (13h36m34.784671274s) [upload=13997]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:16:20.767866726 (13h36m42.511923577s) [upload=14018]: Starting upload Aug 7 23:16:20.767927544 (13h36m42.511984398s) [upload=14018]: Done starting upload Aug 7 23:16:20.767993090 (13h36m42.512049942s) [upload=14018]: Uploading segments... Aug 7 23:16:20.768055054 (13h36m42.512111926s) [upload=14018]: Got next segment. Inline: true Aug 7 23:16:20.768061500 (13h36m42.512118351s) [upload=14018]: Waiting for error group managing segments... Aug 7 23:16:20.768130604 (13h36m42.512187577s) [upload=14018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:20.768139327 (13h36m42.512196192s) [upload=14018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:20.768144987 (13h36m42.512201849s) [upload=14018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:20.496845379 (13h37m42.240902231s) [upload=14018]: Done uploading segments... Aug 7 23:17:20.496873778 (13h37m42.240930632s) [upload=14018]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:17:20.916777280 (13h37m42.660834146s) [upload=14032]: Starting upload Aug 7 23:17:20.916877286 (13h37m42.660934149s) [upload=14032]: Done starting upload Aug 7 23:17:20.917002557 (13h37m42.661059408s) [upload=14032]: Uploading segments... Aug 7 23:17:20.917058206 (13h37m42.661115077s) [upload=14032]: Got next segment. Inline: true Aug 7 23:17:20.917064317 (13h37m42.661121178s) [upload=14032]: Waiting for error group managing segments... Aug 7 23:17:20.917116347 (13h37m42.661173214s) [upload=14032]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:17:20.917123150 (13h37m42.661180014s) [upload=14032]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:17:20.917127779 (13h37m42.661184648s) [upload=14032]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:18:20.640884349 (13h38m42.384941214s) [upload=14032]: Done uploading segments... Aug 7 23:18:20.640914749 (13h38m42.384971617s) [upload=14032]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:18:21.050365938 (13h38m42.79442281s) [upload=14051]: Starting upload Aug 7 23:18:21.050484720 (13h38m42.794541587s) [upload=14051]: Done starting upload Aug 7 23:18:21.050589902 (13h38m42.794646752s) [upload=14051]: Uploading segments... Aug 7 23:18:21.050663797 (13h38m42.794720667s) [upload=14051]: Got next segment. Inline: true Aug 7 23:18:21.050670451 (13h38m42.794727311s) [upload=14051]: Waiting for error group managing segments... Aug 7 23:18:21.050719413 (13h38m42.794776275s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:18:21.050725163 (13h38m42.794782023s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:18:21.050728952 (13h38m42.794785811s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:19:20.780076504 (13h39m42.524133366s) [upload=14051]: Done uploading segments... Aug 7 23:19:20.780099469 (13h39m42.52415633s) [upload=14051]: Upload finished. err: metaclient: context deadline exceeded Duration: 878.424517ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 13 out of 60 is failed (threshold=3) [......XXX.XXXXXX..XX.X...................................X..] duration: 948.080298ms FAIL (2025-08-07 23:52:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7 23:42:52.368849329 (14h3m14.112906191s) [upload=14469]: Done uploading segments... Aug 7 23:42:52.368869169 (14h3m14.112926018s) [upload=14469]: Upload finished. err: <nil> Aug 7 23:44:01.454335967 (14h4m23.198392817s) [upload=14497]: Starting upload Aug 7 23:44:01.454411480 (14h4m23.198468329s) [upload=14497]: Done starting upload Aug 7 23:44:01.454478346 (14h4m23.198535196s) [upload=14497]: Uploading segments... Aug 7 23:44:01.454558538 (14h4m23.198615407s) [upload=14497]: Got next segment. Inline: true Aug 7 23:44:01.454565269 (14h4m23.19862212s) [upload=14497]: Waiting for error group managing segments... Aug 7 23:44:01.454634028 (14h4m23.198690879s) [upload=14497]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:44:01.454640175 (14h4m23.198697042s) [upload=14497]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:44:01.454643973 (14h4m23.198700834s) [upload=14497]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:44:01.608945362 (14h4m23.353002226s) [upload=14497]: Done uploading segments... Aug 7 23:44:01.608968737 (14h4m23.353025586s) [upload=14497]: Upload finished. err: <nil> Aug 7 23:44:17.431921627 (14h4m39.175978475s) [upload=14500]: Starting upload Aug 7 23:44:17.431986941 (14h4m39.176043788s) [upload=14500]: Done starting upload Aug 7 23:44:17.432052901 (14h4m39.176109747s) [upload=14500]: Uploading segments... Aug 7 23:44:17.432152156 (14h4m39.176209002s) [upload=14500]: Got next segment. Inline: true Aug 7 23:44:17.432157449 (14h4m39.176214316s) [upload=14500]: Waiting for error group managing segments... Aug 7 23:44:17.432211045 (14h4m39.176267891s) [upload=14500]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:44:17.432215731 (14h4m39.176272598s) [upload=14500]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:44:17.432218553 (14h4m39.176275416s) [upload=14500]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:44:17.574187435 (14h4m39.318244286s) [upload=14500]: Done uploading segments... Aug 7 23:44:17.574205206 (14h4m39.31826206s) [upload=14500]: Upload finished. err: <nil> Aug 7 23:45:13.608437918 (14h5m35.352494778s) [upload=14514]: Starting upload Aug 7 23:45:13.608496969 (14h5m35.352553819s) [upload=14514]: Done starting upload Aug 7 23:45:13.608551220 (14h5m35.352608068s) [upload=14514]: Uploading segments... Aug 7 23:45:13.608624509 (14h5m35.352681379s) [upload=14514]: Got next segment. Inline: true Aug 7 23:45:13.608630286 (14h5m35.352687147s) [upload=14514]: Waiting for error group managing segments... Aug 7 23:45:13.608683973 (14h5m35.352740835s) [upload=14514]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:45:13.608690269 (14h5m35.352747129s) [upload=14514]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:45:13.608692795 (14h5m35.352749655s) [upload=14514]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:45:13.754427790 (14h5m35.498484651s) [upload=14514]: Done uploading segments... Aug 7 23:45:13.754446774 (14h5m35.498503628s) [upload=14514]: Upload finished. err: <nil> Aug 7 23:46:09.008386295 (14h6m30.752443146s) [upload=14528]: Starting upload Aug 7 23:46:09.008449478 (14h6m30.752506341s) [upload=14528]: Done starting upload Aug 7 23:46:09.008515325 (14h6m30.752572176s) [upload=14528]: Uploading segments... Aug 7 23:46:09.008597175 (14h6m30.75265404s) [upload=14528]: Got next segment. Inline: true Aug 7 23:46:09.008603295 (14h6m30.752660155s) [upload=14528]: Waiting for error group managing segments... Aug 7 23:46:09.008650523 (14h6m30.752707384s) [upload=14528]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:46:09.008656863 (14h6m30.752713712s) [upload=14528]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:46:09.008662476 (14h6m30.75271934s) [upload=14528]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:46:09.155644852 (14h6m30.899701703s) [upload=14528]: Done uploading segments... Aug 7 23:46:09.155664178 (14h6m30.899721028s) [upload=14528]: Upload finished. err: <nil> Aug 7 23:46:47.072912780 (14h7m8.816969628s) [upload=14538]: Starting upload Aug 7 23:46:47.072990499 (14h7m8.817047359s) [upload=14538]: Done starting upload Aug 7 23:46:47.073066127 (14h7m8.817122973s) [upload=14538]: Uploading segments... Aug 7 23:46:47.073122515 (14h7m8.817179381s) [upload=14538]: Got next segment. Inline: true Aug 7 23:46:47.073127584 (14h7m8.817184447s) [upload=14538]: Waiting for error group managing segments... Aug 7 23:46:47.073168839 (14h7m8.817225702s) [upload=14538]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:46:47.073173325 (14h7m8.817230185s) [upload=14538]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:46:47.073176118 (14h7m8.817232978s) [upload=14538]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:46:47.210119395 (14h7m8.954176244s) [upload=14538]: Done uploading segments... Aug 7 23:46:47.210142995 (14h7m8.954199841s) [upload=14538]: Upload finished. err: <nil> Aug 7 23:47:29.298549723 (14h7m51.042606575s) [upload=14552]: Starting upload Aug 7 23:47:29.298617898 (14h7m51.042674767s) [upload=14552]: Done starting upload Aug 7 23:47:29.298698678 (14h7m51.042755528s) [upload=14552]: Uploading segments... Aug 7 23:47:29.298753306 (14h7m51.042810175s) [upload=14552]: Got next segment. Inline: true Aug 7 23:47:29.298758297 (14h7m51.042815158s) [upload=14552]: Waiting for error group managing segments... Aug 7 23:47:29.298818439 (14h7m51.042875289s) [upload=14552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:47:29.298823844 (14h7m51.042880694s) [upload=14552]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:47:29.298827075 (14h7m51.042883925s) [upload=14552]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:47:29.457296385 (14h7m51.201353233s) [upload=14552]: Done uploading segments... Aug 7 23:47:29.457315555 (14h7m51.201372401s) [upload=14552]: Upload finished. err: <nil> Aug 7 23:48:42.140972885 (14h9m3.885029736s) [upload=14571]: Starting upload Aug 7 23:48:42.141036695 (14h9m3.885093558s) [upload=14571]: Done starting upload Aug 7 23:48:42.141117369 (14h9m3.88517422s) [upload=14571]: Uploading segments... Aug 7 23:48:42.141203219 (14h9m3.885260088s) [upload=14571]: Got next segment. Inline: true Aug 7 23:48:42.141210076 (14h9m3.885266937s) [upload=14571]: Waiting for error group managing segments... Aug 7 23:48:42.141288577 (14h9m3.885345438s) [upload=14571]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:48:42.141294246 (14h9m3.885351107s) [upload=14571]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:48:42.141297649 (14h9m3.885354509s) [upload=14571]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:48:42.292290080 (14h9m4.036346927s) [upload=14571]: Done uploading segments... Aug 7 23:48:42.292309333 (14h9m4.036366179s) [upload=14571]: Upload finished. err: <nil> Aug 7 23:49:47.788743937 (14h10m9.532800783s) [upload=14591]: Starting upload Aug 7 23:49:47.788822734 (14h10m9.5328796s) [upload=14591]: Done starting upload Aug 7 23:49:47.788889290 (14h10m9.532946138s) [upload=14591]: Uploading segments... Aug 7 23:49:47.788950938 (14h10m9.533007798s) [upload=14591]: Got next segment. Inline: true Aug 7 23:49:47.788955912 (14h10m9.533012775s) [upload=14591]: Waiting for error group managing segments... Aug 7 23:49:47.789005080 (14h10m9.533061926s) [upload=14591]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:49:47.789009334 (14h10m9.533066194s) [upload=14591]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:49:47.789014285 (14h10m9.53307113s) [upload=14591]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:49:47.932333015 (14h10m9.676389863s) [upload=14591]: Done uploading segments... Aug 7 23:49:47.932362673 (14h10m9.676419519s) [upload=14591]: Upload finished. err: <nil> Aug 7 23:50:23.531847225 (14h10m45.275904072s) [upload=14601]: Starting upload Aug 7 23:50:23.531926184 (14h10m45.275983047s) [upload=14601]: Done starting upload Aug 7 23:50:23.531993110 (14h10m45.27604996s) [upload=14601]: Uploading segments... Aug 7 23:50:23.532035359 (14h10m45.276092227s) [upload=14601]: Got next segment. Inline: true Aug 7 23:50:23.532041373 (14h10m45.276098223s) [upload=14601]: Waiting for error group managing segments... Aug 7 23:50:23.532083061 (14h10m45.276139912s) [upload=14601]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:23.532088436 (14h10m45.276145285s) [upload=14601]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:23.532091688 (14h10m45.276148539s) [upload=14601]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:23.673405160 (14h10m45.417462011s) [upload=14601]: Done uploading segments... Aug 7 23:50:23.673423409 (14h10m45.417480259s) [upload=14601]: Upload finished. err: <nil> Aug 7 23:51:32.204614635 (14h11m53.948671502s) [upload=14620]: Starting upload Aug 7 23:51:32.204700450 (14h11m53.948757313s) [upload=14620]: Done starting upload Aug 7 23:51:32.204781575 (14h11m53.948838427s) [upload=14620]: Uploading segments... Aug 7 23:51:32.204839741 (14h11m53.94889661s) [upload=14620]: Got next segment. Inline: true Aug 7 23:51:32.204847624 (14h11m53.948904474s) [upload=14620]: Waiting for error group managing segments... Aug 7 23:51:32.204902472 (14h11m53.948959327s) [upload=14620]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:51:32.204907775 (14h11m53.948964625s) [upload=14620]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:51:32.204911177 (14h11m53.948968027s) [upload=14620]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:01.937570352 (14h12m23.681627223s) [upload=14620]: Done uploading segments... Aug 7 23:52:01.937615027 (14h12m23.681671888s) [upload=14620]: Upload finished. err: metaclient: context deadline exceeded Duration: 948.170267ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 6 out of 60 is failed (threshold=3) [...........XXX...XX.X.......................................] duration: 999.299132ms FAIL (2025-08-07 23:52:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 999.324813ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 10 out of 60 is failed (threshold=3) [......XXXXX.XX.XX.X.........................................] duration: 879.828591ms FAIL (2025-08-07 23:52:19 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 879.860597ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [............X...X.XX.......................................X] duration: 1m0.288190202s FAIL (2025-08-07 23:52:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== <nil> Aug 7 23:51:17.805308145 (14h11m39.549364991s) [upload=14617]: Starting upload Aug 7 23:51:17.805381511 (14h11m39.549438376s) [upload=14617]: Done starting upload Aug 7 23:51:17.805481161 (14h11m39.549538011s) [upload=14617]: Uploading segments... Aug 7 23:51:17.805645590 (14h11m39.549702439s) [upload=14617]: Got next segment. Inline: false Aug 7 23:51:17.805686268 (14h11m39.549743129s) [upload=14617]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:51:17.805696806 (14h11m39.549753666s) [upload=14617]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:51:37.398831220 (14h11m59.14288807s) [seg_pos={0 0}, upload=14617]: Begin upload segment... Aug 7 23:51:37.398889057 (14h11m59.142945918s) [seg_pos={0 0}, upload=14617]: Done begin upload segment. Aug 7 23:51:37.398939217 (14h11m59.142996077s) [noise=true , address=104.36.82.55:4205 , piece_id=HI7WZRDI3BHRFWQYMDR4Z4IHJUAYOA4B3ECS65MQKQZN2YDEVHZQ, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.399392520 (14h11m59.143449371s) [noise=true , address=64.49.13.169:4707 , piece_id=Y5MWKGWFH3LP2T42UHWOLNJTHV3SCKK2ZA7HRCBVHVDIG2UQEN7Q, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.399686408 (14h11m59.143743258s) [noise=true , address=64.49.13.138:4488 , piece_id=C3DZT3DLUGAFQTRQARWSBWVJ4I77QWHJGIQZTVX4W6LU5HABXAZA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.399996250 (14h11m59.14405311s) [noise=true , address=104.36.82.43:4868 , piece_id=VP5ASYQLM6PAWHV4DMBVHIRSDX7WPMXORUCQQHLXQ7UE5WUNGY7Q, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.400264269 (14h11m59.144321115s) [noise=true , address=104.36.82.53:4136 , piece_id=IOQFF6W5IC5AUDFV76ZVTKWUAIOVNNXK6WSCCINXXTZLFRIJWHWQ, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.400485298 (14h11m59.144542144s) [noise=true , address=104.36.82.61:4387 , piece_id=N3THT2DLNTVJK72OPSHHTBG34WP53NHR7L5VC6IU5EEXW7O7EPIA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.400710393 (14h11m59.144767254s) [noise=true , address=45.91.116.14:4218 , piece_id=4JCROTVISHYY7LBCTQ637ZROGDEDIJUH222CFSKVIYDBJR7BTK3Q, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.400941379 (14h11m59.14499824s) [noise=true , address=104.36.82.23:4453 , piece_id=ZKDR6X6V3GMABOUC66VFXH33QVIEC3PQSMLLVJMMOBQIFNQVOKZA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.401251698 (14h11m59.145308559s) [noise=true , address=104.36.82.42:4808 , piece_id=HBNT2DIZC4L66S7FLKXCK5XL4JDRVXTYZD6NPYYHUBC72ZHRDFKA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.403864642 (14h11m59.147921505s) [noise=true , address=104.36.82.12:4165 , piece_id=XRWHBRGQHCBTIBJUYQMW337V3ZDR2GP67ARSQVLIX5KALUD27AWA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.404219088 (14h11m59.148275951s) [noise=true , address=104.36.82.41:4765 , piece_id=K2M5MCWY2WSDKPTBPCEL5GQ3FDFQYCFUOBW5CNJUVB72ALWM3DRA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.404507731 (14h11m59.148564592s) [noise=true , address=104.36.82.45:4909 , piece_id=CEPK5VMIAWXINNVXZIDAUQMIMO42RGHBSMC5PFECCZ7TM4BZIUUA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.404771250 (14h11m59.148828112s) [noise=true , address=79.127.201.215:4349 , piece_id=NMWPQT2OMODFL4ZPUJMYMAXPQ27KIQOUXSGKJREJGV65ZLZIBY7Q, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.405722883 (14h11m59.149779746s) [noise=true , address=104.36.82.37:4674 , piece_id=CXTRIGLDCWLTVOB3TUPG6S7VAGWBNBGPTNTWZOGJFXJA3CYQQ3EA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.405966619 (14h11m59.150023478s) [noise=true , address=104.36.82.9:4039 , piece_id=TL3QGWAW4ZHOULIJRERYSPBHH5YAA4SRYQ3SMPCZ5T6O7E6SVNUQ, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.406068528 (14h11m59.150125388s) [noise=true , address=104.36.82.62:4434 , piece_id=CQPE5HJRY2LXTHJWWMYBNDHVOR5B5MBM5DGQQ7MRWXJR6M7IKOBA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.406404119 (14h11m59.150460966s) [noise=true , address=104.36.82.60:4352 , piece_id=JZ2HOZNEYEHOFN5JT67EAVLY7EQGP3B5ENHUFRI6NJCH276P72QQ, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.406657446 (14h11m59.150714311s) [noise=true , address=104.36.82.58:4272 , piece_id=5UBUOONWVORSDNUG2D3WW7VYRK4N22D6Y3Y3GNBMHF2ECZUXDAKQ, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.983166607 (14h11m59.727223467s) [upload=14617]: Next returned nil segment Aug 7 23:51:37.983195615 (14h11m59.727252481s) [upload=14617]: Waiting for error group managing segments... Aug 7 23:51:38.130379502 (14h11m59.874436364s) [noise=true , address=104.36.82.53:4136 , piece_id=IOQFF6W5IC5AUDFV76ZVTKWUAIOVNNXK6WSCCINXXTZLFRIJWHWQ, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.151652145 (14h11m59.895708992s) [noise=true , address=104.36.82.41:4765 , piece_id=K2M5MCWY2WSDKPTBPCEL5GQ3FDFQYCFUOBW5CNJUVB72ALWM3DRA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.152381420 (14h11m59.896438274s) [noise=true , address=104.36.82.58:4272 , piece_id=5UBUOONWVORSDNUG2D3WW7VYRK4N22D6Y3Y3GNBMHF2ECZUXDAKQ, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.153368721 (14h11m59.897425572s) [noise=true , address=104.36.82.55:4205 , piece_id=HI7WZRDI3BHRFWQYMDR4Z4IHJUAYOA4B3ECS65MQKQZN2YDEVHZQ, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.166206728 (14h11m59.910263574s) [noise=true , address=104.36.82.60:4352 , piece_id=JZ2HOZNEYEHOFN5JT67EAVLY7EQGP3B5ENHUFRI6NJCH276P72QQ, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.178216859 (14h11m59.922273709s) [noise=true , address=104.36.82.42:4808 , piece_id=HBNT2DIZC4L66S7FLKXCK5XL4JDRVXTYZD6NPYYHUBC72ZHRDFKA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.191290896 (14h11m59.935347742s) [noise=true , address=45.91.116.14:4218 , piece_id=4JCROTVISHYY7LBCTQ637ZROGDEDIJUH222CFSKVIYDBJR7BTK3Q, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.192130895 (14h11m59.936187748s) [noise=true , address=104.36.82.61:4387 , piece_id=N3THT2DLNTVJK72OPSHHTBG34WP53NHR7L5VC6IU5EEXW7O7EPIA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.206383044 (14h11m59.950439898s) [noise=true , address=104.36.82.12:4165 , piece_id=XRWHBRGQHCBTIBJUYQMW337V3ZDR2GP67ARSQVLIX5KALUD27AWA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.214301438 (14h11m59.958358284s) [noise=true , address=104.36.82.9:4039 , piece_id=TL3QGWAW4ZHOULIJRERYSPBHH5YAA4SRYQ3SMPCZ5T6O7E6SVNUQ, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.313332841 (14h12m0.057389691s) [noise=true , address=104.36.82.62:4434 , piece_id=CQPE5HJRY2LXTHJWWMYBNDHVOR5B5MBM5DGQQ7MRWXJR6M7IKOBA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.369163404 (14h12m0.113220261s) [noise=true , address=104.36.82.23:4453 , piece_id=ZKDR6X6V3GMABOUC66VFXH33QVIEC3PQSMLLVJMMOBQIFNQVOKZA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.468307450 (14h12m0.21236431s) [noise=true , address=104.36.82.37:4674 , piece_id=CXTRIGLDCWLTVOB3TUPG6S7VAGWBNBGPTNTWZOGJFXJA3CYQQ3EA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.539243271 (14h12m0.283300147s) [noise=true , address=79.127.201.215:4349 , piece_id=NMWPQT2OMODFL4ZPUJMYMAXPQ27KIQOUXSGKJREJGV65ZLZIBY7Q, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.539285855 (14h12m0.283342716s) [seg_pos={0 0}, upload=14617]: Segment reached optimal threshold of 14 pieces. Aug 7 23:51:38.539484535 (14h12m0.283541382s) [noise=true , address=104.36.82.45:4909 , piece_id=CEPK5VMIAWXINNVXZIDAUQMIMO42RGHBSMC5PFECCZ7TM4BZIUUA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14Pqp5cbhmd44QnikAuD5TgsKFDR3gB2qtLuVSroaWVv4U2cDh): protocol: expected piece hash; context canceled; EOF Aug 7 23:51:38.539517293 (14h12m0.283574153s) [noise=true , address=64.49.13.138:4488 , piece_id=C3DZT3DLUGAFQTRQARWSBWVJ4I77QWHJGIQZTVX4W6LU5HABXAZA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mQqzHsTJvR5wHpEGXDixk3NcRuBKxY2cL82eJMHidcxFiPCZb): protocol: expected piece hash; context canceled; EOF Aug 7 23:51:38.539590397 (14h12m0.283647259s) [noise=true , address=104.36.82.43:4868 , piece_id=VP5ASYQLM6PAWHV4DMBVHIRSDX7WPMXORUCQQHLXQ7UE5WUNGY7Q, seg_pos={0 0}, upload=14617]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124LJtYshPwu1uZJwVyjrjUMk3htyVnp9ntnZWSUb7X3bNkwU6t): protocol: expected piece hash; context canceled; EOF Aug 7 23:51:38.539646799 (14h12m0.28370366s) [noise=true , address=64.49.13.169:4707 , piece_id=Y5MWKGWFH3LP2T42UHWOLNJTHV3SCKK2ZA7HRCBVHVDIG2UQEN7Q, seg_pos={0 0}, upload=14617]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: expected piece hash; context canceled; EOF Aug 7 23:51:38.539673304 (14h12m0.283730165s) [seg_pos={0 0}, upload=14617]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:51:38.539768774 (14h12m0.283825642s) [upload=14617]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:51:38.539775704 (14h12m0.283832576s) [upload=14617]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:17.809583811 (14h12m39.553640671s) [upload=14617]: Done uploading segments... Aug 7 23:52:17.809611159 (14h12m39.55366802s) [upload=14617]: Upload finished. err: metaclient: context deadline exceeded Duration: 1m0.288217748s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [X..................................X........................] duration: 2.891552357s FAIL (2025-08-07 23:52:16 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== 294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:32:45.172127356 (13h53m6.916184241s) [upload=14294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:32:45.390149151 (13h53m7.134206011s) [seg_pos={0 0}, upload=14294]: Begin upload segment... Aug 7 23:32:45.390215910 (13h53m7.134272761s) [seg_pos={0 0}, upload=14294]: Done begin upload segment. Aug 7 23:32:45.390249193 (13h53m7.134306059s) [noise=true , address=104.36.82.19:4405 , piece_id=MHM3MWZJFTXSJT7PTOFSWHUYO3L6NIBUV6VU66XXZDH3MC3EFR2Q, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390272919 (13h53m7.134329788s) [noise=true , address=77.90.31.12:4593 , piece_id=C3RVQHFS4L7SKPFFIZ52MAQWGD7P4JNG3SNPCNY53SQUD72PZRUA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390658970 (13h53m7.13471583s) [noise=true , address=104.36.82.16:4321 , piece_id=YTF4RWV2SD6Z4UV4XRYPBRGPCTSUTC6TQKL3DHVKU65ALTNA4TEQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390852777 (13h53m7.134909659s) [noise=true , address=104.36.82.34:4566 , piece_id=3YEAUSXEL7WNZ3O7WM24Z2DKY2TDVKMRZSHMEPMNEL2LR7BY7DBQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390957991 (13h53m7.135014851s) [noise=true , address=104.36.82.47:4523 , piece_id=MJIMHSO66V7OWFFK2UALPKSI7CGGCTHNPZINDXMVIHDLENIW5LFA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.391483291 (13h53m7.135540162s) [noise=true , address=104.36.82.5:4861 , piece_id=GRDGO5LNCETQWRU3M6LACGRW5MNA2KFLXUBT5R2UH6SEGWP6RVRQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.391931094 (13h53m7.135987964s) [noise=true , address=104.36.82.10:4090 , piece_id=V6NPPVIYEH6WLIW7OMT3NDFPR7UDWIT46GW6AFCAWUJTBLC3O55Q, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392250266 (13h53m7.136307136s) [noise=true , address=104.36.82.62:4439 , piece_id=ILRD4TEFW4PSLV4EGPORG3FM6U2HB3WSL5KP3PN566FAOGWODLYQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392611689 (13h53m7.136668559s) [noise=true , address=104.36.82.54:4160 , piece_id=GDBORRYX5QBQGH7ZMMUUAV7AAR5UHZOTK5FTQUPC7YKZC2KD7I3A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392917410 (13h53m7.136974282s) [noise=true , address=104.36.82.23:4477 , piece_id=HAF6424ZCIBGSR4US5LN5ZWE5JMNMLFGUMQQ4YP2ARP7PQXBH5GQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393270536 (13h53m7.137327398s) [noise=true , address=104.36.82.44:4881 , piece_id=XXVSXG3ILFXG4IVMELHL45TRRHDFVVJMJ5LGOHU5EDHG6XXCDH6A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393582635 (13h53m7.137639498s) [noise=true , address=104.36.82.37:4665 , piece_id=RUKKGC4Y3FIVLZO2GRWJN3J6P36VWP5VSC2HCLEU3MZDDFXODJAQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393856926 (13h53m7.137913788s) [noise=true , address=104.36.82.11:4128 , piece_id=XRPULHI7BYDF2JOIASSN5QKGEAC6FJS2F5C22WIY7EKYILRHZ73A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394196364 (13h53m7.138253227s) [noise=true , address=104.36.82.38:4706 , piece_id=BIHIPQX6LRWBNFN4SDC3K3V22AMGL3RC5B4RVE7QXJFKB4XDN3BA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394484397 (13h53m7.138541257s) [noise=true , address=104.36.82.61:4381 , piece_id=TTZOSRP6YPL6P4C6PY5S4K7Z6CIZJDPX7NUNBCK73CWZPCA3HQHA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394593709 (13h53m7.138650578s) [noise=true , address=45.91.116.5:4300 , piece_id=MADJCANN2O35MKXT2PIR5PGMJ3PNTAQAJNNGOK6ISJNVAH2L7YIA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.395093013 (13h53m7.139149879s) [noise=true , address=104.36.82.22:4530 , piece_id=HMUUW5GYH6TRHYXP4YXBHD566FWQRILEHI53AVURA5QTQGG6USFA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.395447945 (13h53m7.139504805s) [noise=true , address=104.36.82.58:4293 , piece_id=I2ETFXYTHYWYJYSHTVOBQTO6ZPGNMA4D3NVKYDE2W34EH4SQE23A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.954469844 (13h53m7.698526694s) [upload=14294]: Next returned nil segment Aug 7 23:32:45.954487953 (13h53m7.69854487s) [upload=14294]: Waiting for error group managing segments... Aug 7 23:32:46.126888627 (13h53m7.870945474s) [noise=true , address=104.36.82.10:4090 , piece_id=V6NPPVIYEH6WLIW7OMT3NDFPR7UDWIT46GW6AFCAWUJTBLC3O55Q, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.132983844 (13h53m7.877040704s) [noise=true , address=104.36.82.16:4321 , piece_id=YTF4RWV2SD6Z4UV4XRYPBRGPCTSUTC6TQKL3DHVKU65ALTNA4TEQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.138277216 (13h53m7.882334067s) [noise=true , address=104.36.82.23:4477 , piece_id=HAF6424ZCIBGSR4US5LN5ZWE5JMNMLFGUMQQ4YP2ARP7PQXBH5GQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.142033326 (13h53m7.886090185s) [noise=true , address=104.36.82.34:4566 , piece_id=3YEAUSXEL7WNZ3O7WM24Z2DKY2TDVKMRZSHMEPMNEL2LR7BY7DBQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.151247206 (13h53m7.895304058s) [noise=true , address=104.36.82.22:4530 , piece_id=HMUUW5GYH6TRHYXP4YXBHD566FWQRILEHI53AVURA5QTQGG6USFA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.151981081 (13h53m7.896037932s) [noise=true , address=104.36.82.58:4293 , piece_id=I2ETFXYTHYWYJYSHTVOBQTO6ZPGNMA4D3NVKYDE2W34EH4SQE23A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.165022882 (13h53m7.909079743s) [noise=true , address=104.36.82.44:4881 , piece_id=XXVSXG3ILFXG4IVMELHL45TRRHDFVVJMJ5LGOHU5EDHG6XXCDH6A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.184418821 (13h53m7.928475667s) [noise=true , address=104.36.82.47:4523 , piece_id=MJIMHSO66V7OWFFK2UALPKSI7CGGCTHNPZINDXMVIHDLENIW5LFA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.184532669 (13h53m7.928589516s) [noise=true , address=45.91.116.5:4300 , piece_id=MADJCANN2O35MKXT2PIR5PGMJ3PNTAQAJNNGOK6ISJNVAH2L7YIA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.193375808 (13h53m7.937432656s) [noise=true , address=104.36.82.54:4160 , piece_id=GDBORRYX5QBQGH7ZMMUUAV7AAR5UHZOTK5FTQUPC7YKZC2KD7I3A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.204247501 (13h53m7.948304372s) [noise=true , address=104.36.82.38:4706 , piece_id=BIHIPQX6LRWBNFN4SDC3K3V22AMGL3RC5B4RVE7QXJFKB4XDN3BA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.235225782 (13h53m7.979282629s) [noise=true , address=104.36.82.62:4439 , piece_id=ILRD4TEFW4PSLV4EGPORG3FM6U2HB3WSL5KP3PN566FAOGWODLYQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.269129867 (13h53m8.013186721s) [noise=true , address=104.36.82.5:4861 , piece_id=GRDGO5LNCETQWRU3M6LACGRW5MNA2KFLXUBT5R2UH6SEGWP6RVRQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.278216912 (13h53m8.022273764s) [noise=true , address=104.36.82.19:4405 , piece_id=MHM3MWZJFTXSJT7PTOFSWHUYO3L6NIBUV6VU66XXZDH3MC3EFR2Q, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.278247847 (13h53m8.022304708s) [seg_pos={0 0}, upload=14294]: Segment reached optimal threshold of 14 pieces. Aug 7 23:32:46.278485612 (13h53m8.022542463s) [noise=true , address=104.36.82.11:4128 , piece_id=XRPULHI7BYDF2JOIASSN5QKGEAC6FJS2F5C22WIY7EKYILRHZ73A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wABkAtAjoYkKyNoAiaVCxSvY6WgvHvAYFPCjpNojtpiYiGtDS): protocol: expected piece hash; context canceled; EOF Aug 7 23:32:46.278511698 (13h53m8.022568566s) [noise=true , address=77.90.31.12:4593 , piece_id=C3RVQHFS4L7SKPFFIZ52MAQWGD7P4JNG3SNPCNY53SQUD72PZRUA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kSxoZb9nmG6oGNytpFTsoiGh3ZPmebVVe8TZxGzANaw7EfdeF): protocol: expected piece hash; context canceled; EOF Aug 7 23:32:46.278642807 (13h53m8.022699657s) [noise=true , address=104.36.82.61:4381 , piece_id=TTZOSRP6YPL6P4C6PY5S4K7Z6CIZJDPX7NUNBCK73CWZPCA3HQHA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zU35MWWaaC1npkP5qGcWBXAkfxGsd5UsdkM2nwX9x7wUGCDbY): protocol: write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection; write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection Aug 7 23:32:46.278650926 (13h53m8.022707786s) [noise=true , address=104.36.82.37:4665 , piece_id=RUKKGC4Y3FIVLZO2GRWJN3J6P36VWP5VSC2HCLEU3MZDDFXODJAQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jP9JrCmoQSdqrzXZxkNN1ihdFkaUMjGZss6Fh19zbL3YyWj5N): protocol: write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection; write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection Aug 7 23:32:46.278674050 (13h53m8.022730911s) [seg_pos={0 0}, upload=14294]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:32:46.278751154 (13h53m8.022808014s) [upload=14294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:32:46.278756168 (13h53m8.022813029s) [upload=14294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:32:46.498660020 (13h53m8.242716882s) [upload=14294]: Done uploading segments... Aug 7 23:32:46.498701263 (13h53m8.242758128s) [upload=14294]: Upload finished. err: <nil> Duration: 2.891586461s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 310.699468ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 23:52:15 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 310.761183ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 1.047535369s FAIL (2025-08-07 23:52:13 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== pload=14121]: Done uploading segments... Aug 7 23:22:28.505845197 (13h42m50.249902044s) [upload=14121]: Upload finished. err: <nil> Aug 7 23:23:10.377483605 (13h43m32.121540458s) [upload=14132]: Starting upload Aug 7 23:23:10.377535635 (13h43m32.121592495s) [upload=14132]: Done starting upload Aug 7 23:23:10.377603797 (13h43m32.121660647s) [upload=14132]: Uploading segments... Aug 7 23:23:10.377638935 (13h43m32.121695804s) [upload=14132]: Got next segment. Inline: true Aug 7 23:23:10.377643594 (13h43m32.121700454s) [upload=14132]: Waiting for error group managing segments... Aug 7 23:23:10.377688252 (13h43m32.121745104s) [upload=14132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:23:10.377693427 (13h43m32.121750286s) [upload=14132]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:23:10.377696805 (13h43m32.121753666s) [upload=14132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:23:10.562260724 (13h43m32.306317573s) [upload=14132]: Done uploading segments... Aug 7 23:23:10.562279228 (13h43m32.306336078s) [upload=14132]: Upload finished. err: <nil> Aug 7 23:24:07.896404858 (13h44m29.640461721s) [upload=14147]: Starting upload Aug 7 23:24:07.896490216 (13h44m29.640547079s) [upload=14147]: Done starting upload Aug 7 23:24:07.896580488 (13h44m29.640637339s) [upload=14147]: Uploading segments... Aug 7 23:24:07.896655006 (13h44m29.640711874s) [upload=14147]: Got next segment. Inline: true Aug 7 23:24:07.896660179 (13h44m29.640717039s) [upload=14147]: Waiting for error group managing segments... Aug 7 23:24:07.896714840 (13h44m29.6407717s) [upload=14147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:24:07.896719399 (13h44m29.64077626s) [upload=14147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:24:07.896722726 (13h44m29.640779586s) [upload=14147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:24:08.048898130 (13h44m29.792954991s) [upload=14147]: Done uploading segments... Aug 7 23:24:08.048952526 (13h44m29.793009386s) [upload=14147]: Upload finished. err: <nil> Aug 7 23:24:53.794754379 (13h45m15.53881125s) [upload=14162]: Starting upload Aug 7 23:24:53.794842317 (13h45m15.538899177s) [upload=14162]: Done starting upload Aug 7 23:24:53.794935964 (13h45m15.538992814s) [upload=14162]: Uploading segments... Aug 7 23:24:53.795003752 (13h45m15.539060623s) [upload=14162]: Got next segment. Inline: true Aug 7 23:24:53.795010814 (13h45m15.539067676s) [upload=14162]: Waiting for error group managing segments... Aug 7 23:24:53.795066607 (13h45m15.539123488s) [upload=14162]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:24:53.795158449 (13h45m15.5392153s) [upload=14162]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:24:53.795163166 (13h45m15.539220026s) [upload=14162]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:24:53.946994982 (13h45m15.691051833s) [upload=14162]: Done uploading segments... Aug 7 23:24:53.947013862 (13h45m15.691070711s) [upload=14162]: Upload finished. err: <nil> Aug 7 23:25:40.029856966 (13h46m1.773913836s) [upload=14177]: Starting upload Aug 7 23:25:40.029986938 (13h46m1.774043809s) [upload=14177]: Done starting upload Aug 7 23:25:40.030076796 (13h46m1.774133642s) [upload=14177]: Uploading segments... Aug 7 23:25:40.030132567 (13h46m1.774189438s) [upload=14177]: Got next segment. Inline: true Aug 7 23:25:40.030139885 (13h46m1.774196757s) [upload=14177]: Waiting for error group managing segments... Aug 7 23:25:40.030197588 (13h46m1.774254461s) [upload=14177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:25:40.030204146 (13h46m1.774261016s) [upload=14177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:25:40.030207984 (13h46m1.77426485s) [upload=14177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:25:40.184198742 (13h46m1.928255592s) [upload=14177]: Done uploading segments... Aug 7 23:25:40.184219017 (13h46m1.928275867s) [upload=14177]: Upload finished. err: <nil> Aug 7 23:26:54.625649110 (13h47m16.369705971s) [upload=14198]: Starting upload Aug 7 23:26:54.625738563 (13h47m16.369795424s) [upload=14198]: Done starting upload Aug 7 23:26:54.625842691 (13h47m16.369899539s) [upload=14198]: Uploading segments... Aug 7 23:26:54.626016839 (13h47m16.37007371s) [upload=14198]: Got next segment. Inline: true Aug 7 23:26:54.626052112 (13h47m16.370108973s) [upload=14198]: Waiting for error group managing segments... Aug 7 23:26:54.626129780 (13h47m16.370186645s) [upload=14198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:26:54.626137392 (13h47m16.370194264s) [upload=14198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:26:54.626143182 (13h47m16.370200048s) [upload=14198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:26:54.775804171 (13h47m16.519861021s) [upload=14198]: Done uploading segments... Aug 7 23:26:54.775824249 (13h47m16.519881102s) [upload=14198]: Upload finished. err: <nil> Aug 7 23:27:32.263746490 (13h47m54.007803361s) [upload=14208]: Starting upload Aug 7 23:27:32.263833655 (13h47m54.007890501s) [upload=14208]: Done starting upload Aug 7 23:27:32.263976536 (13h47m54.008033385s) [upload=14208]: Uploading segments... Aug 7 23:27:32.264070926 (13h47m54.008127796s) [upload=14208]: Got next segment. Inline: true Aug 7 23:27:32.264080356 (13h47m54.00813733s) [upload=14208]: Waiting for error group managing segments... Aug 7 23:27:32.264154657 (13h47m54.008211526s) [upload=14208]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:27:32.264163587 (13h47m54.008220454s) [upload=14208]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:27:32.264172017 (13h47m54.008228914s) [upload=14208]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:27:32.490188559 (13h47m54.23424541s) [upload=14208]: Done uploading segments... Aug 7 23:27:32.490207326 (13h47m54.234264172s) [upload=14208]: Upload finished. err: <nil> Aug 7 23:27:57.927940477 (13h48m19.671997323s) [upload=14217]: Starting upload Aug 7 23:27:57.928016423 (13h48m19.672073283s) [upload=14217]: Done starting upload Aug 7 23:27:57.928101290 (13h48m19.672158142s) [upload=14217]: Uploading segments... Aug 7 23:27:57.928155692 (13h48m19.672212566s) [upload=14217]: Got next segment. Inline: true Aug 7 23:27:57.928162009 (13h48m19.672218869s) [upload=14217]: Waiting for error group managing segments... Aug 7 23:27:57.928213294 (13h48m19.672270145s) [upload=14217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:27:57.928219212 (13h48m19.672276074s) [upload=14217]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:27:57.928223521 (13h48m19.672280383s) [upload=14217]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:27:58.087544005 (13h48m19.831600874s) [upload=14217]: Done uploading segments... Aug 7 23:27:58.087570226 (13h48m19.831627094s) [upload=14217]: Upload finished. err: <nil> Aug 7 23:29:12.765200935 (13h49m34.509257795s) [upload=14237]: Starting upload Aug 7 23:29:12.765253106 (13h49m34.509309977s) [upload=14237]: Done starting upload Aug 7 23:29:12.765324641 (13h49m34.509381499s) [upload=14237]: Uploading segments... Aug 7 23:29:12.765374187 (13h49m34.509431056s) [upload=14237]: Got next segment. Inline: true Aug 7 23:29:12.765378947 (13h49m34.509435807s) [upload=14237]: Waiting for error group managing segments... Aug 7 23:29:12.765430859 (13h49m34.50948772s) [upload=14237]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:29:12.765438106 (13h49m34.50949497s) [upload=14237]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:29:12.765443127 (13h49m34.509499988s) [upload=14237]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:29:12.926437674 (13h49m34.670494535s) [upload=14237]: Done uploading segments... Aug 7 23:29:12.926457418 (13h49m34.670514268s) [upload=14237]: Upload finished. err: <nil> Aug 7 23:29:42.225760316 (13h50m3.969817177s) [upload=14244]: Starting upload Aug 7 23:29:42.225855063 (13h50m3.969911929s) [upload=14244]: Done starting upload Aug 7 23:29:42.225962489 (13h50m3.970019355s) [upload=14244]: Uploading segments... Aug 7 23:29:42.226078484 (13h50m3.970135331s) [upload=14244]: Got next segment. Inline: true Aug 7 23:29:42.226083996 (13h50m3.970140842s) [upload=14244]: Waiting for error group managing segments... Aug 7 23:29:42.226143067 (13h50m3.970199914s) [upload=14244]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:29:42.226148511 (13h50m3.970205371s) [upload=14244]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:29:42.226153006 (13h50m3.970209865s) [upload=14244]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:29:42.401822947 (13h50m4.14587981s) [upload=14244]: Done uploading segments... Aug 7 23:29:42.401847556 (13h50m4.145904405s) [upload=14244]: Upload finished. err: <nil> Aug 7 23:30:31.529840301 (13h50m53.273897165s) [upload=14254]: Starting upload Aug 7 23:30:31.530002531 (13h50m53.274059378s) [upload=14254]: Done starting upload Aug 7 23:30:31.530201544 (13h50m53.274258391s) [upload=14254]: Uploading segments... Aug 7 23:30:31.530298120 (13h50m53.274354985s) [upload=14254]: Got next segment. Inline: true Aug 7 23:30:31.530303457 (13h50m53.274360317s) [upload=14254]: Waiting for error group managing segments... Aug 7 23:30:31.530353721 (13h50m53.274410567s) [upload=14254]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:30:31.530358927 (13h50m53.274415788s) [upload=14254]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:30:31.530363758 (13h50m53.274420619s) [upload=14254]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:30:31.696350967 (13h50m53.440407828s) [upload=14254]: Done uploading segments... Aug 7 23:30:31.696369460 (13h50m53.440426307s) [upload=14254]: Upload finished. err: <nil> Duration: 1.047647465s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 7 out of 60 is failed (threshold=3) [....XX..XXXXX...............................................] duration: 878.563865ms FAIL (2025-08-07 23:52:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== :10:03.085332650 (13h30m24.829389513s) [upload=13904]: Done starting upload Aug 7 23:10:03.085414170 (13h30m24.829471025s) [upload=13904]: Uploading segments... Aug 7 23:10:03.085475435 (13h30m24.829532308s) [upload=13904]: Got next segment. Inline: true Aug 7 23:10:03.085480843 (13h30m24.829537706s) [upload=13904]: Waiting for error group managing segments... Aug 7 23:10:03.085544347 (13h30m24.829601211s) [upload=13904]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:03.085552551 (13h30m24.829609424s) [upload=13904]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:03.085555974 (13h30m24.829612844s) [upload=13904]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:10:03.247308964 (13h30m24.991365825s) [upload=13904]: Done uploading segments... Aug 7 23:10:03.247396260 (13h30m24.991453126s) [upload=13904]: Upload finished. err: <nil> Aug 7 23:10:40.522399484 (13h31m2.266456348s) [upload=13915]: Starting upload Aug 7 23:10:40.522487646 (13h31m2.266544496s) [upload=13915]: Done starting upload Aug 7 23:10:40.522571041 (13h31m2.266627902s) [upload=13915]: Uploading segments... Aug 7 23:10:40.522619425 (13h31m2.266676294s) [upload=13915]: Got next segment. Inline: true Aug 7 23:10:40.522626307 (13h31m2.266683183s) [upload=13915]: Waiting for error group managing segments... Aug 7 23:10:40.522684106 (13h31m2.266740971s) [upload=13915]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:40.522691484 (13h31m2.26674835s) [upload=13915]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:40.522696090 (13h31m2.266752961s) [upload=13915]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:40.254283342 (13h32m1.998340192s) [upload=13915]: Done uploading segments... Aug 7 23:11:40.254304548 (13h32m1.998361398s) [upload=13915]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:11:40.667902567 (13h32m2.411959413s) [upload=13933]: Starting upload Aug 7 23:11:40.667961940 (13h32m2.412018788s) [upload=13933]: Done starting upload Aug 7 23:11:40.668020524 (13h32m2.41207737s) [upload=13933]: Uploading segments... Aug 7 23:11:40.668075247 (13h32m2.412132113s) [upload=13933]: Got next segment. Inline: true Aug 7 23:11:40.668080664 (13h32m2.412137525s) [upload=13933]: Waiting for error group managing segments... Aug 7 23:11:40.668121190 (13h32m2.412178036s) [upload=13933]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:40.668125756 (13h32m2.412182616s) [upload=13933]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:40.668128431 (13h32m2.412185291s) [upload=13933]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:40.394243254 (13h33m2.138300118s) [upload=13933]: Done uploading segments... Aug 7 23:12:40.394270270 (13h33m2.138327132s) [upload=13933]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:12:40.798630406 (13h33m2.542687267s) [upload=13953]: Starting upload Aug 7 23:12:40.798725809 (13h33m2.54278266s) [upload=13953]: Done starting upload Aug 7 23:12:40.798811946 (13h33m2.542868796s) [upload=13953]: Uploading segments... Aug 7 23:12:40.798867069 (13h33m2.542923932s) [upload=13953]: Got next segment. Inline: true Aug 7 23:12:40.798873108 (13h33m2.542929959s) [upload=13953]: Waiting for error group managing segments... Aug 7 23:12:40.798918573 (13h33m2.542975422s) [upload=13953]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:40.798924603 (13h33m2.542981463s) [upload=13953]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:40.798939855 (13h33m2.542996716s) [upload=13953]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:40.942390421 (13h33m2.686447272s) [upload=13953]: Done uploading segments... Aug 7 23:12:40.942410212 (13h33m2.686467062s) [upload=13953]: Upload finished. err: <nil> Aug 7 23:13:18.137108974 (13h33m39.881165825s) [upload=13967]: Starting upload Aug 7 23:13:18.137180408 (13h33m39.881237292s) [upload=13967]: Done starting upload Aug 7 23:13:18.137258925 (13h33m39.881315786s) [upload=13967]: Uploading segments... Aug 7 23:13:18.137309635 (13h33m39.881366505s) [upload=13967]: Got next segment. Inline: true Aug 7 23:13:18.137314511 (13h33m39.881371371s) [upload=13967]: Waiting for error group managing segments... Aug 7 23:13:18.137374772 (13h33m39.881431633s) [upload=13967]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:18.137380412 (13h33m39.881437273s) [upload=13967]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:18.137383733 (13h33m39.881440593s) [upload=13967]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:18.301480519 (13h33m40.045537379s) [upload=13967]: Done uploading segments... Aug 7 23:13:18.301498918 (13h33m40.045555767s) [upload=13967]: Upload finished. err: <nil> Aug 7 23:14:13.169029066 (13h34m34.913085913s) [upload=13980]: Starting upload Aug 7 23:14:13.169095254 (13h34m34.913152101s) [upload=13980]: Done starting upload Aug 7 23:14:13.169153338 (13h34m34.913210184s) [upload=13980]: Uploading segments... Aug 7 23:14:13.169225662 (13h34m34.913282508s) [upload=13980]: Got next segment. Inline: true Aug 7 23:14:13.169230885 (13h34m34.913287744s) [upload=13980]: Waiting for error group managing segments... Aug 7 23:14:13.169272521 (13h34m34.913329381s) [upload=13980]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:13.169288952 (13h34m34.913345813s) [upload=13980]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:13.169291623 (13h34m34.913348483s) [upload=13980]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:12.895960606 (13h35m34.640017467s) [upload=13980]: Done uploading segments... Aug 7 23:15:12.895983608 (13h35m34.640040456s) [upload=13980]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:15:13.308146067 (13h35m35.052202918s) [upload=13997]: Starting upload Aug 7 23:15:13.308219271 (13h35m35.052276134s) [upload=13997]: Done starting upload Aug 7 23:15:13.308297319 (13h35m35.052354171s) [upload=13997]: Uploading segments... Aug 7 23:15:13.308360203 (13h35m35.052417072s) [upload=13997]: Got next segment. Inline: true Aug 7 23:15:13.308365548 (13h35m35.052422408s) [upload=13997]: Waiting for error group managing segments... Aug 7 23:15:13.308409794 (13h35m35.052466657s) [upload=13997]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:15:13.308426102 (13h35m35.052482963s) [upload=13997]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:15:13.308429725 (13h35m35.052486585s) [upload=13997]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:13.040589970 (13h36m34.784646843s) [upload=13997]: Done uploading segments... Aug 7 23:16:13.040614428 (13h36m34.784671274s) [upload=13997]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:16:20.767866726 (13h36m42.511923577s) [upload=14018]: Starting upload Aug 7 23:16:20.767927544 (13h36m42.511984398s) [upload=14018]: Done starting upload Aug 7 23:16:20.767993090 (13h36m42.512049942s) [upload=14018]: Uploading segments... Aug 7 23:16:20.768055054 (13h36m42.512111926s) [upload=14018]: Got next segment. Inline: true Aug 7 23:16:20.768061500 (13h36m42.512118351s) [upload=14018]: Waiting for error group managing segments... Aug 7 23:16:20.768130604 (13h36m42.512187577s) [upload=14018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:20.768139327 (13h36m42.512196192s) [upload=14018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:20.768144987 (13h36m42.512201849s) [upload=14018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:20.496845379 (13h37m42.240902231s) [upload=14018]: Done uploading segments... Aug 7 23:17:20.496873778 (13h37m42.240930632s) [upload=14018]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:17:20.916777280 (13h37m42.660834146s) [upload=14032]: Starting upload Aug 7 23:17:20.916877286 (13h37m42.660934149s) [upload=14032]: Done starting upload Aug 7 23:17:20.917002557 (13h37m42.661059408s) [upload=14032]: Uploading segments... Aug 7 23:17:20.917058206 (13h37m42.661115077s) [upload=14032]: Got next segment. Inline: true Aug 7 23:17:20.917064317 (13h37m42.661121178s) [upload=14032]: Waiting for error group managing segments... Aug 7 23:17:20.917116347 (13h37m42.661173214s) [upload=14032]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:17:20.917123150 (13h37m42.661180014s) [upload=14032]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:17:20.917127779 (13h37m42.661184648s) [upload=14032]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:18:20.640884349 (13h38m42.384941214s) [upload=14032]: Done uploading segments... Aug 7 23:18:20.640914749 (13h38m42.384971617s) [upload=14032]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:18:21.050365938 (13h38m42.79442281s) [upload=14051]: Starting upload Aug 7 23:18:21.050484720 (13h38m42.794541587s) [upload=14051]: Done starting upload Aug 7 23:18:21.050589902 (13h38m42.794646752s) [upload=14051]: Uploading segments... Aug 7 23:18:21.050663797 (13h38m42.794720667s) [upload=14051]: Got next segment. Inline: true Aug 7 23:18:21.050670451 (13h38m42.794727311s) [upload=14051]: Waiting for error group managing segments... Aug 7 23:18:21.050719413 (13h38m42.794776275s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:18:21.050725163 (13h38m42.794782023s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:18:21.050728952 (13h38m42.794785811s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:19:20.780076504 (13h39m42.524133366s) [upload=14051]: Done uploading segments... Aug 7 23:19:20.780099469 (13h39m42.52415633s) [upload=14051]: Upload finished. err: metaclient: context deadline exceeded Duration: 878.591852ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.......XXXXXXXXXX...........X............................XX.] duration: 2.639594078s FAIL (2025-08-07 23:52:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== 3:50:36.255108510 (14h10m57.999165376s) [upload=14603]: Starting upload Aug 7 23:50:36.255322966 (14h10m57.999379826s) [upload=14603]: Done starting upload Aug 7 23:50:36.255449358 (14h10m57.999506208s) [upload=14603]: Uploading segments... Aug 7 23:50:36.255733821 (14h10m57.999790682s) [upload=14603]: Got next segment. Inline: false Aug 7 23:50:36.255777460 (14h10m57.99983432s) [upload=14603]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:36.255786460 (14h10m57.999843335s) [upload=14603]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:50:46.222662351 (14h11m7.966719201s) [seg_pos={0 0}, upload=14603]: Begin upload segment... Aug 7 23:50:46.222725898 (14h11m7.966782748s) [seg_pos={0 0}, upload=14603]: Done begin upload segment. Aug 7 23:50:46.222761379 (14h11m7.966818229s) [noise=true , address=77.90.31.19:4843 , piece_id=QQ4J6V4Z5AOQWXBKTIN4RBICDZOHVEQBFBSNLCMGNOQIGH4G3MWQ, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.222784304 (14h11m7.966841173s) [noise=true , address=45.91.116.8:4047 , piece_id=VU2O4VOYKHKWXBRCVO6SXSVJH6LQGFVGQPGYWL4V3YXLJ3J2NYQA, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.223264095 (14h11m7.967320958s) [noise=true , address=45.91.116.10:4082 , piece_id=X7WFPMLU32NEDTIWIDR2Q5ITE5WFJNOAZVNBAPXA4MHIROXBX4NQ, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.223459737 (14h11m7.967516608s) [noise=true , address=77.90.31.21:4877 , piece_id=ELOZM44TJDPTQIZWLNNZA73MXEIR6S4JLI2RQP5COZJGGVZQD6PQ, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.223620726 (14h11m7.967677576s) [noise=true , address=45.91.116.13:4154 , piece_id=VWWN45HTVJ7TFPYOJZWJXEMMWISCYNGNTE43EWKLC7E5SKXRBE2Q, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.223932255 (14h11m7.967989115s) [noise=true , address=104.36.82.10:4077 , piece_id=SOH7L25B4NAUXDJJVIIYWVI3DO4PVDLU43DEQDKLN675YYITZEUA, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.226269217 (14h11m7.970326087s) [noise=true , address=77.90.31.18:4805 , piece_id=FUWPA352PW5RC7XMKH6IWRT6RJSCC4OKPXEQ4TVJGOK6ABRXSAOQ, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.226693470 (14h11m7.970750331s) [noise=true , address=104.36.82.13:4185 , piece_id=HJDTBPPPPF46GL57RGKZ6KWB2K3DWSJIH6VHLBM2DITVTIPOUPJQ, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.226998508 (14h11m7.971055361s) [noise=true , address=104.36.82.6:4873 , piece_id=TBVDCR2HUPIFTO6ABIVOIB4T2SRONTXUCKBCXDL4DDMIGXXGOSDQ, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.227341766 (14h11m7.971398634s) [noise=true , address=104.36.82.2:4731 , piece_id=YTZ45YUVWKDI24IZEOX3V5NHI6FKT6OQ7XQMKZR7YPJTJZUXHDMA, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.227669055 (14h11m7.971725906s) [noise=true , address=77.90.31.15:4697 , piece_id=QCGSJ7CYGWLL77CI4OQS3ZRSISFUU4L64I7WKZA6AE5GG6TGKS5Q, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.227978563 (14h11m7.972035425s) [noise=true , address=45.91.116.9:4262 , piece_id=IXOVV3XBACNMCZM5G56FLWO4CQTT3ZUAEDSYAQ43A5XYAM52WSEQ, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.228324525 (14h11m7.972381396s) [noise=true , address=77.90.31.14:4661 , piece_id=JBFMWO33DGX34RISBQFHNJSCVOE7KDH3MD5ZGEYOFTRJPQ5ZRXVA, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.228626760 (14h11m7.97268361s) [noise=true , address=77.90.31.19:4842 , piece_id=CX2ZWHICIWRHGFLPO6VYV3AVCLWEJPP67G6WQFECYNYJBQOYNGNA, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.228926604 (14h11m7.972983453s) [noise=true , address=104.36.82.12:4151 , piece_id=75EV52KGVGXPNUXGURVTKTDM5MIX7TUR245KDUQKY2ZHARAX2RYQ, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.229173185 (14h11m7.973230049s) [noise=true , address=104.36.82.15:4259 , piece_id=3CDYXMPWCUC4ADWBXJ5V7YD2SKODPDP4ILTPCT4GPNZHDUONZCAQ, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.838212244 (14h11m8.582269104s) [upload=14603]: Next returned nil segment Aug 7 23:50:46.838229924 (14h11m8.582286858s) [upload=14603]: Waiting for error group managing segments... Aug 7 23:50:47.031434016 (14h11m8.775490876s) [noise=true , address=104.36.82.6:4873 , piece_id=TBVDCR2HUPIFTO6ABIVOIB4T2SRONTXUCKBCXDL4DDMIGXXGOSDQ, seg_pos={0 0}, upload=14603]: Done uploading piece. err: <nil> Aug 7 23:50:47.031723107 (14h11m8.775779957s) [noise=true , address=104.36.82.2:4731 , piece_id=YTZ45YUVWKDI24IZEOX3V5NHI6FKT6OQ7XQMKZR7YPJTJZUXHDMA, seg_pos={0 0}, upload=14603]: Done uploading piece. err: <nil> Aug 7 23:50:47.087388660 (14h11m8.83144551s) [noise=true , address=77.90.31.19:4843 , piece_id=QQ4J6V4Z5AOQWXBKTIN4RBICDZOHVEQBFBSNLCMGNOQIGH4G3MWQ, seg_pos={0 0}, upload=14603]: Done uploading piece. err: <nil> Aug 7 23:50:47.093254880 (14h11m8.837311735s) [noise=true , address=77.90.31.21:4877 , piece_id=ELOZM44TJDPTQIZWLNNZA73MXEIR6S4JLI2RQP5COZJGGVZQD6PQ, seg_pos={0 0}, upload=14603]: Done uploading piece. err: <nil> Aug 7 23:50:47.094418115 (14h11m8.838474966s) [noise=true , address=45.91.116.10:4082 , piece_id=X7WFPMLU32NEDTIWIDR2Q5ITE5WFJNOAZVNBAPXA4MHIROXBX4NQ, seg_pos={0 0}, upload=14603]: Done uploading piece. err: <nil> Aug 7 23:50:47.101902932 (14h11m8.845959782s) [noise=true , address=77.90.31.14:4661 , piece_id=JBFMWO33DGX34RISBQFHNJSCVOE7KDH3MD5ZGEYOFTRJPQ5ZRXVA, seg_pos={0 0}, upload=14603]: Done uploading piece. err: <nil> Aug 7 23:50:47.110178767 (14h11m8.854235615s) [noise=true , address=104.36.82.15:4259 , piece_id=3CDYXMPWCUC4ADWBXJ5V7YD2SKODPDP4ILTPCT4GPNZHDUONZCAQ, seg_pos={0 0}, upload=14603]: Done uploading piece. err: <nil> Aug 7 23:50:47.134979354 (14h11m8.879036204s) [noise=true , address=45.91.116.8:4047 , piece_id=VU2O4VOYKHKWXBRCVO6SXSVJH6LQGFVGQPGYWL4V3YXLJ3J2NYQA, seg_pos={0 0}, upload=14603]: Done uploading piece. err: <nil> Aug 7 23:50:47.152507541 (14h11m8.896564391s) [noise=true , address=77.90.31.19:4842 , piece_id=CX2ZWHICIWRHGFLPO6VYV3AVCLWEJPP67G6WQFECYNYJBQOYNGNA, seg_pos={0 0}, upload=14603]: Done uploading piece. err: <nil> Aug 7 23:50:47.155387553 (14h11m8.8994444s) [noise=true , address=104.36.82.12:4151 , piece_id=75EV52KGVGXPNUXGURVTKTDM5MIX7TUR245KDUQKY2ZHARAX2RYQ, seg_pos={0 0}, upload=14603]: Done uploading piece. err: <nil> Aug 7 23:50:47.176134922 (14h11m8.920191794s) [noise=true , address=45.91.116.9:4262 , piece_id=IXOVV3XBACNMCZM5G56FLWO4CQTT3ZUAEDSYAQ43A5XYAM52WSEQ, seg_pos={0 0}, upload=14603]: Done uploading piece. err: <nil> Aug 7 23:50:47.185415586 (14h11m8.929472446s) [noise=true , address=45.91.116.13:4154 , piece_id=VWWN45HTVJ7TFPYOJZWJXEMMWISCYNGNTE43EWKLC7E5SKXRBE2Q, seg_pos={0 0}, upload=14603]: Done uploading piece. err: <nil> Aug 7 23:50:47.216710845 (14h11m8.960767707s) [noise=true , address=77.90.31.18:4805 , piece_id=FUWPA352PW5RC7XMKH6IWRT6RJSCC4OKPXEQ4TVJGOK6ABRXSAOQ, seg_pos={0 0}, upload=14603]: Done uploading piece. err: <nil> Aug 7 23:50:47.216759484 (14h11m8.960816346s) [seg_pos={0 0}, upload=14603]: Segment reached optimal threshold of 13 pieces. Aug 7 23:50:47.217007257 (14h11m8.961064104s) [noise=true , address=77.90.31.15:4697 , piece_id=QCGSJ7CYGWLL77CI4OQS3ZRSISFUU4L64I7WKZA6AE5GG6TGKS5Q, seg_pos={0 0}, upload=14603]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5): protocol: expected piece hash; context canceled; EOF Aug 7 23:50:47.217100205 (14h11m8.961157051s) [noise=true , address=104.36.82.10:4077 , piece_id=SOH7L25B4NAUXDJJVIIYWVI3DO4PVDLU43DEQDKLN675YYITZEUA, seg_pos={0 0}, upload=14603]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk): protocol: expected piece hash; context canceled; EOF Aug 7 23:50:47.217352856 (14h11m8.961409703s) [noise=true , address=104.36.82.13:4185 , piece_id=HJDTBPPPPF46GL57RGKZ6KWB2K3DWSJIH6VHLBM2DITVTIPOUPJQ, seg_pos={0 0}, upload=14603]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: write tcp 139.59.144.22:45180->104.36.82.13:4185: use of closed network connection; write tcp 139.59.144.22:45180->104.36.82.13:4185: use of closed network connection; piecestore: piecestore close: context canceled Aug 7 23:50:47.217384902 (14h11m8.961441749s) [seg_pos={0 0}, upload=14603]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:50:47.217465735 (14h11m8.961522581s) [upload=14603]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:50:47.217469650 (14h11m8.96152651s) [upload=14603]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:18.730966069 (14h11m40.475022919s) [upload=14603]: Done uploading segments... Aug 7 23:51:18.730997198 (14h11m40.475054047s) [upload=14603]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:51:19.197274160 (14h11m40.94133101s) [upload=14618]: Starting upload Aug 7 23:51:19.197349470 (14h11m40.941406332s) [upload=14618]: Done starting upload Aug 7 23:51:19.197428134 (14h11m40.941484984s) [upload=14618]: Uploading segments... Aug 7 23:51:19.197739515 (14h11m40.941796365s) [upload=14618]: Got next segment. Inline: false Aug 7 23:51:19.197788199 (14h11m40.94184505s) [upload=14618]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:51:19.197793793 (14h11m40.941850654s) [upload=14618]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:51:48.924407036 (14h12m10.668463905s) [upload=14618]: Next segment err: context deadline exceeded Aug 7 23:51:48.924481126 (14h12m10.668537977s) [upload=14618]: Done uploading segments... Aug 7 23:51:48.924499914 (14h12m10.668556764s) [upload=14618]: Upload finished. err: context deadline exceeded Duration: 2.639627293s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 13 out of 60 is failed (threshold=3) [.......XXX.XXXXXX..XX.X...................................X.] duration: 888.531016ms FAIL (2025-08-07 23:52:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7 23:42:52.368849329 (14h3m14.112906191s) [upload=14469]: Done uploading segments... Aug 7 23:42:52.368869169 (14h3m14.112926018s) [upload=14469]: Upload finished. err: <nil> Aug 7 23:44:01.454335967 (14h4m23.198392817s) [upload=14497]: Starting upload Aug 7 23:44:01.454411480 (14h4m23.198468329s) [upload=14497]: Done starting upload Aug 7 23:44:01.454478346 (14h4m23.198535196s) [upload=14497]: Uploading segments... Aug 7 23:44:01.454558538 (14h4m23.198615407s) [upload=14497]: Got next segment. Inline: true Aug 7 23:44:01.454565269 (14h4m23.19862212s) [upload=14497]: Waiting for error group managing segments... Aug 7 23:44:01.454634028 (14h4m23.198690879s) [upload=14497]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:44:01.454640175 (14h4m23.198697042s) [upload=14497]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:44:01.454643973 (14h4m23.198700834s) [upload=14497]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:44:01.608945362 (14h4m23.353002226s) [upload=14497]: Done uploading segments... Aug 7 23:44:01.608968737 (14h4m23.353025586s) [upload=14497]: Upload finished. err: <nil> Aug 7 23:44:17.431921627 (14h4m39.175978475s) [upload=14500]: Starting upload Aug 7 23:44:17.431986941 (14h4m39.176043788s) [upload=14500]: Done starting upload Aug 7 23:44:17.432052901 (14h4m39.176109747s) [upload=14500]: Uploading segments... Aug 7 23:44:17.432152156 (14h4m39.176209002s) [upload=14500]: Got next segment. Inline: true Aug 7 23:44:17.432157449 (14h4m39.176214316s) [upload=14500]: Waiting for error group managing segments... Aug 7 23:44:17.432211045 (14h4m39.176267891s) [upload=14500]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:44:17.432215731 (14h4m39.176272598s) [upload=14500]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:44:17.432218553 (14h4m39.176275416s) [upload=14500]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:44:17.574187435 (14h4m39.318244286s) [upload=14500]: Done uploading segments... Aug 7 23:44:17.574205206 (14h4m39.31826206s) [upload=14500]: Upload finished. err: <nil> Aug 7 23:45:13.608437918 (14h5m35.352494778s) [upload=14514]: Starting upload Aug 7 23:45:13.608496969 (14h5m35.352553819s) [upload=14514]: Done starting upload Aug 7 23:45:13.608551220 (14h5m35.352608068s) [upload=14514]: Uploading segments... Aug 7 23:45:13.608624509 (14h5m35.352681379s) [upload=14514]: Got next segment. Inline: true Aug 7 23:45:13.608630286 (14h5m35.352687147s) [upload=14514]: Waiting for error group managing segments... Aug 7 23:45:13.608683973 (14h5m35.352740835s) [upload=14514]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:45:13.608690269 (14h5m35.352747129s) [upload=14514]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:45:13.608692795 (14h5m35.352749655s) [upload=14514]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:45:13.754427790 (14h5m35.498484651s) [upload=14514]: Done uploading segments... Aug 7 23:45:13.754446774 (14h5m35.498503628s) [upload=14514]: Upload finished. err: <nil> Aug 7 23:46:09.008386295 (14h6m30.752443146s) [upload=14528]: Starting upload Aug 7 23:46:09.008449478 (14h6m30.752506341s) [upload=14528]: Done starting upload Aug 7 23:46:09.008515325 (14h6m30.752572176s) [upload=14528]: Uploading segments... Aug 7 23:46:09.008597175 (14h6m30.75265404s) [upload=14528]: Got next segment. Inline: true Aug 7 23:46:09.008603295 (14h6m30.752660155s) [upload=14528]: Waiting for error group managing segments... Aug 7 23:46:09.008650523 (14h6m30.752707384s) [upload=14528]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:46:09.008656863 (14h6m30.752713712s) [upload=14528]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:46:09.008662476 (14h6m30.75271934s) [upload=14528]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:46:09.155644852 (14h6m30.899701703s) [upload=14528]: Done uploading segments... Aug 7 23:46:09.155664178 (14h6m30.899721028s) [upload=14528]: Upload finished. err: <nil> Aug 7 23:46:47.072912780 (14h7m8.816969628s) [upload=14538]: Starting upload Aug 7 23:46:47.072990499 (14h7m8.817047359s) [upload=14538]: Done starting upload Aug 7 23:46:47.073066127 (14h7m8.817122973s) [upload=14538]: Uploading segments... Aug 7 23:46:47.073122515 (14h7m8.817179381s) [upload=14538]: Got next segment. Inline: true Aug 7 23:46:47.073127584 (14h7m8.817184447s) [upload=14538]: Waiting for error group managing segments... Aug 7 23:46:47.073168839 (14h7m8.817225702s) [upload=14538]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:46:47.073173325 (14h7m8.817230185s) [upload=14538]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:46:47.073176118 (14h7m8.817232978s) [upload=14538]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:46:47.210119395 (14h7m8.954176244s) [upload=14538]: Done uploading segments... Aug 7 23:46:47.210142995 (14h7m8.954199841s) [upload=14538]: Upload finished. err: <nil> Aug 7 23:47:29.298549723 (14h7m51.042606575s) [upload=14552]: Starting upload Aug 7 23:47:29.298617898 (14h7m51.042674767s) [upload=14552]: Done starting upload Aug 7 23:47:29.298698678 (14h7m51.042755528s) [upload=14552]: Uploading segments... Aug 7 23:47:29.298753306 (14h7m51.042810175s) [upload=14552]: Got next segment. Inline: true Aug 7 23:47:29.298758297 (14h7m51.042815158s) [upload=14552]: Waiting for error group managing segments... Aug 7 23:47:29.298818439 (14h7m51.042875289s) [upload=14552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:47:29.298823844 (14h7m51.042880694s) [upload=14552]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:47:29.298827075 (14h7m51.042883925s) [upload=14552]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:47:29.457296385 (14h7m51.201353233s) [upload=14552]: Done uploading segments... Aug 7 23:47:29.457315555 (14h7m51.201372401s) [upload=14552]: Upload finished. err: <nil> Aug 7 23:48:42.140972885 (14h9m3.885029736s) [upload=14571]: Starting upload Aug 7 23:48:42.141036695 (14h9m3.885093558s) [upload=14571]: Done starting upload Aug 7 23:48:42.141117369 (14h9m3.88517422s) [upload=14571]: Uploading segments... Aug 7 23:48:42.141203219 (14h9m3.885260088s) [upload=14571]: Got next segment. Inline: true Aug 7 23:48:42.141210076 (14h9m3.885266937s) [upload=14571]: Waiting for error group managing segments... Aug 7 23:48:42.141288577 (14h9m3.885345438s) [upload=14571]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:48:42.141294246 (14h9m3.885351107s) [upload=14571]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:48:42.141297649 (14h9m3.885354509s) [upload=14571]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:48:42.292290080 (14h9m4.036346927s) [upload=14571]: Done uploading segments... Aug 7 23:48:42.292309333 (14h9m4.036366179s) [upload=14571]: Upload finished. err: <nil> Aug 7 23:49:47.788743937 (14h10m9.532800783s) [upload=14591]: Starting upload Aug 7 23:49:47.788822734 (14h10m9.5328796s) [upload=14591]: Done starting upload Aug 7 23:49:47.788889290 (14h10m9.532946138s) [upload=14591]: Uploading segments... Aug 7 23:49:47.788950938 (14h10m9.533007798s) [upload=14591]: Got next segment. Inline: true Aug 7 23:49:47.788955912 (14h10m9.533012775s) [upload=14591]: Waiting for error group managing segments... Aug 7 23:49:47.789005080 (14h10m9.533061926s) [upload=14591]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:49:47.789009334 (14h10m9.533066194s) [upload=14591]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:49:47.789014285 (14h10m9.53307113s) [upload=14591]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:49:47.932333015 (14h10m9.676389863s) [upload=14591]: Done uploading segments... Aug 7 23:49:47.932362673 (14h10m9.676419519s) [upload=14591]: Upload finished. err: <nil> Aug 7 23:50:23.531847225 (14h10m45.275904072s) [upload=14601]: Starting upload Aug 7 23:50:23.531926184 (14h10m45.275983047s) [upload=14601]: Done starting upload Aug 7 23:50:23.531993110 (14h10m45.27604996s) [upload=14601]: Uploading segments... Aug 7 23:50:23.532035359 (14h10m45.276092227s) [upload=14601]: Got next segment. Inline: true Aug 7 23:50:23.532041373 (14h10m45.276098223s) [upload=14601]: Waiting for error group managing segments... Aug 7 23:50:23.532083061 (14h10m45.276139912s) [upload=14601]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:23.532088436 (14h10m45.276145285s) [upload=14601]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:23.532091688 (14h10m45.276148539s) [upload=14601]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:23.673405160 (14h10m45.417462011s) [upload=14601]: Done uploading segments... Aug 7 23:50:23.673423409 (14h10m45.417480259s) [upload=14601]: Upload finished. err: <nil> Aug 7 23:51:32.204614635 (14h11m53.948671502s) [upload=14620]: Starting upload Aug 7 23:51:32.204700450 (14h11m53.948757313s) [upload=14620]: Done starting upload Aug 7 23:51:32.204781575 (14h11m53.948838427s) [upload=14620]: Uploading segments... Aug 7 23:51:32.204839741 (14h11m53.94889661s) [upload=14620]: Got next segment. Inline: true Aug 7 23:51:32.204847624 (14h11m53.948904474s) [upload=14620]: Waiting for error group managing segments... Aug 7 23:51:32.204902472 (14h11m53.948959327s) [upload=14620]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:51:32.204907775 (14h11m53.948964625s) [upload=14620]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:51:32.204911177 (14h11m53.948968027s) [upload=14620]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:01.937570352 (14h12m23.681627223s) [upload=14620]: Done uploading segments... Aug 7 23:52:01.937615027 (14h12m23.681671888s) [upload=14620]: Upload finished. err: metaclient: context deadline exceeded Duration: 888.559043ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 13 out of 60 is failed (threshold=3) [........XXX.XXXXXX..XX.X...................................X] duration: 30.001014249s FAIL (2025-08-07 23:52:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7 23:42:52.368849329 (14h3m14.112906191s) [upload=14469]: Done uploading segments... Aug 7 23:42:52.368869169 (14h3m14.112926018s) [upload=14469]: Upload finished. err: <nil> Aug 7 23:44:01.454335967 (14h4m23.198392817s) [upload=14497]: Starting upload Aug 7 23:44:01.454411480 (14h4m23.198468329s) [upload=14497]: Done starting upload Aug 7 23:44:01.454478346 (14h4m23.198535196s) [upload=14497]: Uploading segments... Aug 7 23:44:01.454558538 (14h4m23.198615407s) [upload=14497]: Got next segment. Inline: true Aug 7 23:44:01.454565269 (14h4m23.19862212s) [upload=14497]: Waiting for error group managing segments... Aug 7 23:44:01.454634028 (14h4m23.198690879s) [upload=14497]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:44:01.454640175 (14h4m23.198697042s) [upload=14497]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:44:01.454643973 (14h4m23.198700834s) [upload=14497]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:44:01.608945362 (14h4m23.353002226s) [upload=14497]: Done uploading segments... Aug 7 23:44:01.608968737 (14h4m23.353025586s) [upload=14497]: Upload finished. err: <nil> Aug 7 23:44:17.431921627 (14h4m39.175978475s) [upload=14500]: Starting upload Aug 7 23:44:17.431986941 (14h4m39.176043788s) [upload=14500]: Done starting upload Aug 7 23:44:17.432052901 (14h4m39.176109747s) [upload=14500]: Uploading segments... Aug 7 23:44:17.432152156 (14h4m39.176209002s) [upload=14500]: Got next segment. Inline: true Aug 7 23:44:17.432157449 (14h4m39.176214316s) [upload=14500]: Waiting for error group managing segments... Aug 7 23:44:17.432211045 (14h4m39.176267891s) [upload=14500]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:44:17.432215731 (14h4m39.176272598s) [upload=14500]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:44:17.432218553 (14h4m39.176275416s) [upload=14500]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:44:17.574187435 (14h4m39.318244286s) [upload=14500]: Done uploading segments... Aug 7 23:44:17.574205206 (14h4m39.31826206s) [upload=14500]: Upload finished. err: <nil> Aug 7 23:45:13.608437918 (14h5m35.352494778s) [upload=14514]: Starting upload Aug 7 23:45:13.608496969 (14h5m35.352553819s) [upload=14514]: Done starting upload Aug 7 23:45:13.608551220 (14h5m35.352608068s) [upload=14514]: Uploading segments... Aug 7 23:45:13.608624509 (14h5m35.352681379s) [upload=14514]: Got next segment. Inline: true Aug 7 23:45:13.608630286 (14h5m35.352687147s) [upload=14514]: Waiting for error group managing segments... Aug 7 23:45:13.608683973 (14h5m35.352740835s) [upload=14514]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:45:13.608690269 (14h5m35.352747129s) [upload=14514]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:45:13.608692795 (14h5m35.352749655s) [upload=14514]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:45:13.754427790 (14h5m35.498484651s) [upload=14514]: Done uploading segments... Aug 7 23:45:13.754446774 (14h5m35.498503628s) [upload=14514]: Upload finished. err: <nil> Aug 7 23:46:09.008386295 (14h6m30.752443146s) [upload=14528]: Starting upload Aug 7 23:46:09.008449478 (14h6m30.752506341s) [upload=14528]: Done starting upload Aug 7 23:46:09.008515325 (14h6m30.752572176s) [upload=14528]: Uploading segments... Aug 7 23:46:09.008597175 (14h6m30.75265404s) [upload=14528]: Got next segment. Inline: true Aug 7 23:46:09.008603295 (14h6m30.752660155s) [upload=14528]: Waiting for error group managing segments... Aug 7 23:46:09.008650523 (14h6m30.752707384s) [upload=14528]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:46:09.008656863 (14h6m30.752713712s) [upload=14528]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:46:09.008662476 (14h6m30.75271934s) [upload=14528]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:46:09.155644852 (14h6m30.899701703s) [upload=14528]: Done uploading segments... Aug 7 23:46:09.155664178 (14h6m30.899721028s) [upload=14528]: Upload finished. err: <nil> Aug 7 23:46:47.072912780 (14h7m8.816969628s) [upload=14538]: Starting upload Aug 7 23:46:47.072990499 (14h7m8.817047359s) [upload=14538]: Done starting upload Aug 7 23:46:47.073066127 (14h7m8.817122973s) [upload=14538]: Uploading segments... Aug 7 23:46:47.073122515 (14h7m8.817179381s) [upload=14538]: Got next segment. Inline: true Aug 7 23:46:47.073127584 (14h7m8.817184447s) [upload=14538]: Waiting for error group managing segments... Aug 7 23:46:47.073168839 (14h7m8.817225702s) [upload=14538]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:46:47.073173325 (14h7m8.817230185s) [upload=14538]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:46:47.073176118 (14h7m8.817232978s) [upload=14538]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:46:47.210119395 (14h7m8.954176244s) [upload=14538]: Done uploading segments... Aug 7 23:46:47.210142995 (14h7m8.954199841s) [upload=14538]: Upload finished. err: <nil> Aug 7 23:47:29.298549723 (14h7m51.042606575s) [upload=14552]: Starting upload Aug 7 23:47:29.298617898 (14h7m51.042674767s) [upload=14552]: Done starting upload Aug 7 23:47:29.298698678 (14h7m51.042755528s) [upload=14552]: Uploading segments... Aug 7 23:47:29.298753306 (14h7m51.042810175s) [upload=14552]: Got next segment. Inline: true Aug 7 23:47:29.298758297 (14h7m51.042815158s) [upload=14552]: Waiting for error group managing segments... Aug 7 23:47:29.298818439 (14h7m51.042875289s) [upload=14552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:47:29.298823844 (14h7m51.042880694s) [upload=14552]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:47:29.298827075 (14h7m51.042883925s) [upload=14552]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:47:29.457296385 (14h7m51.201353233s) [upload=14552]: Done uploading segments... Aug 7 23:47:29.457315555 (14h7m51.201372401s) [upload=14552]: Upload finished. err: <nil> Aug 7 23:48:42.140972885 (14h9m3.885029736s) [upload=14571]: Starting upload Aug 7 23:48:42.141036695 (14h9m3.885093558s) [upload=14571]: Done starting upload Aug 7 23:48:42.141117369 (14h9m3.88517422s) [upload=14571]: Uploading segments... Aug 7 23:48:42.141203219 (14h9m3.885260088s) [upload=14571]: Got next segment. Inline: true Aug 7 23:48:42.141210076 (14h9m3.885266937s) [upload=14571]: Waiting for error group managing segments... Aug 7 23:48:42.141288577 (14h9m3.885345438s) [upload=14571]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:48:42.141294246 (14h9m3.885351107s) [upload=14571]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:48:42.141297649 (14h9m3.885354509s) [upload=14571]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:48:42.292290080 (14h9m4.036346927s) [upload=14571]: Done uploading segments... Aug 7 23:48:42.292309333 (14h9m4.036366179s) [upload=14571]: Upload finished. err: <nil> Aug 7 23:49:47.788743937 (14h10m9.532800783s) [upload=14591]: Starting upload Aug 7 23:49:47.788822734 (14h10m9.5328796s) [upload=14591]: Done starting upload Aug 7 23:49:47.788889290 (14h10m9.532946138s) [upload=14591]: Uploading segments... Aug 7 23:49:47.788950938 (14h10m9.533007798s) [upload=14591]: Got next segment. Inline: true Aug 7 23:49:47.788955912 (14h10m9.533012775s) [upload=14591]: Waiting for error group managing segments... Aug 7 23:49:47.789005080 (14h10m9.533061926s) [upload=14591]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:49:47.789009334 (14h10m9.533066194s) [upload=14591]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:49:47.789014285 (14h10m9.53307113s) [upload=14591]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:49:47.932333015 (14h10m9.676389863s) [upload=14591]: Done uploading segments... Aug 7 23:49:47.932362673 (14h10m9.676419519s) [upload=14591]: Upload finished. err: <nil> Aug 7 23:50:23.531847225 (14h10m45.275904072s) [upload=14601]: Starting upload Aug 7 23:50:23.531926184 (14h10m45.275983047s) [upload=14601]: Done starting upload Aug 7 23:50:23.531993110 (14h10m45.27604996s) [upload=14601]: Uploading segments... Aug 7 23:50:23.532035359 (14h10m45.276092227s) [upload=14601]: Got next segment. Inline: true Aug 7 23:50:23.532041373 (14h10m45.276098223s) [upload=14601]: Waiting for error group managing segments... Aug 7 23:50:23.532083061 (14h10m45.276139912s) [upload=14601]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:23.532088436 (14h10m45.276145285s) [upload=14601]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:23.532091688 (14h10m45.276148539s) [upload=14601]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:23.673405160 (14h10m45.417462011s) [upload=14601]: Done uploading segments... Aug 7 23:50:23.673423409 (14h10m45.417480259s) [upload=14601]: Upload finished. err: <nil> Aug 7 23:51:32.204614635 (14h11m53.948671502s) [upload=14620]: Starting upload Aug 7 23:51:32.204700450 (14h11m53.948757313s) [upload=14620]: Done starting upload Aug 7 23:51:32.204781575 (14h11m53.948838427s) [upload=14620]: Uploading segments... Aug 7 23:51:32.204839741 (14h11m53.94889661s) [upload=14620]: Got next segment. Inline: true Aug 7 23:51:32.204847624 (14h11m53.948904474s) [upload=14620]: Waiting for error group managing segments... Aug 7 23:51:32.204902472 (14h11m53.948959327s) [upload=14620]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:51:32.204907775 (14h11m53.948964625s) [upload=14620]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:51:32.204911177 (14h11m53.948968027s) [upload=14620]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:01.937570352 (14h12m23.681627223s) [upload=14620]: Done uploading segments... Aug 7 23:52:01.937615027 (14h12m23.681671888s) [upload=14620]: Upload finished. err: metaclient: context deadline exceeded Duration: 30.001047988s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 9 out of 60 is failed (threshold=3) [..........XXXXXX.XX........................................X] duration: 30.000736667s FAIL (2025-08-07 23:52:00 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 30.000760072s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.X..................................X.......................] duration: 3.375142421s FAIL (2025-08-07 23:51:52 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== 294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:32:45.172127356 (13h53m6.916184241s) [upload=14294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:32:45.390149151 (13h53m7.134206011s) [seg_pos={0 0}, upload=14294]: Begin upload segment... Aug 7 23:32:45.390215910 (13h53m7.134272761s) [seg_pos={0 0}, upload=14294]: Done begin upload segment. Aug 7 23:32:45.390249193 (13h53m7.134306059s) [noise=true , address=104.36.82.19:4405 , piece_id=MHM3MWZJFTXSJT7PTOFSWHUYO3L6NIBUV6VU66XXZDH3MC3EFR2Q, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390272919 (13h53m7.134329788s) [noise=true , address=77.90.31.12:4593 , piece_id=C3RVQHFS4L7SKPFFIZ52MAQWGD7P4JNG3SNPCNY53SQUD72PZRUA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390658970 (13h53m7.13471583s) [noise=true , address=104.36.82.16:4321 , piece_id=YTF4RWV2SD6Z4UV4XRYPBRGPCTSUTC6TQKL3DHVKU65ALTNA4TEQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390852777 (13h53m7.134909659s) [noise=true , address=104.36.82.34:4566 , piece_id=3YEAUSXEL7WNZ3O7WM24Z2DKY2TDVKMRZSHMEPMNEL2LR7BY7DBQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390957991 (13h53m7.135014851s) [noise=true , address=104.36.82.47:4523 , piece_id=MJIMHSO66V7OWFFK2UALPKSI7CGGCTHNPZINDXMVIHDLENIW5LFA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.391483291 (13h53m7.135540162s) [noise=true , address=104.36.82.5:4861 , piece_id=GRDGO5LNCETQWRU3M6LACGRW5MNA2KFLXUBT5R2UH6SEGWP6RVRQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.391931094 (13h53m7.135987964s) [noise=true , address=104.36.82.10:4090 , piece_id=V6NPPVIYEH6WLIW7OMT3NDFPR7UDWIT46GW6AFCAWUJTBLC3O55Q, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392250266 (13h53m7.136307136s) [noise=true , address=104.36.82.62:4439 , piece_id=ILRD4TEFW4PSLV4EGPORG3FM6U2HB3WSL5KP3PN566FAOGWODLYQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392611689 (13h53m7.136668559s) [noise=true , address=104.36.82.54:4160 , piece_id=GDBORRYX5QBQGH7ZMMUUAV7AAR5UHZOTK5FTQUPC7YKZC2KD7I3A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392917410 (13h53m7.136974282s) [noise=true , address=104.36.82.23:4477 , piece_id=HAF6424ZCIBGSR4US5LN5ZWE5JMNMLFGUMQQ4YP2ARP7PQXBH5GQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393270536 (13h53m7.137327398s) [noise=true , address=104.36.82.44:4881 , piece_id=XXVSXG3ILFXG4IVMELHL45TRRHDFVVJMJ5LGOHU5EDHG6XXCDH6A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393582635 (13h53m7.137639498s) [noise=true , address=104.36.82.37:4665 , piece_id=RUKKGC4Y3FIVLZO2GRWJN3J6P36VWP5VSC2HCLEU3MZDDFXODJAQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393856926 (13h53m7.137913788s) [noise=true , address=104.36.82.11:4128 , piece_id=XRPULHI7BYDF2JOIASSN5QKGEAC6FJS2F5C22WIY7EKYILRHZ73A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394196364 (13h53m7.138253227s) [noise=true , address=104.36.82.38:4706 , piece_id=BIHIPQX6LRWBNFN4SDC3K3V22AMGL3RC5B4RVE7QXJFKB4XDN3BA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394484397 (13h53m7.138541257s) [noise=true , address=104.36.82.61:4381 , piece_id=TTZOSRP6YPL6P4C6PY5S4K7Z6CIZJDPX7NUNBCK73CWZPCA3HQHA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394593709 (13h53m7.138650578s) [noise=true , address=45.91.116.5:4300 , piece_id=MADJCANN2O35MKXT2PIR5PGMJ3PNTAQAJNNGOK6ISJNVAH2L7YIA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.395093013 (13h53m7.139149879s) [noise=true , address=104.36.82.22:4530 , piece_id=HMUUW5GYH6TRHYXP4YXBHD566FWQRILEHI53AVURA5QTQGG6USFA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.395447945 (13h53m7.139504805s) [noise=true , address=104.36.82.58:4293 , piece_id=I2ETFXYTHYWYJYSHTVOBQTO6ZPGNMA4D3NVKYDE2W34EH4SQE23A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.954469844 (13h53m7.698526694s) [upload=14294]: Next returned nil segment Aug 7 23:32:45.954487953 (13h53m7.69854487s) [upload=14294]: Waiting for error group managing segments... Aug 7 23:32:46.126888627 (13h53m7.870945474s) [noise=true , address=104.36.82.10:4090 , piece_id=V6NPPVIYEH6WLIW7OMT3NDFPR7UDWIT46GW6AFCAWUJTBLC3O55Q, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.132983844 (13h53m7.877040704s) [noise=true , address=104.36.82.16:4321 , piece_id=YTF4RWV2SD6Z4UV4XRYPBRGPCTSUTC6TQKL3DHVKU65ALTNA4TEQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.138277216 (13h53m7.882334067s) [noise=true , address=104.36.82.23:4477 , piece_id=HAF6424ZCIBGSR4US5LN5ZWE5JMNMLFGUMQQ4YP2ARP7PQXBH5GQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.142033326 (13h53m7.886090185s) [noise=true , address=104.36.82.34:4566 , piece_id=3YEAUSXEL7WNZ3O7WM24Z2DKY2TDVKMRZSHMEPMNEL2LR7BY7DBQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.151247206 (13h53m7.895304058s) [noise=true , address=104.36.82.22:4530 , piece_id=HMUUW5GYH6TRHYXP4YXBHD566FWQRILEHI53AVURA5QTQGG6USFA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.151981081 (13h53m7.896037932s) [noise=true , address=104.36.82.58:4293 , piece_id=I2ETFXYTHYWYJYSHTVOBQTO6ZPGNMA4D3NVKYDE2W34EH4SQE23A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.165022882 (13h53m7.909079743s) [noise=true , address=104.36.82.44:4881 , piece_id=XXVSXG3ILFXG4IVMELHL45TRRHDFVVJMJ5LGOHU5EDHG6XXCDH6A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.184418821 (13h53m7.928475667s) [noise=true , address=104.36.82.47:4523 , piece_id=MJIMHSO66V7OWFFK2UALPKSI7CGGCTHNPZINDXMVIHDLENIW5LFA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.184532669 (13h53m7.928589516s) [noise=true , address=45.91.116.5:4300 , piece_id=MADJCANN2O35MKXT2PIR5PGMJ3PNTAQAJNNGOK6ISJNVAH2L7YIA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.193375808 (13h53m7.937432656s) [noise=true , address=104.36.82.54:4160 , piece_id=GDBORRYX5QBQGH7ZMMUUAV7AAR5UHZOTK5FTQUPC7YKZC2KD7I3A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.204247501 (13h53m7.948304372s) [noise=true , address=104.36.82.38:4706 , piece_id=BIHIPQX6LRWBNFN4SDC3K3V22AMGL3RC5B4RVE7QXJFKB4XDN3BA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.235225782 (13h53m7.979282629s) [noise=true , address=104.36.82.62:4439 , piece_id=ILRD4TEFW4PSLV4EGPORG3FM6U2HB3WSL5KP3PN566FAOGWODLYQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.269129867 (13h53m8.013186721s) [noise=true , address=104.36.82.5:4861 , piece_id=GRDGO5LNCETQWRU3M6LACGRW5MNA2KFLXUBT5R2UH6SEGWP6RVRQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.278216912 (13h53m8.022273764s) [noise=true , address=104.36.82.19:4405 , piece_id=MHM3MWZJFTXSJT7PTOFSWHUYO3L6NIBUV6VU66XXZDH3MC3EFR2Q, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.278247847 (13h53m8.022304708s) [seg_pos={0 0}, upload=14294]: Segment reached optimal threshold of 14 pieces. Aug 7 23:32:46.278485612 (13h53m8.022542463s) [noise=true , address=104.36.82.11:4128 , piece_id=XRPULHI7BYDF2JOIASSN5QKGEAC6FJS2F5C22WIY7EKYILRHZ73A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wABkAtAjoYkKyNoAiaVCxSvY6WgvHvAYFPCjpNojtpiYiGtDS): protocol: expected piece hash; context canceled; EOF Aug 7 23:32:46.278511698 (13h53m8.022568566s) [noise=true , address=77.90.31.12:4593 , piece_id=C3RVQHFS4L7SKPFFIZ52MAQWGD7P4JNG3SNPCNY53SQUD72PZRUA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kSxoZb9nmG6oGNytpFTsoiGh3ZPmebVVe8TZxGzANaw7EfdeF): protocol: expected piece hash; context canceled; EOF Aug 7 23:32:46.278642807 (13h53m8.022699657s) [noise=true , address=104.36.82.61:4381 , piece_id=TTZOSRP6YPL6P4C6PY5S4K7Z6CIZJDPX7NUNBCK73CWZPCA3HQHA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zU35MWWaaC1npkP5qGcWBXAkfxGsd5UsdkM2nwX9x7wUGCDbY): protocol: write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection; write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection Aug 7 23:32:46.278650926 (13h53m8.022707786s) [noise=true , address=104.36.82.37:4665 , piece_id=RUKKGC4Y3FIVLZO2GRWJN3J6P36VWP5VSC2HCLEU3MZDDFXODJAQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jP9JrCmoQSdqrzXZxkNN1ihdFkaUMjGZss6Fh19zbL3YyWj5N): protocol: write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection; write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection Aug 7 23:32:46.278674050 (13h53m8.022730911s) [seg_pos={0 0}, upload=14294]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:32:46.278751154 (13h53m8.022808014s) [upload=14294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:32:46.278756168 (13h53m8.022813029s) [upload=14294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:32:46.498660020 (13h53m8.242716882s) [upload=14294]: Done uploading segments... Aug 7 23:32:46.498701263 (13h53m8.242758128s) [upload=14294]: Upload finished. err: <nil> Duration: 3.375172479s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 6 out of 60 is failed (threshold=3) [............XXX...XX.X......................................] duration: 1.074054578s FAIL (2025-08-07 23:51:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.07408811s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......X.XX..X..............................................] duration: 2.79837961s FAIL (2025-08-07 23:51:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.798403772s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [........XXXXXXXXXX...........X............................XX] duration: 30.000763616s FAIL (2025-08-07 23:51:48 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== 3:50:36.255108510 (14h10m57.999165376s) [upload=14603]: Starting upload Aug 7 23:50:36.255322966 (14h10m57.999379826s) [upload=14603]: Done starting upload Aug 7 23:50:36.255449358 (14h10m57.999506208s) [upload=14603]: Uploading segments... Aug 7 23:50:36.255733821 (14h10m57.999790682s) [upload=14603]: Got next segment. Inline: false Aug 7 23:50:36.255777460 (14h10m57.99983432s) [upload=14603]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:36.255786460 (14h10m57.999843335s) [upload=14603]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:50:46.222662351 (14h11m7.966719201s) [seg_pos={0 0}, upload=14603]: Begin upload segment... Aug 7 23:50:46.222725898 (14h11m7.966782748s) [seg_pos={0 0}, upload=14603]: Done begin upload segment. Aug 7 23:50:46.222761379 (14h11m7.966818229s) [noise=true , address=77.90.31.19:4843 , piece_id=QQ4J6V4Z5AOQWXBKTIN4RBICDZOHVEQBFBSNLCMGNOQIGH4G3MWQ, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.222784304 (14h11m7.966841173s) [noise=true , address=45.91.116.8:4047 , piece_id=VU2O4VOYKHKWXBRCVO6SXSVJH6LQGFVGQPGYWL4V3YXLJ3J2NYQA, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.223264095 (14h11m7.967320958s) [noise=true , address=45.91.116.10:4082 , piece_id=X7WFPMLU32NEDTIWIDR2Q5ITE5WFJNOAZVNBAPXA4MHIROXBX4NQ, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.223459737 (14h11m7.967516608s) [noise=true , address=77.90.31.21:4877 , piece_id=ELOZM44TJDPTQIZWLNNZA73MXEIR6S4JLI2RQP5COZJGGVZQD6PQ, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.223620726 (14h11m7.967677576s) [noise=true , address=45.91.116.13:4154 , piece_id=VWWN45HTVJ7TFPYOJZWJXEMMWISCYNGNTE43EWKLC7E5SKXRBE2Q, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.223932255 (14h11m7.967989115s) [noise=true , address=104.36.82.10:4077 , piece_id=SOH7L25B4NAUXDJJVIIYWVI3DO4PVDLU43DEQDKLN675YYITZEUA, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.226269217 (14h11m7.970326087s) [noise=true , address=77.90.31.18:4805 , piece_id=FUWPA352PW5RC7XMKH6IWRT6RJSCC4OKPXEQ4TVJGOK6ABRXSAOQ, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.226693470 (14h11m7.970750331s) [noise=true , address=104.36.82.13:4185 , piece_id=HJDTBPPPPF46GL57RGKZ6KWB2K3DWSJIH6VHLBM2DITVTIPOUPJQ, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.226998508 (14h11m7.971055361s) [noise=true , address=104.36.82.6:4873 , piece_id=TBVDCR2HUPIFTO6ABIVOIB4T2SRONTXUCKBCXDL4DDMIGXXGOSDQ, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.227341766 (14h11m7.971398634s) [noise=true , address=104.36.82.2:4731 , piece_id=YTZ45YUVWKDI24IZEOX3V5NHI6FKT6OQ7XQMKZR7YPJTJZUXHDMA, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.227669055 (14h11m7.971725906s) [noise=true , address=77.90.31.15:4697 , piece_id=QCGSJ7CYGWLL77CI4OQS3ZRSISFUU4L64I7WKZA6AE5GG6TGKS5Q, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.227978563 (14h11m7.972035425s) [noise=true , address=45.91.116.9:4262 , piece_id=IXOVV3XBACNMCZM5G56FLWO4CQTT3ZUAEDSYAQ43A5XYAM52WSEQ, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.228324525 (14h11m7.972381396s) [noise=true , address=77.90.31.14:4661 , piece_id=JBFMWO33DGX34RISBQFHNJSCVOE7KDH3MD5ZGEYOFTRJPQ5ZRXVA, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.228626760 (14h11m7.97268361s) [noise=true , address=77.90.31.19:4842 , piece_id=CX2ZWHICIWRHGFLPO6VYV3AVCLWEJPP67G6WQFECYNYJBQOYNGNA, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.228926604 (14h11m7.972983453s) [noise=true , address=104.36.82.12:4151 , piece_id=75EV52KGVGXPNUXGURVTKTDM5MIX7TUR245KDUQKY2ZHARAX2RYQ, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.229173185 (14h11m7.973230049s) [noise=true , address=104.36.82.15:4259 , piece_id=3CDYXMPWCUC4ADWBXJ5V7YD2SKODPDP4ILTPCT4GPNZHDUONZCAQ, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.838212244 (14h11m8.582269104s) [upload=14603]: Next returned nil segment Aug 7 23:50:46.838229924 (14h11m8.582286858s) [upload=14603]: Waiting for error group managing segments... Aug 7 23:50:47.031434016 (14h11m8.775490876s) [noise=true , address=104.36.82.6:4873 , piece_id=TBVDCR2HUPIFTO6ABIVOIB4T2SRONTXUCKBCXDL4DDMIGXXGOSDQ, seg_pos={0 0}, upload=14603]: Done uploading piece. err: <nil> Aug 7 23:50:47.031723107 (14h11m8.775779957s) [noise=true , address=104.36.82.2:4731 , piece_id=YTZ45YUVWKDI24IZEOX3V5NHI6FKT6OQ7XQMKZR7YPJTJZUXHDMA, seg_pos={0 0}, upload=14603]: Done uploading piece. err: <nil> Aug 7 23:50:47.087388660 (14h11m8.83144551s) [noise=true , address=77.90.31.19:4843 , piece_id=QQ4J6V4Z5AOQWXBKTIN4RBICDZOHVEQBFBSNLCMGNOQIGH4G3MWQ, seg_pos={0 0}, upload=14603]: Done uploading piece. err: <nil> Aug 7 23:50:47.093254880 (14h11m8.837311735s) [noise=true , address=77.90.31.21:4877 , piece_id=ELOZM44TJDPTQIZWLNNZA73MXEIR6S4JLI2RQP5COZJGGVZQD6PQ, seg_pos={0 0}, upload=14603]: Done uploading piece. err: <nil> Aug 7 23:50:47.094418115 (14h11m8.838474966s) [noise=true , address=45.91.116.10:4082 , piece_id=X7WFPMLU32NEDTIWIDR2Q5ITE5WFJNOAZVNBAPXA4MHIROXBX4NQ, seg_pos={0 0}, upload=14603]: Done uploading piece. err: <nil> Aug 7 23:50:47.101902932 (14h11m8.845959782s) [noise=true , address=77.90.31.14:4661 , piece_id=JBFMWO33DGX34RISBQFHNJSCVOE7KDH3MD5ZGEYOFTRJPQ5ZRXVA, seg_pos={0 0}, upload=14603]: Done uploading piece. err: <nil> Aug 7 23:50:47.110178767 (14h11m8.854235615s) [noise=true , address=104.36.82.15:4259 , piece_id=3CDYXMPWCUC4ADWBXJ5V7YD2SKODPDP4ILTPCT4GPNZHDUONZCAQ, seg_pos={0 0}, upload=14603]: Done uploading piece. err: <nil> Aug 7 23:50:47.134979354 (14h11m8.879036204s) [noise=true , address=45.91.116.8:4047 , piece_id=VU2O4VOYKHKWXBRCVO6SXSVJH6LQGFVGQPGYWL4V3YXLJ3J2NYQA, seg_pos={0 0}, upload=14603]: Done uploading piece. err: <nil> Aug 7 23:50:47.152507541 (14h11m8.896564391s) [noise=true , address=77.90.31.19:4842 , piece_id=CX2ZWHICIWRHGFLPO6VYV3AVCLWEJPP67G6WQFECYNYJBQOYNGNA, seg_pos={0 0}, upload=14603]: Done uploading piece. err: <nil> Aug 7 23:50:47.155387553 (14h11m8.8994444s) [noise=true , address=104.36.82.12:4151 , piece_id=75EV52KGVGXPNUXGURVTKTDM5MIX7TUR245KDUQKY2ZHARAX2RYQ, seg_pos={0 0}, upload=14603]: Done uploading piece. err: <nil> Aug 7 23:50:47.176134922 (14h11m8.920191794s) [noise=true , address=45.91.116.9:4262 , piece_id=IXOVV3XBACNMCZM5G56FLWO4CQTT3ZUAEDSYAQ43A5XYAM52WSEQ, seg_pos={0 0}, upload=14603]: Done uploading piece. err: <nil> Aug 7 23:50:47.185415586 (14h11m8.929472446s) [noise=true , address=45.91.116.13:4154 , piece_id=VWWN45HTVJ7TFPYOJZWJXEMMWISCYNGNTE43EWKLC7E5SKXRBE2Q, seg_pos={0 0}, upload=14603]: Done uploading piece. err: <nil> Aug 7 23:50:47.216710845 (14h11m8.960767707s) [noise=true , address=77.90.31.18:4805 , piece_id=FUWPA352PW5RC7XMKH6IWRT6RJSCC4OKPXEQ4TVJGOK6ABRXSAOQ, seg_pos={0 0}, upload=14603]: Done uploading piece. err: <nil> Aug 7 23:50:47.216759484 (14h11m8.960816346s) [seg_pos={0 0}, upload=14603]: Segment reached optimal threshold of 13 pieces. Aug 7 23:50:47.217007257 (14h11m8.961064104s) [noise=true , address=77.90.31.15:4697 , piece_id=QCGSJ7CYGWLL77CI4OQS3ZRSISFUU4L64I7WKZA6AE5GG6TGKS5Q, seg_pos={0 0}, upload=14603]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5): protocol: expected piece hash; context canceled; EOF Aug 7 23:50:47.217100205 (14h11m8.961157051s) [noise=true , address=104.36.82.10:4077 , piece_id=SOH7L25B4NAUXDJJVIIYWVI3DO4PVDLU43DEQDKLN675YYITZEUA, seg_pos={0 0}, upload=14603]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk): protocol: expected piece hash; context canceled; EOF Aug 7 23:50:47.217352856 (14h11m8.961409703s) [noise=true , address=104.36.82.13:4185 , piece_id=HJDTBPPPPF46GL57RGKZ6KWB2K3DWSJIH6VHLBM2DITVTIPOUPJQ, seg_pos={0 0}, upload=14603]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: write tcp 139.59.144.22:45180->104.36.82.13:4185: use of closed network connection; write tcp 139.59.144.22:45180->104.36.82.13:4185: use of closed network connection; piecestore: piecestore close: context canceled Aug 7 23:50:47.217384902 (14h11m8.961441749s) [seg_pos={0 0}, upload=14603]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:50:47.217465735 (14h11m8.961522581s) [upload=14603]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:50:47.217469650 (14h11m8.96152651s) [upload=14603]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:18.730966069 (14h11m40.475022919s) [upload=14603]: Done uploading segments... Aug 7 23:51:18.730997198 (14h11m40.475054047s) [upload=14603]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:51:19.197274160 (14h11m40.94133101s) [upload=14618]: Starting upload Aug 7 23:51:19.197349470 (14h11m40.941406332s) [upload=14618]: Done starting upload Aug 7 23:51:19.197428134 (14h11m40.941484984s) [upload=14618]: Uploading segments... Aug 7 23:51:19.197739515 (14h11m40.941796365s) [upload=14618]: Got next segment. Inline: false Aug 7 23:51:19.197788199 (14h11m40.94184505s) [upload=14618]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:51:19.197793793 (14h11m40.941850654s) [upload=14618]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:51:48.924407036 (14h12m10.668463905s) [upload=14618]: Next segment err: context deadline exceeded Aug 7 23:51:48.924481126 (14h12m10.668537977s) [upload=14618]: Done uploading segments... Aug 7 23:51:48.924499914 (14h12m10.668556764s) [upload=14618]: Upload finished. err: context deadline exceeded Duration: 30.00079606s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 7 out of 60 is failed (threshold=3) [.......X.XXXXX.X............................................] duration: 3.064269842s FAIL (2025-08-07 23:51:24 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.064297786s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [.........XXXXXXXXXX...........X............................X] duration: 42.751868475s FAIL (2025-08-07 23:51:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== dress=45.91.116.4:4937 , piece_id=2TCS4T7KMVH4P2U2CC5SIGY4ABJS3OKAWI7QZVH7JTZFWBPINOJQ, seg_pos={0 0}, upload=14586]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Aug 7 23:49:39.537928320 (14h10m1.281985172s) [seg_pos={0 0}, upload=14586]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:49:39.538005033 (14h10m1.282061894s) [upload=14586]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:49:39.538009960 (14h10m1.282066821s) [upload=14586]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:49:39.681058417 (14h10m1.425115287s) [upload=14586]: Done uploading segments... Aug 7 23:49:39.681084250 (14h10m1.425141101s) [upload=14586]: Upload finished. err: <nil> Aug 7 23:50:36.255108510 (14h10m57.999165376s) [upload=14603]: Starting upload Aug 7 23:50:36.255322966 (14h10m57.999379826s) [upload=14603]: Done starting upload Aug 7 23:50:36.255449358 (14h10m57.999506208s) [upload=14603]: Uploading segments... Aug 7 23:50:36.255733821 (14h10m57.999790682s) [upload=14603]: Got next segment. Inline: false Aug 7 23:50:36.255777460 (14h10m57.99983432s) [upload=14603]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:36.255786460 (14h10m57.999843335s) [upload=14603]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:50:46.222662351 (14h11m7.966719201s) [seg_pos={0 0}, upload=14603]: Begin upload segment... Aug 7 23:50:46.222725898 (14h11m7.966782748s) [seg_pos={0 0}, upload=14603]: Done begin upload segment. Aug 7 23:50:46.222761379 (14h11m7.966818229s) [noise=true , address=77.90.31.19:4843 , piece_id=QQ4J6V4Z5AOQWXBKTIN4RBICDZOHVEQBFBSNLCMGNOQIGH4G3MWQ, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.222784304 (14h11m7.966841173s) [noise=true , address=45.91.116.8:4047 , piece_id=VU2O4VOYKHKWXBRCVO6SXSVJH6LQGFVGQPGYWL4V3YXLJ3J2NYQA, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.223264095 (14h11m7.967320958s) [noise=true , address=45.91.116.10:4082 , piece_id=X7WFPMLU32NEDTIWIDR2Q5ITE5WFJNOAZVNBAPXA4MHIROXBX4NQ, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.223459737 (14h11m7.967516608s) [noise=true , address=77.90.31.21:4877 , piece_id=ELOZM44TJDPTQIZWLNNZA73MXEIR6S4JLI2RQP5COZJGGVZQD6PQ, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.223620726 (14h11m7.967677576s) [noise=true , address=45.91.116.13:4154 , piece_id=VWWN45HTVJ7TFPYOJZWJXEMMWISCYNGNTE43EWKLC7E5SKXRBE2Q, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.223932255 (14h11m7.967989115s) [noise=true , address=104.36.82.10:4077 , piece_id=SOH7L25B4NAUXDJJVIIYWVI3DO4PVDLU43DEQDKLN675YYITZEUA, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.226269217 (14h11m7.970326087s) [noise=true , address=77.90.31.18:4805 , piece_id=FUWPA352PW5RC7XMKH6IWRT6RJSCC4OKPXEQ4TVJGOK6ABRXSAOQ, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.226693470 (14h11m7.970750331s) [noise=true , address=104.36.82.13:4185 , piece_id=HJDTBPPPPF46GL57RGKZ6KWB2K3DWSJIH6VHLBM2DITVTIPOUPJQ, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.226998508 (14h11m7.971055361s) [noise=true , address=104.36.82.6:4873 , piece_id=TBVDCR2HUPIFTO6ABIVOIB4T2SRONTXUCKBCXDL4DDMIGXXGOSDQ, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.227341766 (14h11m7.971398634s) [noise=true , address=104.36.82.2:4731 , piece_id=YTZ45YUVWKDI24IZEOX3V5NHI6FKT6OQ7XQMKZR7YPJTJZUXHDMA, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.227669055 (14h11m7.971725906s) [noise=true , address=77.90.31.15:4697 , piece_id=QCGSJ7CYGWLL77CI4OQS3ZRSISFUU4L64I7WKZA6AE5GG6TGKS5Q, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.227978563 (14h11m7.972035425s) [noise=true , address=45.91.116.9:4262 , piece_id=IXOVV3XBACNMCZM5G56FLWO4CQTT3ZUAEDSYAQ43A5XYAM52WSEQ, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.228324525 (14h11m7.972381396s) [noise=true , address=77.90.31.14:4661 , piece_id=JBFMWO33DGX34RISBQFHNJSCVOE7KDH3MD5ZGEYOFTRJPQ5ZRXVA, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.228626760 (14h11m7.97268361s) [noise=true , address=77.90.31.19:4842 , piece_id=CX2ZWHICIWRHGFLPO6VYV3AVCLWEJPP67G6WQFECYNYJBQOYNGNA, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.228926604 (14h11m7.972983453s) [noise=true , address=104.36.82.12:4151 , piece_id=75EV52KGVGXPNUXGURVTKTDM5MIX7TUR245KDUQKY2ZHARAX2RYQ, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.229173185 (14h11m7.973230049s) [noise=true , address=104.36.82.15:4259 , piece_id=3CDYXMPWCUC4ADWBXJ5V7YD2SKODPDP4ILTPCT4GPNZHDUONZCAQ, seg_pos={0 0}, upload=14603]: Uploading piece... Aug 7 23:50:46.838212244 (14h11m8.582269104s) [upload=14603]: Next returned nil segment Aug 7 23:50:46.838229924 (14h11m8.582286858s) [upload=14603]: Waiting for error group managing segments... Aug 7 23:50:47.031434016 (14h11m8.775490876s) [noise=true , address=104.36.82.6:4873 , piece_id=TBVDCR2HUPIFTO6ABIVOIB4T2SRONTXUCKBCXDL4DDMIGXXGOSDQ, seg_pos={0 0}, upload=14603]: Done uploading piece. err: <nil> Aug 7 23:50:47.031723107 (14h11m8.775779957s) [noise=true , address=104.36.82.2:4731 , piece_id=YTZ45YUVWKDI24IZEOX3V5NHI6FKT6OQ7XQMKZR7YPJTJZUXHDMA, seg_pos={0 0}, upload=14603]: Done uploading piece. err: <nil> Aug 7 23:50:47.087388660 (14h11m8.83144551s) [noise=true , address=77.90.31.19:4843 , piece_id=QQ4J6V4Z5AOQWXBKTIN4RBICDZOHVEQBFBSNLCMGNOQIGH4G3MWQ, seg_pos={0 0}, upload=14603]: Done uploading piece. err: <nil> Aug 7 23:50:47.093254880 (14h11m8.837311735s) [noise=true , address=77.90.31.21:4877 , piece_id=ELOZM44TJDPTQIZWLNNZA73MXEIR6S4JLI2RQP5COZJGGVZQD6PQ, seg_pos={0 0}, upload=14603]: Done uploading piece. err: <nil> Aug 7 23:50:47.094418115 (14h11m8.838474966s) [noise=true , address=45.91.116.10:4082 , piece_id=X7WFPMLU32NEDTIWIDR2Q5ITE5WFJNOAZVNBAPXA4MHIROXBX4NQ, seg_pos={0 0}, upload=14603]: Done uploading piece. err: <nil> Aug 7 23:50:47.101902932 (14h11m8.845959782s) [noise=true , address=77.90.31.14:4661 , piece_id=JBFMWO33DGX34RISBQFHNJSCVOE7KDH3MD5ZGEYOFTRJPQ5ZRXVA, seg_pos={0 0}, upload=14603]: Done uploading piece. err: <nil> Aug 7 23:50:47.110178767 (14h11m8.854235615s) [noise=true , address=104.36.82.15:4259 , piece_id=3CDYXMPWCUC4ADWBXJ5V7YD2SKODPDP4ILTPCT4GPNZHDUONZCAQ, seg_pos={0 0}, upload=14603]: Done uploading piece. err: <nil> Aug 7 23:50:47.134979354 (14h11m8.879036204s) [noise=true , address=45.91.116.8:4047 , piece_id=VU2O4VOYKHKWXBRCVO6SXSVJH6LQGFVGQPGYWL4V3YXLJ3J2NYQA, seg_pos={0 0}, upload=14603]: Done uploading piece. err: <nil> Aug 7 23:50:47.152507541 (14h11m8.896564391s) [noise=true , address=77.90.31.19:4842 , piece_id=CX2ZWHICIWRHGFLPO6VYV3AVCLWEJPP67G6WQFECYNYJBQOYNGNA, seg_pos={0 0}, upload=14603]: Done uploading piece. err: <nil> Aug 7 23:50:47.155387553 (14h11m8.8994444s) [noise=true , address=104.36.82.12:4151 , piece_id=75EV52KGVGXPNUXGURVTKTDM5MIX7TUR245KDUQKY2ZHARAX2RYQ, seg_pos={0 0}, upload=14603]: Done uploading piece. err: <nil> Aug 7 23:50:47.176134922 (14h11m8.920191794s) [noise=true , address=45.91.116.9:4262 , piece_id=IXOVV3XBACNMCZM5G56FLWO4CQTT3ZUAEDSYAQ43A5XYAM52WSEQ, seg_pos={0 0}, upload=14603]: Done uploading piece. err: <nil> Aug 7 23:50:47.185415586 (14h11m8.929472446s) [noise=true , address=45.91.116.13:4154 , piece_id=VWWN45HTVJ7TFPYOJZWJXEMMWISCYNGNTE43EWKLC7E5SKXRBE2Q, seg_pos={0 0}, upload=14603]: Done uploading piece. err: <nil> Aug 7 23:50:47.216710845 (14h11m8.960767707s) [noise=true , address=77.90.31.18:4805 , piece_id=FUWPA352PW5RC7XMKH6IWRT6RJSCC4OKPXEQ4TVJGOK6ABRXSAOQ, seg_pos={0 0}, upload=14603]: Done uploading piece. err: <nil> Aug 7 23:50:47.216759484 (14h11m8.960816346s) [seg_pos={0 0}, upload=14603]: Segment reached optimal threshold of 13 pieces. Aug 7 23:50:47.217007257 (14h11m8.961064104s) [noise=true , address=77.90.31.15:4697 , piece_id=QCGSJ7CYGWLL77CI4OQS3ZRSISFUU4L64I7WKZA6AE5GG6TGKS5Q, seg_pos={0 0}, upload=14603]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5): protocol: expected piece hash; context canceled; EOF Aug 7 23:50:47.217100205 (14h11m8.961157051s) [noise=true , address=104.36.82.10:4077 , piece_id=SOH7L25B4NAUXDJJVIIYWVI3DO4PVDLU43DEQDKLN675YYITZEUA, seg_pos={0 0}, upload=14603]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk): protocol: expected piece hash; context canceled; EOF Aug 7 23:50:47.217352856 (14h11m8.961409703s) [noise=true , address=104.36.82.13:4185 , piece_id=HJDTBPPPPF46GL57RGKZ6KWB2K3DWSJIH6VHLBM2DITVTIPOUPJQ, seg_pos={0 0}, upload=14603]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: write tcp 139.59.144.22:45180->104.36.82.13:4185: use of closed network connection; write tcp 139.59.144.22:45180->104.36.82.13:4185: use of closed network connection; piecestore: piecestore close: context canceled Aug 7 23:50:47.217384902 (14h11m8.961441749s) [seg_pos={0 0}, upload=14603]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:50:47.217465735 (14h11m8.961522581s) [upload=14603]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:50:47.217469650 (14h11m8.96152651s) [upload=14603]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:18.730966069 (14h11m40.475022919s) [upload=14603]: Done uploading segments... Aug 7 23:51:18.730997198 (14h11m40.475054047s) [upload=14603]: Upload finished. err: metaclient: context deadline exceeded Duration: 42.751889905s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 10 out of 60 is failed (threshold=3) [.......XXXXX.XX.XX.X........................................] duration: 888.048223ms FAIL (2025-08-07 23:51:17 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 888.083595ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 818.606305ms FAIL (2025-08-07 23:51:14 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== pload=14121]: Done uploading segments... Aug 7 23:22:28.505845197 (13h42m50.249902044s) [upload=14121]: Upload finished. err: <nil> Aug 7 23:23:10.377483605 (13h43m32.121540458s) [upload=14132]: Starting upload Aug 7 23:23:10.377535635 (13h43m32.121592495s) [upload=14132]: Done starting upload Aug 7 23:23:10.377603797 (13h43m32.121660647s) [upload=14132]: Uploading segments... Aug 7 23:23:10.377638935 (13h43m32.121695804s) [upload=14132]: Got next segment. Inline: true Aug 7 23:23:10.377643594 (13h43m32.121700454s) [upload=14132]: Waiting for error group managing segments... Aug 7 23:23:10.377688252 (13h43m32.121745104s) [upload=14132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:23:10.377693427 (13h43m32.121750286s) [upload=14132]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:23:10.377696805 (13h43m32.121753666s) [upload=14132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:23:10.562260724 (13h43m32.306317573s) [upload=14132]: Done uploading segments... Aug 7 23:23:10.562279228 (13h43m32.306336078s) [upload=14132]: Upload finished. err: <nil> Aug 7 23:24:07.896404858 (13h44m29.640461721s) [upload=14147]: Starting upload Aug 7 23:24:07.896490216 (13h44m29.640547079s) [upload=14147]: Done starting upload Aug 7 23:24:07.896580488 (13h44m29.640637339s) [upload=14147]: Uploading segments... Aug 7 23:24:07.896655006 (13h44m29.640711874s) [upload=14147]: Got next segment. Inline: true Aug 7 23:24:07.896660179 (13h44m29.640717039s) [upload=14147]: Waiting for error group managing segments... Aug 7 23:24:07.896714840 (13h44m29.6407717s) [upload=14147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:24:07.896719399 (13h44m29.64077626s) [upload=14147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:24:07.896722726 (13h44m29.640779586s) [upload=14147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:24:08.048898130 (13h44m29.792954991s) [upload=14147]: Done uploading segments... Aug 7 23:24:08.048952526 (13h44m29.793009386s) [upload=14147]: Upload finished. err: <nil> Aug 7 23:24:53.794754379 (13h45m15.53881125s) [upload=14162]: Starting upload Aug 7 23:24:53.794842317 (13h45m15.538899177s) [upload=14162]: Done starting upload Aug 7 23:24:53.794935964 (13h45m15.538992814s) [upload=14162]: Uploading segments... Aug 7 23:24:53.795003752 (13h45m15.539060623s) [upload=14162]: Got next segment. Inline: true Aug 7 23:24:53.795010814 (13h45m15.539067676s) [upload=14162]: Waiting for error group managing segments... Aug 7 23:24:53.795066607 (13h45m15.539123488s) [upload=14162]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:24:53.795158449 (13h45m15.5392153s) [upload=14162]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:24:53.795163166 (13h45m15.539220026s) [upload=14162]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:24:53.946994982 (13h45m15.691051833s) [upload=14162]: Done uploading segments... Aug 7 23:24:53.947013862 (13h45m15.691070711s) [upload=14162]: Upload finished. err: <nil> Aug 7 23:25:40.029856966 (13h46m1.773913836s) [upload=14177]: Starting upload Aug 7 23:25:40.029986938 (13h46m1.774043809s) [upload=14177]: Done starting upload Aug 7 23:25:40.030076796 (13h46m1.774133642s) [upload=14177]: Uploading segments... Aug 7 23:25:40.030132567 (13h46m1.774189438s) [upload=14177]: Got next segment. Inline: true Aug 7 23:25:40.030139885 (13h46m1.774196757s) [upload=14177]: Waiting for error group managing segments... Aug 7 23:25:40.030197588 (13h46m1.774254461s) [upload=14177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:25:40.030204146 (13h46m1.774261016s) [upload=14177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:25:40.030207984 (13h46m1.77426485s) [upload=14177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:25:40.184198742 (13h46m1.928255592s) [upload=14177]: Done uploading segments... Aug 7 23:25:40.184219017 (13h46m1.928275867s) [upload=14177]: Upload finished. err: <nil> Aug 7 23:26:54.625649110 (13h47m16.369705971s) [upload=14198]: Starting upload Aug 7 23:26:54.625738563 (13h47m16.369795424s) [upload=14198]: Done starting upload Aug 7 23:26:54.625842691 (13h47m16.369899539s) [upload=14198]: Uploading segments... Aug 7 23:26:54.626016839 (13h47m16.37007371s) [upload=14198]: Got next segment. Inline: true Aug 7 23:26:54.626052112 (13h47m16.370108973s) [upload=14198]: Waiting for error group managing segments... Aug 7 23:26:54.626129780 (13h47m16.370186645s) [upload=14198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:26:54.626137392 (13h47m16.370194264s) [upload=14198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:26:54.626143182 (13h47m16.370200048s) [upload=14198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:26:54.775804171 (13h47m16.519861021s) [upload=14198]: Done uploading segments... Aug 7 23:26:54.775824249 (13h47m16.519881102s) [upload=14198]: Upload finished. err: <nil> Aug 7 23:27:32.263746490 (13h47m54.007803361s) [upload=14208]: Starting upload Aug 7 23:27:32.263833655 (13h47m54.007890501s) [upload=14208]: Done starting upload Aug 7 23:27:32.263976536 (13h47m54.008033385s) [upload=14208]: Uploading segments... Aug 7 23:27:32.264070926 (13h47m54.008127796s) [upload=14208]: Got next segment. Inline: true Aug 7 23:27:32.264080356 (13h47m54.00813733s) [upload=14208]: Waiting for error group managing segments... Aug 7 23:27:32.264154657 (13h47m54.008211526s) [upload=14208]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:27:32.264163587 (13h47m54.008220454s) [upload=14208]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:27:32.264172017 (13h47m54.008228914s) [upload=14208]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:27:32.490188559 (13h47m54.23424541s) [upload=14208]: Done uploading segments... Aug 7 23:27:32.490207326 (13h47m54.234264172s) [upload=14208]: Upload finished. err: <nil> Aug 7 23:27:57.927940477 (13h48m19.671997323s) [upload=14217]: Starting upload Aug 7 23:27:57.928016423 (13h48m19.672073283s) [upload=14217]: Done starting upload Aug 7 23:27:57.928101290 (13h48m19.672158142s) [upload=14217]: Uploading segments... Aug 7 23:27:57.928155692 (13h48m19.672212566s) [upload=14217]: Got next segment. Inline: true Aug 7 23:27:57.928162009 (13h48m19.672218869s) [upload=14217]: Waiting for error group managing segments... Aug 7 23:27:57.928213294 (13h48m19.672270145s) [upload=14217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:27:57.928219212 (13h48m19.672276074s) [upload=14217]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:27:57.928223521 (13h48m19.672280383s) [upload=14217]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:27:58.087544005 (13h48m19.831600874s) [upload=14217]: Done uploading segments... Aug 7 23:27:58.087570226 (13h48m19.831627094s) [upload=14217]: Upload finished. err: <nil> Aug 7 23:29:12.765200935 (13h49m34.509257795s) [upload=14237]: Starting upload Aug 7 23:29:12.765253106 (13h49m34.509309977s) [upload=14237]: Done starting upload Aug 7 23:29:12.765324641 (13h49m34.509381499s) [upload=14237]: Uploading segments... Aug 7 23:29:12.765374187 (13h49m34.509431056s) [upload=14237]: Got next segment. Inline: true Aug 7 23:29:12.765378947 (13h49m34.509435807s) [upload=14237]: Waiting for error group managing segments... Aug 7 23:29:12.765430859 (13h49m34.50948772s) [upload=14237]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:29:12.765438106 (13h49m34.50949497s) [upload=14237]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:29:12.765443127 (13h49m34.509499988s) [upload=14237]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:29:12.926437674 (13h49m34.670494535s) [upload=14237]: Done uploading segments... Aug 7 23:29:12.926457418 (13h49m34.670514268s) [upload=14237]: Upload finished. err: <nil> Aug 7 23:29:42.225760316 (13h50m3.969817177s) [upload=14244]: Starting upload Aug 7 23:29:42.225855063 (13h50m3.969911929s) [upload=14244]: Done starting upload Aug 7 23:29:42.225962489 (13h50m3.970019355s) [upload=14244]: Uploading segments... Aug 7 23:29:42.226078484 (13h50m3.970135331s) [upload=14244]: Got next segment. Inline: true Aug 7 23:29:42.226083996 (13h50m3.970140842s) [upload=14244]: Waiting for error group managing segments... Aug 7 23:29:42.226143067 (13h50m3.970199914s) [upload=14244]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:29:42.226148511 (13h50m3.970205371s) [upload=14244]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:29:42.226153006 (13h50m3.970209865s) [upload=14244]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:29:42.401822947 (13h50m4.14587981s) [upload=14244]: Done uploading segments... Aug 7 23:29:42.401847556 (13h50m4.145904405s) [upload=14244]: Upload finished. err: <nil> Aug 7 23:30:31.529840301 (13h50m53.273897165s) [upload=14254]: Starting upload Aug 7 23:30:31.530002531 (13h50m53.274059378s) [upload=14254]: Done starting upload Aug 7 23:30:31.530201544 (13h50m53.274258391s) [upload=14254]: Uploading segments... Aug 7 23:30:31.530298120 (13h50m53.274354985s) [upload=14254]: Got next segment. Inline: true Aug 7 23:30:31.530303457 (13h50m53.274360317s) [upload=14254]: Waiting for error group managing segments... Aug 7 23:30:31.530353721 (13h50m53.274410567s) [upload=14254]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:30:31.530358927 (13h50m53.274415788s) [upload=14254]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:30:31.530363758 (13h50m53.274420619s) [upload=14254]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:30:31.696350967 (13h50m53.440407828s) [upload=14254]: Done uploading segments... Aug 7 23:30:31.696369460 (13h50m53.440426307s) [upload=14254]: Upload finished. err: <nil> Duration: 818.638119ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 7 out of 60 is failed (threshold=3) [........X.XXXXX.X...........................................] duration: 12.156264171s FAIL (2025-08-07 23:51:04 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 12.156288638s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 7 out of 60 is failed (threshold=3) [.....XX..XXXXX..............................................] duration: 1.059450912s FAIL (2025-08-07 23:51:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== :10:03.085332650 (13h30m24.829389513s) [upload=13904]: Done starting upload Aug 7 23:10:03.085414170 (13h30m24.829471025s) [upload=13904]: Uploading segments... Aug 7 23:10:03.085475435 (13h30m24.829532308s) [upload=13904]: Got next segment. Inline: true Aug 7 23:10:03.085480843 (13h30m24.829537706s) [upload=13904]: Waiting for error group managing segments... Aug 7 23:10:03.085544347 (13h30m24.829601211s) [upload=13904]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:03.085552551 (13h30m24.829609424s) [upload=13904]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:03.085555974 (13h30m24.829612844s) [upload=13904]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:10:03.247308964 (13h30m24.991365825s) [upload=13904]: Done uploading segments... Aug 7 23:10:03.247396260 (13h30m24.991453126s) [upload=13904]: Upload finished. err: <nil> Aug 7 23:10:40.522399484 (13h31m2.266456348s) [upload=13915]: Starting upload Aug 7 23:10:40.522487646 (13h31m2.266544496s) [upload=13915]: Done starting upload Aug 7 23:10:40.522571041 (13h31m2.266627902s) [upload=13915]: Uploading segments... Aug 7 23:10:40.522619425 (13h31m2.266676294s) [upload=13915]: Got next segment. Inline: true Aug 7 23:10:40.522626307 (13h31m2.266683183s) [upload=13915]: Waiting for error group managing segments... Aug 7 23:10:40.522684106 (13h31m2.266740971s) [upload=13915]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:40.522691484 (13h31m2.26674835s) [upload=13915]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:40.522696090 (13h31m2.266752961s) [upload=13915]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:40.254283342 (13h32m1.998340192s) [upload=13915]: Done uploading segments... Aug 7 23:11:40.254304548 (13h32m1.998361398s) [upload=13915]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:11:40.667902567 (13h32m2.411959413s) [upload=13933]: Starting upload Aug 7 23:11:40.667961940 (13h32m2.412018788s) [upload=13933]: Done starting upload Aug 7 23:11:40.668020524 (13h32m2.41207737s) [upload=13933]: Uploading segments... Aug 7 23:11:40.668075247 (13h32m2.412132113s) [upload=13933]: Got next segment. Inline: true Aug 7 23:11:40.668080664 (13h32m2.412137525s) [upload=13933]: Waiting for error group managing segments... Aug 7 23:11:40.668121190 (13h32m2.412178036s) [upload=13933]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:40.668125756 (13h32m2.412182616s) [upload=13933]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:40.668128431 (13h32m2.412185291s) [upload=13933]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:40.394243254 (13h33m2.138300118s) [upload=13933]: Done uploading segments... Aug 7 23:12:40.394270270 (13h33m2.138327132s) [upload=13933]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:12:40.798630406 (13h33m2.542687267s) [upload=13953]: Starting upload Aug 7 23:12:40.798725809 (13h33m2.54278266s) [upload=13953]: Done starting upload Aug 7 23:12:40.798811946 (13h33m2.542868796s) [upload=13953]: Uploading segments... Aug 7 23:12:40.798867069 (13h33m2.542923932s) [upload=13953]: Got next segment. Inline: true Aug 7 23:12:40.798873108 (13h33m2.542929959s) [upload=13953]: Waiting for error group managing segments... Aug 7 23:12:40.798918573 (13h33m2.542975422s) [upload=13953]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:40.798924603 (13h33m2.542981463s) [upload=13953]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:40.798939855 (13h33m2.542996716s) [upload=13953]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:40.942390421 (13h33m2.686447272s) [upload=13953]: Done uploading segments... Aug 7 23:12:40.942410212 (13h33m2.686467062s) [upload=13953]: Upload finished. err: <nil> Aug 7 23:13:18.137108974 (13h33m39.881165825s) [upload=13967]: Starting upload Aug 7 23:13:18.137180408 (13h33m39.881237292s) [upload=13967]: Done starting upload Aug 7 23:13:18.137258925 (13h33m39.881315786s) [upload=13967]: Uploading segments... Aug 7 23:13:18.137309635 (13h33m39.881366505s) [upload=13967]: Got next segment. Inline: true Aug 7 23:13:18.137314511 (13h33m39.881371371s) [upload=13967]: Waiting for error group managing segments... Aug 7 23:13:18.137374772 (13h33m39.881431633s) [upload=13967]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:18.137380412 (13h33m39.881437273s) [upload=13967]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:18.137383733 (13h33m39.881440593s) [upload=13967]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:18.301480519 (13h33m40.045537379s) [upload=13967]: Done uploading segments... Aug 7 23:13:18.301498918 (13h33m40.045555767s) [upload=13967]: Upload finished. err: <nil> Aug 7 23:14:13.169029066 (13h34m34.913085913s) [upload=13980]: Starting upload Aug 7 23:14:13.169095254 (13h34m34.913152101s) [upload=13980]: Done starting upload Aug 7 23:14:13.169153338 (13h34m34.913210184s) [upload=13980]: Uploading segments... Aug 7 23:14:13.169225662 (13h34m34.913282508s) [upload=13980]: Got next segment. Inline: true Aug 7 23:14:13.169230885 (13h34m34.913287744s) [upload=13980]: Waiting for error group managing segments... Aug 7 23:14:13.169272521 (13h34m34.913329381s) [upload=13980]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:13.169288952 (13h34m34.913345813s) [upload=13980]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:13.169291623 (13h34m34.913348483s) [upload=13980]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:12.895960606 (13h35m34.640017467s) [upload=13980]: Done uploading segments... Aug 7 23:15:12.895983608 (13h35m34.640040456s) [upload=13980]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:15:13.308146067 (13h35m35.052202918s) [upload=13997]: Starting upload Aug 7 23:15:13.308219271 (13h35m35.052276134s) [upload=13997]: Done starting upload Aug 7 23:15:13.308297319 (13h35m35.052354171s) [upload=13997]: Uploading segments... Aug 7 23:15:13.308360203 (13h35m35.052417072s) [upload=13997]: Got next segment. Inline: true Aug 7 23:15:13.308365548 (13h35m35.052422408s) [upload=13997]: Waiting for error group managing segments... Aug 7 23:15:13.308409794 (13h35m35.052466657s) [upload=13997]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:15:13.308426102 (13h35m35.052482963s) [upload=13997]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:15:13.308429725 (13h35m35.052486585s) [upload=13997]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:13.040589970 (13h36m34.784646843s) [upload=13997]: Done uploading segments... Aug 7 23:16:13.040614428 (13h36m34.784671274s) [upload=13997]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:16:20.767866726 (13h36m42.511923577s) [upload=14018]: Starting upload Aug 7 23:16:20.767927544 (13h36m42.511984398s) [upload=14018]: Done starting upload Aug 7 23:16:20.767993090 (13h36m42.512049942s) [upload=14018]: Uploading segments... Aug 7 23:16:20.768055054 (13h36m42.512111926s) [upload=14018]: Got next segment. Inline: true Aug 7 23:16:20.768061500 (13h36m42.512118351s) [upload=14018]: Waiting for error group managing segments... Aug 7 23:16:20.768130604 (13h36m42.512187577s) [upload=14018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:20.768139327 (13h36m42.512196192s) [upload=14018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:20.768144987 (13h36m42.512201849s) [upload=14018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:20.496845379 (13h37m42.240902231s) [upload=14018]: Done uploading segments... Aug 7 23:17:20.496873778 (13h37m42.240930632s) [upload=14018]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:17:20.916777280 (13h37m42.660834146s) [upload=14032]: Starting upload Aug 7 23:17:20.916877286 (13h37m42.660934149s) [upload=14032]: Done starting upload Aug 7 23:17:20.917002557 (13h37m42.661059408s) [upload=14032]: Uploading segments... Aug 7 23:17:20.917058206 (13h37m42.661115077s) [upload=14032]: Got next segment. Inline: true Aug 7 23:17:20.917064317 (13h37m42.661121178s) [upload=14032]: Waiting for error group managing segments... Aug 7 23:17:20.917116347 (13h37m42.661173214s) [upload=14032]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:17:20.917123150 (13h37m42.661180014s) [upload=14032]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:17:20.917127779 (13h37m42.661184648s) [upload=14032]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:18:20.640884349 (13h38m42.384941214s) [upload=14032]: Done uploading segments... Aug 7 23:18:20.640914749 (13h38m42.384971617s) [upload=14032]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:18:21.050365938 (13h38m42.79442281s) [upload=14051]: Starting upload Aug 7 23:18:21.050484720 (13h38m42.794541587s) [upload=14051]: Done starting upload Aug 7 23:18:21.050589902 (13h38m42.794646752s) [upload=14051]: Uploading segments... Aug 7 23:18:21.050663797 (13h38m42.794720667s) [upload=14051]: Got next segment. Inline: true Aug 7 23:18:21.050670451 (13h38m42.794727311s) [upload=14051]: Waiting for error group managing segments... Aug 7 23:18:21.050719413 (13h38m42.794776275s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:18:21.050725163 (13h38m42.794782023s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:18:21.050728952 (13h38m42.794785811s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:19:20.780076504 (13h39m42.524133366s) [upload=14051]: Done uploading segments... Aug 7 23:19:20.780099469 (13h39m42.52415633s) [upload=14051]: Upload finished. err: metaclient: context deadline exceeded Duration: 1.059489091s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 329.775539ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 23:51:03 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 329.799903ms edge gateway vhost us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)