Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 11 out of 60 is failed (threshold=3) [XX.XX.X.........................................X.XXX.X.X...] duration: 910.961385ms FAIL (2025-08-08 00:00:57 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 910.989204ms core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 10 out of 60 is failed (threshold=3) [XXXX.X............................................XXX.X.X...] duration: 2.876521822s FAIL (2025-08-08 00:00:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.87654644s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 15 out of 60 is failed (threshold=3) [XXXXXX..XX.X...................................X...XX.X...XX] duration: 30.000747973s FAIL (2025-08-08 00:00:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 140355 (14h13m13.036197215s) [upload=14641]: Starting upload Aug 7 23:52:51.292216133 (14h13m13.036272985s) [upload=14641]: Done starting upload Aug 7 23:52:51.292307446 (14h13m13.036364307s) [upload=14641]: Uploading segments... Aug 7 23:52:51.292389087 (14h13m13.036445956s) [upload=14641]: Got next segment. Inline: true Aug 7 23:52:51.292394933 (14h13m13.036451793s) [upload=14641]: Waiting for error group managing segments... Aug 7 23:52:51.292456032 (14h13m13.036512883s) [upload=14641]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:51.292461787 (14h13m13.036518647s) [upload=14641]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:51.292467676 (14h13m13.036524537s) [upload=14641]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:51.436915828 (14h13m13.180972678s) [upload=14641]: Done uploading segments... Aug 7 23:52:51.436934458 (14h13m13.18099131s) [upload=14641]: Upload finished. err: <nil> Aug 7 23:53:10.946261153 (14h13m32.690318028s) [upload=14646]: Starting upload Aug 7 23:53:10.946350198 (14h13m32.690407065s) [upload=14646]: Done starting upload Aug 7 23:53:10.946435423 (14h13m32.690492276s) [upload=14646]: Uploading segments... Aug 7 23:53:10.946489255 (14h13m32.690546124s) [upload=14646]: Got next segment. Inline: true Aug 7 23:53:10.946497660 (14h13m32.690554519s) [upload=14646]: Waiting for error group managing segments... Aug 7 23:53:10.946542678 (14h13m32.690599529s) [upload=14646]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:10.946548267 (14h13m32.690605128s) [upload=14646]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:10.946553792 (14h13m32.690610652s) [upload=14646]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:14.554651436 (14h13m36.298708309s) [upload=14646]: Done uploading segments... Aug 7 23:53:14.554675183 (14h13m36.298732045s) [upload=14646]: Upload finished. err: <nil> Aug 7 23:53:53.584502029 (14h14m15.328558897s) [upload=14658]: Starting upload Aug 7 23:53:53.584599757 (14h14m15.328656616s) [upload=14658]: Done starting upload Aug 7 23:53:53.584702364 (14h14m15.328759218s) [upload=14658]: Uploading segments... Aug 7 23:53:53.584798610 (14h14m15.32885548s) [upload=14658]: Got next segment. Inline: true Aug 7 23:53:53.584806594 (14h14m15.328863461s) [upload=14658]: Waiting for error group managing segments... Aug 7 23:53:53.584875193 (14h14m15.328932062s) [upload=14658]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:53.584882324 (14h14m15.328939193s) [upload=14658]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:53.584901657 (14h14m15.328958528s) [upload=14658]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:23.318821682 (14h14m45.062878542s) [upload=14658]: Done uploading segments... Aug 7 23:54:23.318863033 (14h14m45.062919884s) [upload=14658]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:36.775209359 (14h14m58.519266209s) [upload=14671]: Starting upload Aug 7 23:54:36.775292235 (14h14m58.519349112s) [upload=14671]: Done starting upload Aug 7 23:54:36.775412031 (14h14m58.519468883s) [upload=14671]: Uploading segments... Aug 7 23:54:36.775480767 (14h14m58.519537636s) [upload=14671]: Got next segment. Inline: true Aug 7 23:54:36.775487320 (14h14m58.519544181s) [upload=14671]: Waiting for error group managing segments... Aug 7 23:54:36.775539752 (14h14m58.519596612s) [upload=14671]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:36.775545893 (14h14m58.519602753s) [upload=14671]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:36.775549619 (14h14m58.519606479s) [upload=14671]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:36.926276427 (14h14m58.670333277s) [upload=14671]: Done uploading segments... Aug 7 23:54:36.926295206 (14h14m58.670352057s) [upload=14671]: Upload finished. err: <nil> Aug 7 23:54:57.675347843 (14h15m19.419404694s) [upload=14676]: Starting upload Aug 7 23:54:57.675412352 (14h15m19.419469212s) [upload=14676]: Done starting upload Aug 7 23:54:57.675475670 (14h15m19.41953252s) [upload=14676]: Uploading segments... Aug 7 23:54:57.675516336 (14h15m19.419573205s) [upload=14676]: Got next segment. Inline: true Aug 7 23:54:57.675521071 (14h15m19.419577931s) [upload=14676]: Waiting for error group managing segments... Aug 7 23:54:57.675565512 (14h15m19.419622375s) [upload=14676]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:57.675574255 (14h15m19.419631202s) [upload=14676]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:57.675580694 (14h15m19.419637563s) [upload=14676]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:27.407552485 (14h15m49.151609344s) [upload=14676]: Done uploading segments... Aug 7 23:55:27.407592978 (14h15m49.151649829s) [upload=14676]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:56:06.851646232 (14h16m28.595703082s) [upload=14696]: Starting upload Aug 7 23:56:06.851717330 (14h16m28.595774191s) [upload=14696]: Done starting upload Aug 7 23:56:06.851786008 (14h16m28.59584286s) [upload=14696]: Uploading segments... Aug 7 23:56:06.851834232 (14h16m28.595891101s) [upload=14696]: Got next segment. Inline: true Aug 7 23:56:06.851839675 (14h16m28.595896535s) [upload=14696]: Waiting for error group managing segments... Aug 7 23:56:06.851896275 (14h16m28.595953125s) [upload=14696]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:06.851901659 (14h16m28.59595852s) [upload=14696]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:06.851905505 (14h16m28.595962364s) [upload=14696]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:07.006202008 (14h16m28.750258868s) [upload=14696]: Done uploading segments... Aug 7 23:56:07.006222000 (14h16m28.750278861s) [upload=14696]: Upload finished. err: <nil> Aug 7 23:56:59.020763574 (14h17m20.764820421s) [upload=14712]: Starting upload Aug 7 23:56:59.020858955 (14h17m20.764915806s) [upload=14712]: Done starting upload Aug 7 23:56:59.020935444 (14h17m20.764992295s) [upload=14712]: Uploading segments... Aug 7 23:56:59.021007501 (14h17m20.76506437s) [upload=14712]: Got next segment. Inline: true Aug 7 23:56:59.021013631 (14h17m20.765070491s) [upload=14712]: Waiting for error group managing segments... Aug 7 23:56:59.021070471 (14h17m20.76512733s) [upload=14712]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:59.021076966 (14h17m20.765133821s) [upload=14712]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:59.021081305 (14h17m20.76513816s) [upload=14712]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:59.154820708 (14h17m20.898877569s) [upload=14712]: Done uploading segments... Aug 7 23:56:59.154839835 (14h17m20.898896685s) [upload=14712]: Upload finished. err: <nil> Aug 7 23:58:10.963677657 (14h18m32.707734508s) [upload=14728]: Starting upload Aug 7 23:58:10.963770332 (14h18m32.70782718s) [upload=14728]: Done starting upload Aug 7 23:58:10.963851725 (14h18m32.707908586s) [upload=14728]: Uploading segments... Aug 7 23:58:10.963901559 (14h18m32.707958431s) [upload=14728]: Got next segment. Inline: true Aug 7 23:58:10.963907663 (14h18m32.707964525s) [upload=14728]: Waiting for error group managing segments... Aug 7 23:58:10.963960167 (14h18m32.708017032s) [upload=14728]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:10.963966034 (14h18m32.708022896s) [upload=14728]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:10.963969944 (14h18m32.708026806s) [upload=14728]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:11.106069715 (14h18m32.850126571s) [upload=14728]: Done uploading segments... Aug 7 23:58:11.106112099 (14h18m32.850168961s) [upload=14728]: Upload finished. err: <nil> Aug 7 23:59:19.078128070 (14h19m40.822184931s) [upload=14749]: Starting upload Aug 7 23:59:19.078217499 (14h19m40.822274359s) [upload=14749]: Done starting upload Aug 7 23:59:19.078291040 (14h19m40.822347895s) [upload=14749]: Uploading segments... Aug 7 23:59:19.078333863 (14h19m40.822390732s) [upload=14749]: Got next segment. Inline: true Aug 7 23:59:19.078339134 (14h19m40.822395984s) [upload=14749]: Waiting for error group managing segments... Aug 7 23:59:19.078391868 (14h19m40.822448717s) [upload=14749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:19.078397213 (14h19m40.822454064s) [upload=14749]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:19.078401054 (14h19m40.822457905s) [upload=14749]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:59:48.808547234 (14h20m10.552604081s) [upload=14749]: Done uploading segments... Aug 7 23:59:48.808568563 (14h20m10.552625409s) [upload=14749]: Upload finished. err: metaclient: context deadline exceeded Aug 8 00:00:20.896255037 (14h20m42.640311905s) [upload=14765]: Starting upload Aug 8 00:00:20.896381648 (14h20m42.640438517s) [upload=14765]: Done starting upload Aug 8 00:00:20.896532416 (14h20m42.640589282s) [upload=14765]: Uploading segments... Aug 8 00:00:20.896610121 (14h20m42.640666994s) [upload=14765]: Got next segment. Inline: true Aug 8 00:00:20.896618785 (14h20m42.640675649s) [upload=14765]: Waiting for error group managing segments... Aug 8 00:00:20.896682486 (14h20m42.640739601s) [upload=14765]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:00:20.896690306 (14h20m42.640747187s) [upload=14765]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:00:20.896694946 (14h20m42.640751814s) [upload=14765]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:50.607106781 (14h21m12.351163663s) [upload=14765]: Done uploading segments... Aug 8 00:00:50.607134684 (14h21m12.351191558s) [upload=14765]: Upload finished. err: metaclient: context deadline exceeded Duration: 30.000952768s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [XXXXXXX...........X............................XX.X.X...XX..] duration: 2.605408205s FAIL (2025-08-08 00:00:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== 6]: Uploading piece... Aug 7 23:57:10.893816491 (14h17m32.637873354s) [noise=true , address=45.91.116.12:4225 , piece_id=NTUY2VEJFWSFOAGWUPTJYNYNWL3XMK4C3FJUE6MSTMTS2TZJ7VUA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.894285524 (14h17m32.638342391s) [noise=true , address=104.36.82.16:4294 , piece_id=SIMVCA63FAX5EQZJO5TBOSGPVYLFOAICTABKMXS5SCQGNOUPFBIA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.895348932 (14h17m32.639405787s) [noise=true , address=77.90.31.14:4661 , piece_id=WFDXKM726LBL4MH4CJ4AQIUSXTQCQ3BJV34QBUGWPHNFFPZYGZKA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.895932061 (14h17m32.639988921s) [noise=true , address=104.36.82.10:4078 , piece_id=3DPLCXZLQWMZSYSSO3JPLZFP2VXJ5NKYO4FDVFYOWJLT43C7XSTQ, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.896545320 (14h17m32.640602184s) [noise=true , address=77.90.31.10:4518 , piece_id=LU6UTDDO6HUIGXV2QXCL45CIBFQMOQGS7CTQHBRKCBE2JJKLI6FA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.897117472 (14h17m32.641174341s) [noise=true , address=77.90.31.21:4877 , piece_id=SF7XDMNH7KGS26UE6MKUK3VLFZ6IOP6WZKMVO4H6SIAULM623GLA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.897621325 (14h17m32.641678192s) [noise=true , address=77.90.31.12:4589 , piece_id=5I5BMAQARX67CB2RRMCXG2U7KELJ6HSHRT3573EH3ZX6NJ257R4A, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.898101338 (14h17m32.642158204s) [noise=true , address=77.90.31.21:4878 , piece_id=XAXEYDOX54GUKMI22UKTA6BSTSY5WYATTITTGSRQ3BBYDGDCO5DQ, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.898693607 (14h17m32.642750474s) [noise=true , address=45.91.116.6:4975 , piece_id=VDEP57UE6G4CBU7DOYMMOIRXJUTTWPJKDJDGCGUOHBGVPAYCM6PA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.900512475 (14h17m32.644569335s) [noise=true , address=45.91.116.12:4227 , piece_id=J2FM3FAWY5HYXX3VFC6CSHUSWDQDNKUUB6NL6MWSF2ZG3BZFWBEA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.902337804 (14h17m32.646394669s) [noise=true , address=77.90.31.13:4626 , piece_id=PR6GY2Y7HDUORSLYZXU4IVLVA6RIMBFSRFJW4OTLXUZM7MJZ4KOQ, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.904215667 (14h17m32.648272537s) [noise=true , address=77.90.31.19:4841 , piece_id=Q3XFTIVGOSRQARSFONAKX26VAFTJF7THPIQV6USKI7N2LN3JBTLQ, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.906034481 (14h17m32.650091365s) [noise=true , address=104.36.82.5:4839 , piece_id=AV3ZBQ4W44IZEDAPHJGHKK5D5VRNU5XGOQGDWEARNGIKWRQAGYBQ, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.907857426 (14h17m32.651914295s) [noise=true , address=77.90.31.15:4698 , piece_id=WBU562MADMX3KSCRCD2H5R7MQN5IPFBLKYSKU47WMGYYBKZGPYJA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.908666188 (14h17m32.652723069s) [noise=true , address=104.36.82.14:4222 , piece_id=JFWGS6VCDTFOQYG32ZFTWNYLFGVVQPXD7K6TZXC4FR2SGEOUS7FA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:11.510238622 (14h17m33.254295484s) [upload=14716]: Next returned nil segment Aug 7 23:57:11.510261464 (14h17m33.254318324s) [upload=14716]: Waiting for error group managing segments... Aug 7 23:57:11.657094388 (14h17m33.401151236s) [noise=true , address=77.90.31.10:4518 , piece_id=LU6UTDDO6HUIGXV2QXCL45CIBFQMOQGS7CTQHBRKCBE2JJKLI6FA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.674408374 (14h17m33.418465234s) [noise=true , address=77.90.31.15:4698 , piece_id=WBU562MADMX3KSCRCD2H5R7MQN5IPFBLKYSKU47WMGYYBKZGPYJA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.681203586 (14h17m33.425260449s) [noise=true , address=77.90.31.12:4589 , piece_id=5I5BMAQARX67CB2RRMCXG2U7KELJ6HSHRT3573EH3ZX6NJ257R4A, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.682780850 (14h17m33.4268377s) [noise=true , address=77.90.31.21:4877 , piece_id=SF7XDMNH7KGS26UE6MKUK3VLFZ6IOP6WZKMVO4H6SIAULM623GLA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.684042315 (14h17m33.428099176s) [noise=true , address=104.36.82.16:4294 , piece_id=SIMVCA63FAX5EQZJO5TBOSGPVYLFOAICTABKMXS5SCQGNOUPFBIA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.719658547 (14h17m33.463715401s) [noise=true , address=45.91.116.12:4225 , piece_id=NTUY2VEJFWSFOAGWUPTJYNYNWL3XMK4C3FJUE6MSTMTS2TZJ7VUA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.730348391 (14h17m33.474405242s) [noise=true , address=77.90.31.21:4878 , piece_id=XAXEYDOX54GUKMI22UKTA6BSTSY5WYATTITTGSRQ3BBYDGDCO5DQ, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.748264399 (14h17m33.492321249s) [noise=true , address=77.90.31.19:4841 , piece_id=Q3XFTIVGOSRQARSFONAKX26VAFTJF7THPIQV6USKI7N2LN3JBTLQ, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.769764028 (14h17m33.513820878s) [noise=true , address=45.91.116.6:4975 , piece_id=VDEP57UE6G4CBU7DOYMMOIRXJUTTWPJKDJDGCGUOHBGVPAYCM6PA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.774177874 (14h17m33.518234733s) [noise=true , address=77.90.31.14:4661 , piece_id=WFDXKM726LBL4MH4CJ4AQIUSXTQCQ3BJV34QBUGWPHNFFPZYGZKA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.789864472 (14h17m33.533921326s) [noise=true , address=45.91.116.13:4153 , piece_id=WBFFQT5I6ZR5ZKUJRFULNOJZ455KIJMA5EEATDDOCVFUKRD67NFA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.799519597 (14h17m33.543576447s) [noise=true , address=45.91.116.12:4227 , piece_id=J2FM3FAWY5HYXX3VFC6CSHUSWDQDNKUUB6NL6MWSF2ZG3BZFWBEA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.800753821 (14h17m33.544810667s) [noise=true , address=77.90.31.13:4626 , piece_id=PR6GY2Y7HDUORSLYZXU4IVLVA6RIMBFSRFJW4OTLXUZM7MJZ4KOQ, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.800785983 (14h17m33.544842842s) [seg_pos={0 0}, upload=14716]: Segment reached optimal threshold of 13 pieces. Aug 7 23:57:11.801026052 (14h17m33.545082898s) [noise=true , address=104.36.82.14:4222 , piece_id=JFWGS6VCDTFOQYG32ZFTWNYLFGVVQPXD7K6TZXC4FR2SGEOUS7FA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1): protocol: expected piece hash; context canceled; EOF Aug 7 23:57:11.801098020 (14h17m33.545154867s) [noise=true , address=104.36.82.10:4078 , piece_id=3DPLCXZLQWMZSYSSO3JPLZFP2VXJ5NKYO4FDVFYOWJLT43C7XSTQ, seg_pos={0 0}, upload=14716]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Aug 7 23:57:11.801174944 (14h17m33.54523179s) [noise=true , address=104.36.82.5:4839 , piece_id=AV3ZBQ4W44IZEDAPHJGHKK5D5VRNU5XGOQGDWEARNGIKWRQAGYBQ, seg_pos={0 0}, upload=14716]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 139.59.144.22:48474->104.36.82.5:4839: use of closed network connection; write tcp 139.59.144.22:48474->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48474->104.36.82.5:4839: use of closed network connection Aug 7 23:57:11.801184926 (14h17m33.545241773s) [seg_pos={0 0}, upload=14716]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:57:11.801257096 (14h17m33.545313956s) [upload=14716]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:57:11.801261009 (14h17m33.545317869s) [upload=14716]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:11.942784613 (14h17m33.686841513s) [upload=14716]: Done uploading segments... Aug 7 23:57:11.942815424 (14h17m33.686872279s) [upload=14716]: Upload finished. err: <nil> Aug 7 23:58:20.561350310 (14h18m42.305407184s) [upload=14732]: Starting upload Aug 7 23:58:20.561483402 (14h18m42.30554027s) [upload=14732]: Done starting upload Aug 7 23:58:20.561650297 (14h18m42.305707159s) [upload=14732]: Uploading segments... Aug 7 23:58:20.562196341 (14h18m42.306253212s) [upload=14732]: Got next segment. Inline: false Aug 7 23:58:20.562334166 (14h18m42.306391031s) [upload=14732]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:20.562343255 (14h18m42.306400125s) [upload=14732]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:58:50.281195862 (14h19m12.025252712s) [upload=14732]: Next segment err: context deadline exceeded Aug 7 23:58:50.281383705 (14h19m12.025440555s) [upload=14732]: Done uploading segments... Aug 7 23:58:50.281405600 (14h19m12.02546245s) [upload=14732]: Upload finished. err: context deadline exceeded Aug 7 23:58:50.716777496 (14h19m12.460834346s) [upload=14742]: Starting upload Aug 7 23:58:50.716857905 (14h19m12.460914751s) [upload=14742]: Done starting upload Aug 7 23:58:50.716999972 (14h19m12.461056825s) [upload=14742]: Uploading segments... Aug 7 23:58:50.717186124 (14h19m12.461242976s) [upload=14742]: Got next segment. Inline: false Aug 7 23:58:50.717245601 (14h19m12.461302466s) [upload=14742]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:50.717252185 (14h19m12.461309053s) [upload=14742]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:59:20.436529435 (14h19m42.180586305s) [upload=14742]: Next segment err: context deadline exceeded Aug 7 23:59:20.436734882 (14h19m42.180791733s) [upload=14742]: Done uploading segments... Aug 7 23:59:20.436756200 (14h19m42.180813054s) [upload=14742]: Upload finished. err: context deadline exceeded Duration: 2.605445568s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................X.................................X.] duration: 2.893742326s FAIL (2025-08-08 00:00:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after e4bc51e4c3741c8242928f6ea44cd583e1de2244d7996b5999976d42b698bf65 != before 74576f6679446814770053fb5ecf3ed143c5a0d1682c10852d0fdad56f6f62fd ======LOGS====== em_ObjectBegin Aug 8 00:00:04.218463528 (14h20m25.962520394s) [upload=14759]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:00:04.422837522 (14h20m26.166894398s) [seg_pos={0 0}, upload=14759]: Begin upload segment... Aug 8 00:00:04.422913763 (14h20m26.166970631s) [seg_pos={0 0}, upload=14759]: Done begin upload segment. Aug 8 00:00:04.422953631 (14h20m26.167010505s) [noise=true , address=64.49.13.138:4490 , piece_id=VWBCADPZQBKS6HWUJZ6YWK7FDXN6Z2TSBJKK4Z7P3SDYS476UG5A, seg_pos={0 0}, upload=14759]: Uploading piece... Aug 8 00:00:04.423659818 (14h20m26.167716937s) [noise=true , address=104.36.82.38:4713 , piece_id=44IRUAVNYKZVR74WG64GQPGH3J337QYLGYWC3BUOZ3VS5NPJL27A, seg_pos={0 0}, upload=14759]: Uploading piece... Aug 8 00:00:04.424122548 (14h20m26.168179422s) [noise=true , address=79.127.201.214:4295 , piece_id=CCDJBXRBUJ3NL4PDRRCTXBYXWYBIPYSNU3T3I6R4CTJWJRZTGGQA, seg_pos={0 0}, upload=14759]: Uploading piece... Aug 8 00:00:04.424515211 (14h20m26.168572087s) [noise=true , address=104.36.82.19:4404 , piece_id=LHQKAL4W24GGRXIZJ672W6CXY3NDMUN6NJVH2SFRHETKJMEHW62Q, seg_pos={0 0}, upload=14759]: Uploading piece... Aug 8 00:00:04.424927726 (14h20m26.1689846s) [noise=true , address=104.36.82.3:4796 , piece_id=APJB3DPBNQBOFK34LZ5MPOOR45ZXRQSMFWB4SOCHFOJCSDW3GWVQ, seg_pos={0 0}, upload=14759]: Uploading piece... Aug 8 00:00:04.425328696 (14h20m26.169385571s) [noise=true , address=64.49.13.139:4470 , piece_id=WOXVGGQF2Q5PM6V3LGUOMMA4ZU7UPVW2KOR25TQ32PXN6OWCGGLA, seg_pos={0 0}, upload=14759]: Uploading piece... Aug 8 00:00:04.425736715 (14h20m26.169793589s) [noise=true , address=104.36.82.17:4335 , piece_id=TB54EDHJ5XMYQPXWQB2NMUECRW5VTECH7H6KOPLCFUJ5SXS7U4OQ, seg_pos={0 0}, upload=14759]: Uploading piece... Aug 8 00:00:04.426503554 (14h20m26.170560432s) [noise=true , address=104.36.82.59:4318 , piece_id=EHUOWZNGTRHSK7HXYEYYREX66ES4MPYKEU34EW66PFUPHI7VS3XQ, seg_pos={0 0}, upload=14759]: Uploading piece... Aug 8 00:00:04.430790810 (14h20m26.174847683s) [noise=true , address=64.49.13.155:4916 , piece_id=54U5CI5CPB7DR56LS2GCOWNANROABC65PXTMSSVUTMZDQEZJY7CA, seg_pos={0 0}, upload=14759]: Uploading piece... Aug 8 00:00:04.431475312 (14h20m26.175532189s) [noise=true , address=104.36.82.62:4425 , piece_id=BOX4GUARMRXU5IIE64CELJHQZFT6UWG4TFDATZHX5YFCJTFV2N4A, seg_pos={0 0}, upload=14759]: Uploading piece... Aug 8 00:00:04.431922656 (14h20m26.175979526s) [noise=true , address=104.36.82.58:4268 , piece_id=YFPLOYXHUU3SOZJHFEMSTCNBM55AYJ5NONP7GLTDFU3OTZTW4BIQ, seg_pos={0 0}, upload=14759]: Uploading piece... Aug 8 00:00:04.432313695 (14h20m26.176370573s) [noise=true , address=104.36.82.55:4221 , piece_id=G3QN6J3Q4ZSDPAESESJMAWK3ICJLFBWPWY76QGCSMTHKYP4SUJ6Q, seg_pos={0 0}, upload=14759]: Uploading piece... Aug 8 00:00:04.432740279 (14h20m26.17679715s) [noise=true , address=45.91.116.14:4195 , piece_id=OFPVRGPPQRMNV3FV46GHPFWSXSNSMDDM7MZ7MFO5D4SP7NS6UZGA, seg_pos={0 0}, upload=14759]: Uploading piece... Aug 8 00:00:04.433162962 (14h20m26.177219835s) [noise=true , address=104.36.82.44:4885 , piece_id=ZHPCTPQUPDXJDPZQPBFSOD6KYKAVCHXQNXESSKEXNDU3XSL6OY3A, seg_pos={0 0}, upload=14759]: Uploading piece... Aug 8 00:00:04.433650237 (14h20m26.177707112s) [noise=true , address=104.36.82.43:4854 , piece_id=CY3WAFZSTIQ2AV6DYPIOZSOORUBPL3FBZK4CZNWL3SMOVG7DSZGQ, seg_pos={0 0}, upload=14759]: Uploading piece... Aug 8 00:00:04.434316480 (14h20m26.178373354s) [noise=true , address=64.49.13.26:4948 , piece_id=TN7TAN4AYCIDUXZHESWSGMPXYDEM3ZN5RIS5HV3OVOALIFLJDQ7A, seg_pos={0 0}, upload=14759]: Uploading piece... Aug 8 00:00:04.443916849 (14h20m26.187973724s) [noise=true , address=104.36.82.61:4393 , piece_id=MIKYUC5I7GASL3XCYWW7QHLYNJIKSLIHAPNF6NMV45ACHHJLTJJQ, seg_pos={0 0}, upload=14759]: Uploading piece... Aug 8 00:00:04.444505957 (14h20m26.188562832s) [noise=true , address=45.91.116.6:4982 , piece_id=M3LYDOCIFKY5YHDBN6ZX6XGSUASZP74QRPZX3VFYSWD37QGHFNBA, seg_pos={0 0}, upload=14759]: Uploading piece... Aug 8 00:00:05.200540577 (14h20m26.944597434s) [upload=14759]: Next returned nil segment Aug 8 00:00:05.200687222 (14h20m26.944744097s) [upload=14759]: Waiting for error group managing segments... Aug 8 00:00:05.355754277 (14h20m27.099811131s) [noise=true , address=104.36.82.59:4318 , piece_id=EHUOWZNGTRHSK7HXYEYYREX66ES4MPYKEU34EW66PFUPHI7VS3XQ, seg_pos={0 0}, upload=14759]: Done uploading piece. err: <nil> Aug 8 00:00:05.358198826 (14h20m27.102255692s) [noise=true , address=104.36.82.58:4268 , piece_id=YFPLOYXHUU3SOZJHFEMSTCNBM55AYJ5NONP7GLTDFU3OTZTW4BIQ, seg_pos={0 0}, upload=14759]: Done uploading piece. err: <nil> Aug 8 00:00:05.386961372 (14h20m27.13101822s) [noise=true , address=104.36.82.19:4404 , piece_id=LHQKAL4W24GGRXIZJ672W6CXY3NDMUN6NJVH2SFRHETKJMEHW62Q, seg_pos={0 0}, upload=14759]: Done uploading piece. err: <nil> Aug 8 00:00:05.391547939 (14h20m27.135604807s) [noise=true , address=104.36.82.55:4221 , piece_id=G3QN6J3Q4ZSDPAESESJMAWK3ICJLFBWPWY76QGCSMTHKYP4SUJ6Q, seg_pos={0 0}, upload=14759]: Done uploading piece. err: <nil> Aug 8 00:00:05.391724549 (14h20m27.135781419s) [noise=true , address=104.36.82.43:4854 , piece_id=CY3WAFZSTIQ2AV6DYPIOZSOORUBPL3FBZK4CZNWL3SMOVG7DSZGQ, seg_pos={0 0}, upload=14759]: Done uploading piece. err: <nil> Aug 8 00:00:05.625387364 (14h20m27.369444214s) [noise=true , address=104.36.82.17:4335 , piece_id=TB54EDHJ5XMYQPXWQB2NMUECRW5VTECH7H6KOPLCFUJ5SXS7U4OQ, seg_pos={0 0}, upload=14759]: Done uploading piece. err: <nil> Aug 8 00:00:05.685379596 (14h20m27.429436466s) [noise=true , address=104.36.82.38:4713 , piece_id=44IRUAVNYKZVR74WG64GQPGH3J337QYLGYWC3BUOZ3VS5NPJL27A, seg_pos={0 0}, upload=14759]: Done uploading piece. err: <nil> Aug 8 00:00:05.696642486 (14h20m27.440699351s) [noise=true , address=45.91.116.6:4982 , piece_id=M3LYDOCIFKY5YHDBN6ZX6XGSUASZP74QRPZX3VFYSWD37QGHFNBA, seg_pos={0 0}, upload=14759]: Done uploading piece. err: <nil> Aug 8 00:00:05.722664674 (14h20m27.466721544s) [noise=true , address=104.36.82.3:4796 , piece_id=APJB3DPBNQBOFK34LZ5MPOOR45ZXRQSMFWB4SOCHFOJCSDW3GWVQ, seg_pos={0 0}, upload=14759]: Done uploading piece. err: <nil> Aug 8 00:00:05.733358308 (14h20m27.477415158s) [noise=true , address=104.36.82.62:4425 , piece_id=BOX4GUARMRXU5IIE64CELJHQZFT6UWG4TFDATZHX5YFCJTFV2N4A, seg_pos={0 0}, upload=14759]: Done uploading piece. err: <nil> Aug 8 00:00:05.781882083 (14h20m27.525938951s) [noise=true , address=45.91.116.14:4195 , piece_id=OFPVRGPPQRMNV3FV46GHPFWSXSNSMDDM7MZ7MFO5D4SP7NS6UZGA, seg_pos={0 0}, upload=14759]: Done uploading piece. err: <nil> Aug 8 00:00:05.908897662 (14h20m27.652954524s) [noise=true , address=79.127.201.214:4295 , piece_id=CCDJBXRBUJ3NL4PDRRCTXBYXWYBIPYSNU3T3I6R4CTJWJRZTGGQA, seg_pos={0 0}, upload=14759]: Done uploading piece. err: <nil> Aug 8 00:00:06.052322455 (14h20m27.796379308s) [noise=true , address=64.49.13.139:4470 , piece_id=WOXVGGQF2Q5PM6V3LGUOMMA4ZU7UPVW2KOR25TQ32PXN6OWCGGLA, seg_pos={0 0}, upload=14759]: Done uploading piece. err: <nil> Aug 8 00:00:06.059613546 (14h20m27.803670401s) [noise=true , address=64.49.13.155:4916 , piece_id=54U5CI5CPB7DR56LS2GCOWNANROABC65PXTMSSVUTMZDQEZJY7CA, seg_pos={0 0}, upload=14759]: Done uploading piece. err: <nil> Aug 8 00:00:06.059650800 (14h20m27.803707652s) [seg_pos={0 0}, upload=14759]: Segment reached optimal threshold of 14 pieces. Aug 8 00:00:06.061105436 (14h20m27.805162307s) [noise=true , address=64.49.13.138:4490 , piece_id=VWBCADPZQBKS6HWUJZ6YWK7FDXN6Z2TSBJKK4Z7P3SDYS476UG5A, seg_pos={0 0}, upload=14759]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Jp1vcm8dpppQHEfAtYmM29CDS3mqhojhuVrzkL3PsYHW2q8zA): protocol: expected piece hash; context canceled; EOF Aug 8 00:00:06.061315764 (14h20m27.805372629s) [noise=true , address=64.49.13.26:4948 , piece_id=TN7TAN4AYCIDUXZHESWSGMPXYDEM3ZN5RIS5HV3OVOALIFLJDQ7A, seg_pos={0 0}, upload=14759]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nVV4rAAZRd4DcLsakTTyPRNWck7AjBA6StKvsVLH2dCKVkwHB): protocol: expected piece hash; context canceled; EOF Aug 8 00:00:06.061491394 (14h20m27.805548286s) [noise=true , address=104.36.82.44:4885 , piece_id=ZHPCTPQUPDXJDPZQPBFSOD6KYKAVCHXQNXESSKEXNDU3XSL6OY3A, seg_pos={0 0}, upload=14759]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:118qGQLFSNbsYsvFrKjTXXWbFYu2sRaRHrUNkKNCrNVpzVQRtG): protocol: write tcp 139.59.144.22:35754->104.36.82.44:4885: use of closed network connection; write tcp 139.59.144.22:35754->104.36.82.44:4885: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35754->104.36.82.44:4885: use of closed network connection Aug 8 00:00:06.078661046 (14h20m27.822717918s) [noise=true , address=104.36.82.61:4393 , piece_id=MIKYUC5I7GASL3XCYWW7QHLYNJIKSLIHAPNF6NMV45ACHHJLTJJQ, seg_pos={0 0}, upload=14759]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xfCSdYjRRxMaTw47dQmaXzGmMWyV1oCCGx7JdhvGqaaoN14Ff): protocol: write tcp 139.59.144.22:48102->104.36.82.61:4393: use of closed network connection; write tcp 139.59.144.22:48102->104.36.82.61:4393: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48102->104.36.82.61:4393: use of closed network connection Aug 8 00:00:06.078739080 (14h20m27.822795929s) [seg_pos={0 0}, upload=14759]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 00:00:06.078890768 (14h20m27.82294764s) [upload=14759]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:00:06.078899977 (14h20m27.82295685s) [upload=14759]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:06.568180745 (14h20m28.312237613s) [upload=14759]: Done uploading segments... Aug 8 00:00:06.568206007 (14h20m28.312262863s) [upload=14759]: Upload finished. err: <nil> Duration:* 2.893833436s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 812.462517ms FAIL (2025-08-08 00:00:36 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== pload=14121]: Done uploading segments... Aug 7 23:22:28.505845197 (13h42m50.249902044s) [upload=14121]: Upload finished. err: <nil> Aug 7 23:23:10.377483605 (13h43m32.121540458s) [upload=14132]: Starting upload Aug 7 23:23:10.377535635 (13h43m32.121592495s) [upload=14132]: Done starting upload Aug 7 23:23:10.377603797 (13h43m32.121660647s) [upload=14132]: Uploading segments... Aug 7 23:23:10.377638935 (13h43m32.121695804s) [upload=14132]: Got next segment. Inline: true Aug 7 23:23:10.377643594 (13h43m32.121700454s) [upload=14132]: Waiting for error group managing segments... Aug 7 23:23:10.377688252 (13h43m32.121745104s) [upload=14132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:23:10.377693427 (13h43m32.121750286s) [upload=14132]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:23:10.377696805 (13h43m32.121753666s) [upload=14132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:23:10.562260724 (13h43m32.306317573s) [upload=14132]: Done uploading segments... Aug 7 23:23:10.562279228 (13h43m32.306336078s) [upload=14132]: Upload finished. err: <nil> Aug 7 23:24:07.896404858 (13h44m29.640461721s) [upload=14147]: Starting upload Aug 7 23:24:07.896490216 (13h44m29.640547079s) [upload=14147]: Done starting upload Aug 7 23:24:07.896580488 (13h44m29.640637339s) [upload=14147]: Uploading segments... Aug 7 23:24:07.896655006 (13h44m29.640711874s) [upload=14147]: Got next segment. Inline: true Aug 7 23:24:07.896660179 (13h44m29.640717039s) [upload=14147]: Waiting for error group managing segments... Aug 7 23:24:07.896714840 (13h44m29.6407717s) [upload=14147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:24:07.896719399 (13h44m29.64077626s) [upload=14147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:24:07.896722726 (13h44m29.640779586s) [upload=14147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:24:08.048898130 (13h44m29.792954991s) [upload=14147]: Done uploading segments... Aug 7 23:24:08.048952526 (13h44m29.793009386s) [upload=14147]: Upload finished. err: <nil> Aug 7 23:24:53.794754379 (13h45m15.53881125s) [upload=14162]: Starting upload Aug 7 23:24:53.794842317 (13h45m15.538899177s) [upload=14162]: Done starting upload Aug 7 23:24:53.794935964 (13h45m15.538992814s) [upload=14162]: Uploading segments... Aug 7 23:24:53.795003752 (13h45m15.539060623s) [upload=14162]: Got next segment. Inline: true Aug 7 23:24:53.795010814 (13h45m15.539067676s) [upload=14162]: Waiting for error group managing segments... Aug 7 23:24:53.795066607 (13h45m15.539123488s) [upload=14162]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:24:53.795158449 (13h45m15.5392153s) [upload=14162]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:24:53.795163166 (13h45m15.539220026s) [upload=14162]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:24:53.946994982 (13h45m15.691051833s) [upload=14162]: Done uploading segments... Aug 7 23:24:53.947013862 (13h45m15.691070711s) [upload=14162]: Upload finished. err: <nil> Aug 7 23:25:40.029856966 (13h46m1.773913836s) [upload=14177]: Starting upload Aug 7 23:25:40.029986938 (13h46m1.774043809s) [upload=14177]: Done starting upload Aug 7 23:25:40.030076796 (13h46m1.774133642s) [upload=14177]: Uploading segments... Aug 7 23:25:40.030132567 (13h46m1.774189438s) [upload=14177]: Got next segment. Inline: true Aug 7 23:25:40.030139885 (13h46m1.774196757s) [upload=14177]: Waiting for error group managing segments... Aug 7 23:25:40.030197588 (13h46m1.774254461s) [upload=14177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:25:40.030204146 (13h46m1.774261016s) [upload=14177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:25:40.030207984 (13h46m1.77426485s) [upload=14177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:25:40.184198742 (13h46m1.928255592s) [upload=14177]: Done uploading segments... Aug 7 23:25:40.184219017 (13h46m1.928275867s) [upload=14177]: Upload finished. err: <nil> Aug 7 23:26:54.625649110 (13h47m16.369705971s) [upload=14198]: Starting upload Aug 7 23:26:54.625738563 (13h47m16.369795424s) [upload=14198]: Done starting upload Aug 7 23:26:54.625842691 (13h47m16.369899539s) [upload=14198]: Uploading segments... Aug 7 23:26:54.626016839 (13h47m16.37007371s) [upload=14198]: Got next segment. Inline: true Aug 7 23:26:54.626052112 (13h47m16.370108973s) [upload=14198]: Waiting for error group managing segments... Aug 7 23:26:54.626129780 (13h47m16.370186645s) [upload=14198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:26:54.626137392 (13h47m16.370194264s) [upload=14198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:26:54.626143182 (13h47m16.370200048s) [upload=14198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:26:54.775804171 (13h47m16.519861021s) [upload=14198]: Done uploading segments... Aug 7 23:26:54.775824249 (13h47m16.519881102s) [upload=14198]: Upload finished. err: <nil> Aug 7 23:27:32.263746490 (13h47m54.007803361s) [upload=14208]: Starting upload Aug 7 23:27:32.263833655 (13h47m54.007890501s) [upload=14208]: Done starting upload Aug 7 23:27:32.263976536 (13h47m54.008033385s) [upload=14208]: Uploading segments... Aug 7 23:27:32.264070926 (13h47m54.008127796s) [upload=14208]: Got next segment. Inline: true Aug 7 23:27:32.264080356 (13h47m54.00813733s) [upload=14208]: Waiting for error group managing segments... Aug 7 23:27:32.264154657 (13h47m54.008211526s) [upload=14208]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:27:32.264163587 (13h47m54.008220454s) [upload=14208]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:27:32.264172017 (13h47m54.008228914s) [upload=14208]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:27:32.490188559 (13h47m54.23424541s) [upload=14208]: Done uploading segments... Aug 7 23:27:32.490207326 (13h47m54.234264172s) [upload=14208]: Upload finished. err: <nil> Aug 7 23:27:57.927940477 (13h48m19.671997323s) [upload=14217]: Starting upload Aug 7 23:27:57.928016423 (13h48m19.672073283s) [upload=14217]: Done starting upload Aug 7 23:27:57.928101290 (13h48m19.672158142s) [upload=14217]: Uploading segments... Aug 7 23:27:57.928155692 (13h48m19.672212566s) [upload=14217]: Got next segment. Inline: true Aug 7 23:27:57.928162009 (13h48m19.672218869s) [upload=14217]: Waiting for error group managing segments... Aug 7 23:27:57.928213294 (13h48m19.672270145s) [upload=14217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:27:57.928219212 (13h48m19.672276074s) [upload=14217]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:27:57.928223521 (13h48m19.672280383s) [upload=14217]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:27:58.087544005 (13h48m19.831600874s) [upload=14217]: Done uploading segments... Aug 7 23:27:58.087570226 (13h48m19.831627094s) [upload=14217]: Upload finished. err: <nil> Aug 7 23:29:12.765200935 (13h49m34.509257795s) [upload=14237]: Starting upload Aug 7 23:29:12.765253106 (13h49m34.509309977s) [upload=14237]: Done starting upload Aug 7 23:29:12.765324641 (13h49m34.509381499s) [upload=14237]: Uploading segments... Aug 7 23:29:12.765374187 (13h49m34.509431056s) [upload=14237]: Got next segment. Inline: true Aug 7 23:29:12.765378947 (13h49m34.509435807s) [upload=14237]: Waiting for error group managing segments... Aug 7 23:29:12.765430859 (13h49m34.50948772s) [upload=14237]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:29:12.765438106 (13h49m34.50949497s) [upload=14237]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:29:12.765443127 (13h49m34.509499988s) [upload=14237]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:29:12.926437674 (13h49m34.670494535s) [upload=14237]: Done uploading segments... Aug 7 23:29:12.926457418 (13h49m34.670514268s) [upload=14237]: Upload finished. err: <nil> Aug 7 23:29:42.225760316 (13h50m3.969817177s) [upload=14244]: Starting upload Aug 7 23:29:42.225855063 (13h50m3.969911929s) [upload=14244]: Done starting upload Aug 7 23:29:42.225962489 (13h50m3.970019355s) [upload=14244]: Uploading segments... Aug 7 23:29:42.226078484 (13h50m3.970135331s) [upload=14244]: Got next segment. Inline: true Aug 7 23:29:42.226083996 (13h50m3.970140842s) [upload=14244]: Waiting for error group managing segments... Aug 7 23:29:42.226143067 (13h50m3.970199914s) [upload=14244]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:29:42.226148511 (13h50m3.970205371s) [upload=14244]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:29:42.226153006 (13h50m3.970209865s) [upload=14244]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:29:42.401822947 (13h50m4.14587981s) [upload=14244]: Done uploading segments... Aug 7 23:29:42.401847556 (13h50m4.145904405s) [upload=14244]: Upload finished. err: <nil> Aug 7 23:30:31.529840301 (13h50m53.273897165s) [upload=14254]: Starting upload Aug 7 23:30:31.530002531 (13h50m53.274059378s) [upload=14254]: Done starting upload Aug 7 23:30:31.530201544 (13h50m53.274258391s) [upload=14254]: Uploading segments... Aug 7 23:30:31.530298120 (13h50m53.274354985s) [upload=14254]: Got next segment. Inline: true Aug 7 23:30:31.530303457 (13h50m53.274360317s) [upload=14254]: Waiting for error group managing segments... Aug 7 23:30:31.530353721 (13h50m53.274410567s) [upload=14254]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:30:31.530358927 (13h50m53.274415788s) [upload=14254]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:30:31.530363758 (13h50m53.274420619s) [upload=14254]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:30:31.696350967 (13h50m53.440407828s) [upload=14254]: Done uploading segments... Aug 7 23:30:31.696369460 (13h50m53.440426307s) [upload=14254]: Upload finished. err: <nil> Duration: 812.528803ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 11 out of 60 is failed (threshold=3) [.XX.XX.X.........................................X.XXX.X.X..] duration: 903.649991ms FAIL (2025-08-08 00:00:33 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 903.681511ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 8 out of 60 is failed (threshold=3) [XXX................................................XX..XX.X.] duration: 881.106746ms FAIL (2025-08-08 00:00:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 12m31.486413495s) [upload=14628]: Starting upload Aug 7 23:52:09.742413553 (14h12m31.486470415s) [upload=14628]: Done starting upload Aug 7 23:52:09.742476702 (14h12m31.486533548s) [upload=14628]: Uploading segments... Aug 7 23:52:09.742511425 (14h12m31.486568294s) [upload=14628]: Got next segment. Inline: true Aug 7 23:52:09.742515305 (14h12m31.486572152s) [upload=14628]: Waiting for error group managing segments... Aug 7 23:52:09.742550700 (14h12m31.486607545s) [upload=14628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:09.742555041 (14h12m31.486611889s) [upload=14628]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:09.742557771 (14h12m31.486614632s) [upload=14628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:09.895389395 (14h12m31.639446255s) [upload=14628]: Done uploading segments... Aug 7 23:52:09.895421188 (14h12m31.639478039s) [upload=14628]: Upload finished. err: <nil> Aug 7 23:52:28.792030311 (14h12m50.536087163s) [upload=14635]: Starting upload Aug 7 23:52:28.792092644 (14h12m50.536149508s) [upload=14635]: Done starting upload Aug 7 23:52:28.792152087 (14h12m50.53620894s) [upload=14635]: Uploading segments... Aug 7 23:52:28.792190462 (14h12m50.536247332s) [upload=14635]: Got next segment. Inline: true Aug 7 23:52:28.792195687 (14h12m50.536252548s) [upload=14635]: Waiting for error group managing segments... Aug 7 23:52:28.792239958 (14h12m50.536296808s) [upload=14635]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:28.792245830 (14h12m50.536302691s) [upload=14635]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:28.792249165 (14h12m50.536306026s) [upload=14635]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:28.947698658 (14h12m50.691755509s) [upload=14635]: Done uploading segments... Aug 7 23:52:28.947717721 (14h12m50.691774571s) [upload=14635]: Upload finished. err: <nil> Aug 7 23:53:17.152926358 (14h13m38.896983229s) [upload=14650]: Starting upload Aug 7 23:53:17.153036870 (14h13m38.89709373s) [upload=14650]: Done starting upload Aug 7 23:53:17.153154678 (14h13m38.897211544s) [upload=14650]: Uploading segments... Aug 7 23:53:17.153233089 (14h13m38.897289957s) [upload=14650]: Got next segment. Inline: true Aug 7 23:53:17.153240256 (14h13m38.897297124s) [upload=14650]: Waiting for error group managing segments... Aug 7 23:53:17.153318892 (14h13m38.897375757s) [upload=14650]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:17.153325816 (14h13m38.89738268s) [upload=14650]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:17.153330453 (14h13m38.897387314s) [upload=14650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:16.875757982 (14h14m38.619814845s) [upload=14650]: Done uploading segments... Aug 7 23:54:16.875781849 (14h14m38.619838701s) [upload=14650]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:24.158448119 (14h14m45.90250498s) [upload=14667]: Starting upload Aug 7 23:54:24.158525946 (14h14m45.902582806s) [upload=14667]: Done starting upload Aug 7 23:54:24.158598972 (14h14m45.902655824s) [upload=14667]: Uploading segments... Aug 7 23:54:24.158654148 (14h14m45.902711017s) [upload=14667]: Got next segment. Inline: true Aug 7 23:54:24.158659995 (14h14m45.902716856s) [upload=14667]: Waiting for error group managing segments... Aug 7 23:54:24.158714896 (14h14m45.902771747s) [upload=14667]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:24.158720357 (14h14m45.902777209s) [upload=14667]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:24.158724105 (14h14m45.902780965s) [upload=14667]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:23.888400190 (14h15m45.63245704s) [upload=14667]: Done uploading segments... Aug 7 23:55:23.888419087 (14h15m45.632475938s) [upload=14667]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:24.297146309 (14h15m46.041203174s) [upload=14687]: Starting upload Aug 7 23:55:24.297232144 (14h15m46.04128901s) [upload=14687]: Done starting upload Aug 7 23:55:24.297315772 (14h15m46.041372619s) [upload=14687]: Uploading segments... Aug 7 23:55:24.297396238 (14h15m46.041453151s) [upload=14687]: Got next segment. Inline: true Aug 7 23:55:24.297403729 (14h15m46.04146059s) [upload=14687]: Waiting for error group managing segments... Aug 7 23:55:24.297444116 (14h15m46.041500976s) [upload=14687]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:24.297450971 (14h15m46.041507831s) [upload=14687]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:24.297453883 (14h15m46.041510749s) [upload=14687]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:24.439126204 (14h15m46.183183055s) [upload=14687]: Done uploading segments... Aug 7 23:55:24.439141121 (14h15m46.183197972s) [upload=14687]: Upload finished. err: <nil> Aug 7 23:56:03.068174127 (14h16m24.812230977s) [upload=14695]: Starting upload Aug 7 23:56:03.068258755 (14h16m24.812315617s) [upload=14695]: Done starting upload Aug 7 23:56:03.068368961 (14h16m24.812425812s) [upload=14695]: Uploading segments... Aug 7 23:56:03.068437851 (14h16m24.812494719s) [upload=14695]: Got next segment. Inline: true Aug 7 23:56:03.068445058 (14h16m24.812501927s) [upload=14695]: Waiting for error group managing segments... Aug 7 23:56:03.068516175 (14h16m24.812573049s) [upload=14695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:03.068523275 (14h16m24.812580137s) [upload=14695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:03.068527513 (14h16m24.812584488s) [upload=14695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:03.370378590 (14h16m25.114435452s) [upload=14695]: Done uploading segments... Aug 7 23:56:03.370401742 (14h16m25.114458656s) [upload=14695]: Upload finished. err: <nil> Aug 7 23:56:41.636734874 (14h17m3.380791742s) [upload=14707]: Starting upload Aug 7 23:56:41.636844337 (14h17m3.380901201s) [upload=14707]: Done starting upload Aug 7 23:56:41.636938361 (14h17m3.380995223s) [upload=14707]: Uploading segments... Aug 7 23:56:41.637004566 (14h17m3.381061436s) [upload=14707]: Got next segment. Inline: true Aug 7 23:56:41.637011720 (14h17m3.381068581s) [upload=14707]: Waiting for error group managing segments... Aug 7 23:56:41.637073380 (14h17m3.381130242s) [upload=14707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:41.637079684 (14h17m3.381136546s) [upload=14707]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:41.637083892 (14h17m3.381140753s) [upload=14707]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:41.369074470 (14h18m3.113131318s) [upload=14707]: Done uploading segments... Aug 7 23:57:41.369096313 (14h18m3.11315316s) [upload=14707]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:57:41.785857243 (14h18m3.529914104s) [upload=14726]: Starting upload Aug 7 23:57:41.785973295 (14h18m3.530030155s) [upload=14726]: Done starting upload Aug 7 23:57:41.786053418 (14h18m3.530110268s) [upload=14726]: Uploading segments... Aug 7 23:57:41.786115407 (14h18m3.530172273s) [upload=14726]: Got next segment. Inline: true Aug 7 23:57:41.786121268 (14h18m3.530178127s) [upload=14726]: Waiting for error group managing segments... Aug 7 23:57:41.786162041 (14h18m3.530218901s) [upload=14726]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:57:41.786167247 (14h18m3.530224108s) [upload=14726]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:57:41.786170583 (14h18m3.530227443s) [upload=14726]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:41.518215592 (14h19m3.262272452s) [upload=14726]: Done uploading segments... Aug 7 23:58:41.518237426 (14h19m3.262294275s) [upload=14726]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:58:49.080161447 (14h19m10.824218292s) [upload=14741]: Starting upload Aug 7 23:58:49.080236560 (14h19m10.824293407s) [upload=14741]: Done starting upload Aug 7 23:58:49.080313897 (14h19m10.824370744s) [upload=14741]: Uploading segments... Aug 7 23:58:49.080371825 (14h19m10.824428691s) [upload=14741]: Got next segment. Inline: true Aug 7 23:58:49.080376725 (14h19m10.824433586s) [upload=14741]: Waiting for error group managing segments... Aug 7 23:58:49.080418334 (14h19m10.824475197s) [upload=14741]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:49.080422560 (14h19m10.824479419s) [upload=14741]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:49.080438323 (14h19m10.824495186s) [upload=14741]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:49.220444289 (14h19m10.964501139s) [upload=14741]: Done uploading segments... Aug 7 23:58:49.220462660 (14h19m10.96451951s) [upload=14741]: Upload finished. err: <nil> Aug 7 23:59:26.271206280 (14h19m48.015263132s) [upload=14752]: Starting upload Aug 7 23:59:26.271300842 (14h19m48.015357706s) [upload=14752]: Done starting upload Aug 7 23:59:26.271409542 (14h19m48.015466389s) [upload=14752]: Uploading segments... Aug 7 23:59:26.271471969 (14h19m48.015528839s) [upload=14752]: Got next segment. Inline: true Aug 7 23:59:26.271478422 (14h19m48.015535286s) [upload=14752]: Waiting for error group managing segments... Aug 7 23:59:26.271524304 (14h19m48.015581168s) [upload=14752]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:26.271531803 (14h19m48.01558867s) [upload=14752]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:26.271535740 (14h19m48.015592604s) [upload=14752]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:26.006104454 (14h20m47.750161323s) [upload=14752]: Done uploading segments... Aug 8 00:00:26.006140966 (14h20m47.750197816s) [upload=14752]: Upload finished. err: metaclient: context deadline exceeded Duration: 881.131036ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 9 out of 60 is failed (threshold=3) [XXXX................................................XX..XX.X] duration: 1m0.009057746s FAIL (2025-08-08 00:00:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 12m31.486413495s) [upload=14628]: Starting upload Aug 7 23:52:09.742413553 (14h12m31.486470415s) [upload=14628]: Done starting upload Aug 7 23:52:09.742476702 (14h12m31.486533548s) [upload=14628]: Uploading segments... Aug 7 23:52:09.742511425 (14h12m31.486568294s) [upload=14628]: Got next segment. Inline: true Aug 7 23:52:09.742515305 (14h12m31.486572152s) [upload=14628]: Waiting for error group managing segments... Aug 7 23:52:09.742550700 (14h12m31.486607545s) [upload=14628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:09.742555041 (14h12m31.486611889s) [upload=14628]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:09.742557771 (14h12m31.486614632s) [upload=14628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:09.895389395 (14h12m31.639446255s) [upload=14628]: Done uploading segments... Aug 7 23:52:09.895421188 (14h12m31.639478039s) [upload=14628]: Upload finished. err: <nil> Aug 7 23:52:28.792030311 (14h12m50.536087163s) [upload=14635]: Starting upload Aug 7 23:52:28.792092644 (14h12m50.536149508s) [upload=14635]: Done starting upload Aug 7 23:52:28.792152087 (14h12m50.53620894s) [upload=14635]: Uploading segments... Aug 7 23:52:28.792190462 (14h12m50.536247332s) [upload=14635]: Got next segment. Inline: true Aug 7 23:52:28.792195687 (14h12m50.536252548s) [upload=14635]: Waiting for error group managing segments... Aug 7 23:52:28.792239958 (14h12m50.536296808s) [upload=14635]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:28.792245830 (14h12m50.536302691s) [upload=14635]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:28.792249165 (14h12m50.536306026s) [upload=14635]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:28.947698658 (14h12m50.691755509s) [upload=14635]: Done uploading segments... Aug 7 23:52:28.947717721 (14h12m50.691774571s) [upload=14635]: Upload finished. err: <nil> Aug 7 23:53:17.152926358 (14h13m38.896983229s) [upload=14650]: Starting upload Aug 7 23:53:17.153036870 (14h13m38.89709373s) [upload=14650]: Done starting upload Aug 7 23:53:17.153154678 (14h13m38.897211544s) [upload=14650]: Uploading segments... Aug 7 23:53:17.153233089 (14h13m38.897289957s) [upload=14650]: Got next segment. Inline: true Aug 7 23:53:17.153240256 (14h13m38.897297124s) [upload=14650]: Waiting for error group managing segments... Aug 7 23:53:17.153318892 (14h13m38.897375757s) [upload=14650]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:17.153325816 (14h13m38.89738268s) [upload=14650]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:17.153330453 (14h13m38.897387314s) [upload=14650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:16.875757982 (14h14m38.619814845s) [upload=14650]: Done uploading segments... Aug 7 23:54:16.875781849 (14h14m38.619838701s) [upload=14650]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:24.158448119 (14h14m45.90250498s) [upload=14667]: Starting upload Aug 7 23:54:24.158525946 (14h14m45.902582806s) [upload=14667]: Done starting upload Aug 7 23:54:24.158598972 (14h14m45.902655824s) [upload=14667]: Uploading segments... Aug 7 23:54:24.158654148 (14h14m45.902711017s) [upload=14667]: Got next segment. Inline: true Aug 7 23:54:24.158659995 (14h14m45.902716856s) [upload=14667]: Waiting for error group managing segments... Aug 7 23:54:24.158714896 (14h14m45.902771747s) [upload=14667]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:24.158720357 (14h14m45.902777209s) [upload=14667]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:24.158724105 (14h14m45.902780965s) [upload=14667]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:23.888400190 (14h15m45.63245704s) [upload=14667]: Done uploading segments... Aug 7 23:55:23.888419087 (14h15m45.632475938s) [upload=14667]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:24.297146309 (14h15m46.041203174s) [upload=14687]: Starting upload Aug 7 23:55:24.297232144 (14h15m46.04128901s) [upload=14687]: Done starting upload Aug 7 23:55:24.297315772 (14h15m46.041372619s) [upload=14687]: Uploading segments... Aug 7 23:55:24.297396238 (14h15m46.041453151s) [upload=14687]: Got next segment. Inline: true Aug 7 23:55:24.297403729 (14h15m46.04146059s) [upload=14687]: Waiting for error group managing segments... Aug 7 23:55:24.297444116 (14h15m46.041500976s) [upload=14687]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:24.297450971 (14h15m46.041507831s) [upload=14687]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:24.297453883 (14h15m46.041510749s) [upload=14687]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:24.439126204 (14h15m46.183183055s) [upload=14687]: Done uploading segments... Aug 7 23:55:24.439141121 (14h15m46.183197972s) [upload=14687]: Upload finished. err: <nil> Aug 7 23:56:03.068174127 (14h16m24.812230977s) [upload=14695]: Starting upload Aug 7 23:56:03.068258755 (14h16m24.812315617s) [upload=14695]: Done starting upload Aug 7 23:56:03.068368961 (14h16m24.812425812s) [upload=14695]: Uploading segments... Aug 7 23:56:03.068437851 (14h16m24.812494719s) [upload=14695]: Got next segment. Inline: true Aug 7 23:56:03.068445058 (14h16m24.812501927s) [upload=14695]: Waiting for error group managing segments... Aug 7 23:56:03.068516175 (14h16m24.812573049s) [upload=14695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:03.068523275 (14h16m24.812580137s) [upload=14695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:03.068527513 (14h16m24.812584488s) [upload=14695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:03.370378590 (14h16m25.114435452s) [upload=14695]: Done uploading segments... Aug 7 23:56:03.370401742 (14h16m25.114458656s) [upload=14695]: Upload finished. err: <nil> Aug 7 23:56:41.636734874 (14h17m3.380791742s) [upload=14707]: Starting upload Aug 7 23:56:41.636844337 (14h17m3.380901201s) [upload=14707]: Done starting upload Aug 7 23:56:41.636938361 (14h17m3.380995223s) [upload=14707]: Uploading segments... Aug 7 23:56:41.637004566 (14h17m3.381061436s) [upload=14707]: Got next segment. Inline: true Aug 7 23:56:41.637011720 (14h17m3.381068581s) [upload=14707]: Waiting for error group managing segments... Aug 7 23:56:41.637073380 (14h17m3.381130242s) [upload=14707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:41.637079684 (14h17m3.381136546s) [upload=14707]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:41.637083892 (14h17m3.381140753s) [upload=14707]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:41.369074470 (14h18m3.113131318s) [upload=14707]: Done uploading segments... Aug 7 23:57:41.369096313 (14h18m3.11315316s) [upload=14707]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:57:41.785857243 (14h18m3.529914104s) [upload=14726]: Starting upload Aug 7 23:57:41.785973295 (14h18m3.530030155s) [upload=14726]: Done starting upload Aug 7 23:57:41.786053418 (14h18m3.530110268s) [upload=14726]: Uploading segments... Aug 7 23:57:41.786115407 (14h18m3.530172273s) [upload=14726]: Got next segment. Inline: true Aug 7 23:57:41.786121268 (14h18m3.530178127s) [upload=14726]: Waiting for error group managing segments... Aug 7 23:57:41.786162041 (14h18m3.530218901s) [upload=14726]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:57:41.786167247 (14h18m3.530224108s) [upload=14726]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:57:41.786170583 (14h18m3.530227443s) [upload=14726]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:41.518215592 (14h19m3.262272452s) [upload=14726]: Done uploading segments... Aug 7 23:58:41.518237426 (14h19m3.262294275s) [upload=14726]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:58:49.080161447 (14h19m10.824218292s) [upload=14741]: Starting upload Aug 7 23:58:49.080236560 (14h19m10.824293407s) [upload=14741]: Done starting upload Aug 7 23:58:49.080313897 (14h19m10.824370744s) [upload=14741]: Uploading segments... Aug 7 23:58:49.080371825 (14h19m10.824428691s) [upload=14741]: Got next segment. Inline: true Aug 7 23:58:49.080376725 (14h19m10.824433586s) [upload=14741]: Waiting for error group managing segments... Aug 7 23:58:49.080418334 (14h19m10.824475197s) [upload=14741]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:49.080422560 (14h19m10.824479419s) [upload=14741]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:49.080438323 (14h19m10.824495186s) [upload=14741]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:49.220444289 (14h19m10.964501139s) [upload=14741]: Done uploading segments... Aug 7 23:58:49.220462660 (14h19m10.96451951s) [upload=14741]: Upload finished. err: <nil> Aug 7 23:59:26.271206280 (14h19m48.015263132s) [upload=14752]: Starting upload Aug 7 23:59:26.271300842 (14h19m48.015357706s) [upload=14752]: Done starting upload Aug 7 23:59:26.271409542 (14h19m48.015466389s) [upload=14752]: Uploading segments... Aug 7 23:59:26.271471969 (14h19m48.015528839s) [upload=14752]: Got next segment. Inline: true Aug 7 23:59:26.271478422 (14h19m48.015535286s) [upload=14752]: Waiting for error group managing segments... Aug 7 23:59:26.271524304 (14h19m48.015581168s) [upload=14752]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:26.271531803 (14h19m48.01558867s) [upload=14752]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:26.271535740 (14h19m48.015592604s) [upload=14752]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:26.006104454 (14h20m47.750161323s) [upload=14752]: Done uploading segments... Aug 8 00:00:26.006140966 (14h20m47.750197816s) [upload=14752]: Upload finished. err: metaclient: context deadline exceeded Duration: 1m0.009087263s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 326.960794ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-08 00:00:25 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 326.994373ms edge gateway vhost us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [X..X........................................................] duration: 2.698101087s FAIL (2025-08-08 00:00:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.698135735s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 7 out of 60 is failed (threshold=3) [XXX...XX.X..............................................X...] duration: 1.130561517s FAIL (2025-08-08 00:00:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.130590532s edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 126.517093ms remote: "tcp:185.244.226.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-08 00:00:13 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 126.552429ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 12 out of 60 is failed (threshold=3) [X.XX.XX.X.........................................X.XXX.X.X.] duration: 967.371671ms FAIL (2025-08-08 00:00:12 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 967.425701ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 13 out of 60 is failed (threshold=3) [XXXXX.XX........................................X.XX...XX.X.] duration: 4.257238161s FAIL (2025-08-08 00:00:08 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 4.257316993s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................X.................................X] duration: 4.227664542s FAIL (2025-08-08 00:00:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after e4bc51e4c3741c8242928f6ea44cd583e1de2244d7996b5999976d42b698bf65 != before 74576f6679446814770053fb5ecf3ed143c5a0d1682c10852d0fdad56f6f62fd ======LOGS====== em_ObjectBegin Aug 8 00:00:04.218463528 (14h20m25.962520394s) [upload=14759]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:00:04.422837522 (14h20m26.166894398s) [seg_pos={0 0}, upload=14759]: Begin upload segment... Aug 8 00:00:04.422913763 (14h20m26.166970631s) [seg_pos={0 0}, upload=14759]: Done begin upload segment. Aug 8 00:00:04.422953631 (14h20m26.167010505s) [noise=true , address=64.49.13.138:4490 , piece_id=VWBCADPZQBKS6HWUJZ6YWK7FDXN6Z2TSBJKK4Z7P3SDYS476UG5A, seg_pos={0 0}, upload=14759]: Uploading piece... Aug 8 00:00:04.423659818 (14h20m26.167716937s) [noise=true , address=104.36.82.38:4713 , piece_id=44IRUAVNYKZVR74WG64GQPGH3J337QYLGYWC3BUOZ3VS5NPJL27A, seg_pos={0 0}, upload=14759]: Uploading piece... Aug 8 00:00:04.424122548 (14h20m26.168179422s) [noise=true , address=79.127.201.214:4295 , piece_id=CCDJBXRBUJ3NL4PDRRCTXBYXWYBIPYSNU3T3I6R4CTJWJRZTGGQA, seg_pos={0 0}, upload=14759]: Uploading piece... Aug 8 00:00:04.424515211 (14h20m26.168572087s) [noise=true , address=104.36.82.19:4404 , piece_id=LHQKAL4W24GGRXIZJ672W6CXY3NDMUN6NJVH2SFRHETKJMEHW62Q, seg_pos={0 0}, upload=14759]: Uploading piece... Aug 8 00:00:04.424927726 (14h20m26.1689846s) [noise=true , address=104.36.82.3:4796 , piece_id=APJB3DPBNQBOFK34LZ5MPOOR45ZXRQSMFWB4SOCHFOJCSDW3GWVQ, seg_pos={0 0}, upload=14759]: Uploading piece... Aug 8 00:00:04.425328696 (14h20m26.169385571s) [noise=true , address=64.49.13.139:4470 , piece_id=WOXVGGQF2Q5PM6V3LGUOMMA4ZU7UPVW2KOR25TQ32PXN6OWCGGLA, seg_pos={0 0}, upload=14759]: Uploading piece... Aug 8 00:00:04.425736715 (14h20m26.169793589s) [noise=true , address=104.36.82.17:4335 , piece_id=TB54EDHJ5XMYQPXWQB2NMUECRW5VTECH7H6KOPLCFUJ5SXS7U4OQ, seg_pos={0 0}, upload=14759]: Uploading piece... Aug 8 00:00:04.426503554 (14h20m26.170560432s) [noise=true , address=104.36.82.59:4318 , piece_id=EHUOWZNGTRHSK7HXYEYYREX66ES4MPYKEU34EW66PFUPHI7VS3XQ, seg_pos={0 0}, upload=14759]: Uploading piece... Aug 8 00:00:04.430790810 (14h20m26.174847683s) [noise=true , address=64.49.13.155:4916 , piece_id=54U5CI5CPB7DR56LS2GCOWNANROABC65PXTMSSVUTMZDQEZJY7CA, seg_pos={0 0}, upload=14759]: Uploading piece... Aug 8 00:00:04.431475312 (14h20m26.175532189s) [noise=true , address=104.36.82.62:4425 , piece_id=BOX4GUARMRXU5IIE64CELJHQZFT6UWG4TFDATZHX5YFCJTFV2N4A, seg_pos={0 0}, upload=14759]: Uploading piece... Aug 8 00:00:04.431922656 (14h20m26.175979526s) [noise=true , address=104.36.82.58:4268 , piece_id=YFPLOYXHUU3SOZJHFEMSTCNBM55AYJ5NONP7GLTDFU3OTZTW4BIQ, seg_pos={0 0}, upload=14759]: Uploading piece... Aug 8 00:00:04.432313695 (14h20m26.176370573s) [noise=true , address=104.36.82.55:4221 , piece_id=G3QN6J3Q4ZSDPAESESJMAWK3ICJLFBWPWY76QGCSMTHKYP4SUJ6Q, seg_pos={0 0}, upload=14759]: Uploading piece... Aug 8 00:00:04.432740279 (14h20m26.17679715s) [noise=true , address=45.91.116.14:4195 , piece_id=OFPVRGPPQRMNV3FV46GHPFWSXSNSMDDM7MZ7MFO5D4SP7NS6UZGA, seg_pos={0 0}, upload=14759]: Uploading piece... Aug 8 00:00:04.433162962 (14h20m26.177219835s) [noise=true , address=104.36.82.44:4885 , piece_id=ZHPCTPQUPDXJDPZQPBFSOD6KYKAVCHXQNXESSKEXNDU3XSL6OY3A, seg_pos={0 0}, upload=14759]: Uploading piece... Aug 8 00:00:04.433650237 (14h20m26.177707112s) [noise=true , address=104.36.82.43:4854 , piece_id=CY3WAFZSTIQ2AV6DYPIOZSOORUBPL3FBZK4CZNWL3SMOVG7DSZGQ, seg_pos={0 0}, upload=14759]: Uploading piece... Aug 8 00:00:04.434316480 (14h20m26.178373354s) [noise=true , address=64.49.13.26:4948 , piece_id=TN7TAN4AYCIDUXZHESWSGMPXYDEM3ZN5RIS5HV3OVOALIFLJDQ7A, seg_pos={0 0}, upload=14759]: Uploading piece... Aug 8 00:00:04.443916849 (14h20m26.187973724s) [noise=true , address=104.36.82.61:4393 , piece_id=MIKYUC5I7GASL3XCYWW7QHLYNJIKSLIHAPNF6NMV45ACHHJLTJJQ, seg_pos={0 0}, upload=14759]: Uploading piece... Aug 8 00:00:04.444505957 (14h20m26.188562832s) [noise=true , address=45.91.116.6:4982 , piece_id=M3LYDOCIFKY5YHDBN6ZX6XGSUASZP74QRPZX3VFYSWD37QGHFNBA, seg_pos={0 0}, upload=14759]: Uploading piece... Aug 8 00:00:05.200540577 (14h20m26.944597434s) [upload=14759]: Next returned nil segment Aug 8 00:00:05.200687222 (14h20m26.944744097s) [upload=14759]: Waiting for error group managing segments... Aug 8 00:00:05.355754277 (14h20m27.099811131s) [noise=true , address=104.36.82.59:4318 , piece_id=EHUOWZNGTRHSK7HXYEYYREX66ES4MPYKEU34EW66PFUPHI7VS3XQ, seg_pos={0 0}, upload=14759]: Done uploading piece. err: <nil> Aug 8 00:00:05.358198826 (14h20m27.102255692s) [noise=true , address=104.36.82.58:4268 , piece_id=YFPLOYXHUU3SOZJHFEMSTCNBM55AYJ5NONP7GLTDFU3OTZTW4BIQ, seg_pos={0 0}, upload=14759]: Done uploading piece. err: <nil> Aug 8 00:00:05.386961372 (14h20m27.13101822s) [noise=true , address=104.36.82.19:4404 , piece_id=LHQKAL4W24GGRXIZJ672W6CXY3NDMUN6NJVH2SFRHETKJMEHW62Q, seg_pos={0 0}, upload=14759]: Done uploading piece. err: <nil> Aug 8 00:00:05.391547939 (14h20m27.135604807s) [noise=true , address=104.36.82.55:4221 , piece_id=G3QN6J3Q4ZSDPAESESJMAWK3ICJLFBWPWY76QGCSMTHKYP4SUJ6Q, seg_pos={0 0}, upload=14759]: Done uploading piece. err: <nil> Aug 8 00:00:05.391724549 (14h20m27.135781419s) [noise=true , address=104.36.82.43:4854 , piece_id=CY3WAFZSTIQ2AV6DYPIOZSOORUBPL3FBZK4CZNWL3SMOVG7DSZGQ, seg_pos={0 0}, upload=14759]: Done uploading piece. err: <nil> Aug 8 00:00:05.625387364 (14h20m27.369444214s) [noise=true , address=104.36.82.17:4335 , piece_id=TB54EDHJ5XMYQPXWQB2NMUECRW5VTECH7H6KOPLCFUJ5SXS7U4OQ, seg_pos={0 0}, upload=14759]: Done uploading piece. err: <nil> Aug 8 00:00:05.685379596 (14h20m27.429436466s) [noise=true , address=104.36.82.38:4713 , piece_id=44IRUAVNYKZVR74WG64GQPGH3J337QYLGYWC3BUOZ3VS5NPJL27A, seg_pos={0 0}, upload=14759]: Done uploading piece. err: <nil> Aug 8 00:00:05.696642486 (14h20m27.440699351s) [noise=true , address=45.91.116.6:4982 , piece_id=M3LYDOCIFKY5YHDBN6ZX6XGSUASZP74QRPZX3VFYSWD37QGHFNBA, seg_pos={0 0}, upload=14759]: Done uploading piece. err: <nil> Aug 8 00:00:05.722664674 (14h20m27.466721544s) [noise=true , address=104.36.82.3:4796 , piece_id=APJB3DPBNQBOFK34LZ5MPOOR45ZXRQSMFWB4SOCHFOJCSDW3GWVQ, seg_pos={0 0}, upload=14759]: Done uploading piece. err: <nil> Aug 8 00:00:05.733358308 (14h20m27.477415158s) [noise=true , address=104.36.82.62:4425 , piece_id=BOX4GUARMRXU5IIE64CELJHQZFT6UWG4TFDATZHX5YFCJTFV2N4A, seg_pos={0 0}, upload=14759]: Done uploading piece. err: <nil> Aug 8 00:00:05.781882083 (14h20m27.525938951s) [noise=true , address=45.91.116.14:4195 , piece_id=OFPVRGPPQRMNV3FV46GHPFWSXSNSMDDM7MZ7MFO5D4SP7NS6UZGA, seg_pos={0 0}, upload=14759]: Done uploading piece. err: <nil> Aug 8 00:00:05.908897662 (14h20m27.652954524s) [noise=true , address=79.127.201.214:4295 , piece_id=CCDJBXRBUJ3NL4PDRRCTXBYXWYBIPYSNU3T3I6R4CTJWJRZTGGQA, seg_pos={0 0}, upload=14759]: Done uploading piece. err: <nil> Aug 8 00:00:06.052322455 (14h20m27.796379308s) [noise=true , address=64.49.13.139:4470 , piece_id=WOXVGGQF2Q5PM6V3LGUOMMA4ZU7UPVW2KOR25TQ32PXN6OWCGGLA, seg_pos={0 0}, upload=14759]: Done uploading piece. err: <nil> Aug 8 00:00:06.059613546 (14h20m27.803670401s) [noise=true , address=64.49.13.155:4916 , piece_id=54U5CI5CPB7DR56LS2GCOWNANROABC65PXTMSSVUTMZDQEZJY7CA, seg_pos={0 0}, upload=14759]: Done uploading piece. err: <nil> Aug 8 00:00:06.059650800 (14h20m27.803707652s) [seg_pos={0 0}, upload=14759]: Segment reached optimal threshold of 14 pieces. Aug 8 00:00:06.061105436 (14h20m27.805162307s) [noise=true , address=64.49.13.138:4490 , piece_id=VWBCADPZQBKS6HWUJZ6YWK7FDXN6Z2TSBJKK4Z7P3SDYS476UG5A, seg_pos={0 0}, upload=14759]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Jp1vcm8dpppQHEfAtYmM29CDS3mqhojhuVrzkL3PsYHW2q8zA): protocol: expected piece hash; context canceled; EOF Aug 8 00:00:06.061315764 (14h20m27.805372629s) [noise=true , address=64.49.13.26:4948 , piece_id=TN7TAN4AYCIDUXZHESWSGMPXYDEM3ZN5RIS5HV3OVOALIFLJDQ7A, seg_pos={0 0}, upload=14759]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nVV4rAAZRd4DcLsakTTyPRNWck7AjBA6StKvsVLH2dCKVkwHB): protocol: expected piece hash; context canceled; EOF Aug 8 00:00:06.061491394 (14h20m27.805548286s) [noise=true , address=104.36.82.44:4885 , piece_id=ZHPCTPQUPDXJDPZQPBFSOD6KYKAVCHXQNXESSKEXNDU3XSL6OY3A, seg_pos={0 0}, upload=14759]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:118qGQLFSNbsYsvFrKjTXXWbFYu2sRaRHrUNkKNCrNVpzVQRtG): protocol: write tcp 139.59.144.22:35754->104.36.82.44:4885: use of closed network connection; write tcp 139.59.144.22:35754->104.36.82.44:4885: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35754->104.36.82.44:4885: use of closed network connection Aug 8 00:00:06.078661046 (14h20m27.822717918s) [noise=true , address=104.36.82.61:4393 , piece_id=MIKYUC5I7GASL3XCYWW7QHLYNJIKSLIHAPNF6NMV45ACHHJLTJJQ, seg_pos={0 0}, upload=14759]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xfCSdYjRRxMaTw47dQmaXzGmMWyV1oCCGx7JdhvGqaaoN14Ff): protocol: write tcp 139.59.144.22:48102->104.36.82.61:4393: use of closed network connection; write tcp 139.59.144.22:48102->104.36.82.61:4393: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48102->104.36.82.61:4393: use of closed network connection Aug 8 00:00:06.078739080 (14h20m27.822795929s) [seg_pos={0 0}, upload=14759]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 00:00:06.078890768 (14h20m27.82294764s) [upload=14759]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:00:06.078899977 (14h20m27.82295685s) [upload=14759]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:06.568180745 (14h20m28.312237613s) [upload=14759]: Done uploading segments... Aug 8 00:00:06.568206007 (14h20m28.312262863s) [upload=14759]: Upload finished. err: <nil> Duration:* 4.227701675s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 11 out of 60 is failed (threshold=3) [XXXXX.X............................................XXX.X.X..] duration: 3.431547078s FAIL (2025-08-08 00:00:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.43157026s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 11 out of 60 is failed (threshold=3) [.XXXXX.X............................................XXX.X.X.] duration: 1m0.000751277s FAIL (2025-08-08 00:00:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000786574s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 13 out of 60 is failed (threshold=3) [XX.XX.XX.X.........................................X.XXX.X.X] duration: 30.001324844s FAIL (2025-08-07 23:59:53 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.00135517s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 3.633702742s FAIL (2025-08-07 23:59:52 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== 294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:32:45.172127356 (13h53m6.916184241s) [upload=14294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:32:45.390149151 (13h53m7.134206011s) [seg_pos={0 0}, upload=14294]: Begin upload segment... Aug 7 23:32:45.390215910 (13h53m7.134272761s) [seg_pos={0 0}, upload=14294]: Done begin upload segment. Aug 7 23:32:45.390249193 (13h53m7.134306059s) [noise=true , address=104.36.82.19:4405 , piece_id=MHM3MWZJFTXSJT7PTOFSWHUYO3L6NIBUV6VU66XXZDH3MC3EFR2Q, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390272919 (13h53m7.134329788s) [noise=true , address=77.90.31.12:4593 , piece_id=C3RVQHFS4L7SKPFFIZ52MAQWGD7P4JNG3SNPCNY53SQUD72PZRUA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390658970 (13h53m7.13471583s) [noise=true , address=104.36.82.16:4321 , piece_id=YTF4RWV2SD6Z4UV4XRYPBRGPCTSUTC6TQKL3DHVKU65ALTNA4TEQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390852777 (13h53m7.134909659s) [noise=true , address=104.36.82.34:4566 , piece_id=3YEAUSXEL7WNZ3O7WM24Z2DKY2TDVKMRZSHMEPMNEL2LR7BY7DBQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390957991 (13h53m7.135014851s) [noise=true , address=104.36.82.47:4523 , piece_id=MJIMHSO66V7OWFFK2UALPKSI7CGGCTHNPZINDXMVIHDLENIW5LFA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.391483291 (13h53m7.135540162s) [noise=true , address=104.36.82.5:4861 , piece_id=GRDGO5LNCETQWRU3M6LACGRW5MNA2KFLXUBT5R2UH6SEGWP6RVRQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.391931094 (13h53m7.135987964s) [noise=true , address=104.36.82.10:4090 , piece_id=V6NPPVIYEH6WLIW7OMT3NDFPR7UDWIT46GW6AFCAWUJTBLC3O55Q, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392250266 (13h53m7.136307136s) [noise=true , address=104.36.82.62:4439 , piece_id=ILRD4TEFW4PSLV4EGPORG3FM6U2HB3WSL5KP3PN566FAOGWODLYQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392611689 (13h53m7.136668559s) [noise=true , address=104.36.82.54:4160 , piece_id=GDBORRYX5QBQGH7ZMMUUAV7AAR5UHZOTK5FTQUPC7YKZC2KD7I3A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392917410 (13h53m7.136974282s) [noise=true , address=104.36.82.23:4477 , piece_id=HAF6424ZCIBGSR4US5LN5ZWE5JMNMLFGUMQQ4YP2ARP7PQXBH5GQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393270536 (13h53m7.137327398s) [noise=true , address=104.36.82.44:4881 , piece_id=XXVSXG3ILFXG4IVMELHL45TRRHDFVVJMJ5LGOHU5EDHG6XXCDH6A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393582635 (13h53m7.137639498s) [noise=true , address=104.36.82.37:4665 , piece_id=RUKKGC4Y3FIVLZO2GRWJN3J6P36VWP5VSC2HCLEU3MZDDFXODJAQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393856926 (13h53m7.137913788s) [noise=true , address=104.36.82.11:4128 , piece_id=XRPULHI7BYDF2JOIASSN5QKGEAC6FJS2F5C22WIY7EKYILRHZ73A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394196364 (13h53m7.138253227s) [noise=true , address=104.36.82.38:4706 , piece_id=BIHIPQX6LRWBNFN4SDC3K3V22AMGL3RC5B4RVE7QXJFKB4XDN3BA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394484397 (13h53m7.138541257s) [noise=true , address=104.36.82.61:4381 , piece_id=TTZOSRP6YPL6P4C6PY5S4K7Z6CIZJDPX7NUNBCK73CWZPCA3HQHA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394593709 (13h53m7.138650578s) [noise=true , address=45.91.116.5:4300 , piece_id=MADJCANN2O35MKXT2PIR5PGMJ3PNTAQAJNNGOK6ISJNVAH2L7YIA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.395093013 (13h53m7.139149879s) [noise=true , address=104.36.82.22:4530 , piece_id=HMUUW5GYH6TRHYXP4YXBHD566FWQRILEHI53AVURA5QTQGG6USFA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.395447945 (13h53m7.139504805s) [noise=true , address=104.36.82.58:4293 , piece_id=I2ETFXYTHYWYJYSHTVOBQTO6ZPGNMA4D3NVKYDE2W34EH4SQE23A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.954469844 (13h53m7.698526694s) [upload=14294]: Next returned nil segment Aug 7 23:32:45.954487953 (13h53m7.69854487s) [upload=14294]: Waiting for error group managing segments... Aug 7 23:32:46.126888627 (13h53m7.870945474s) [noise=true , address=104.36.82.10:4090 , piece_id=V6NPPVIYEH6WLIW7OMT3NDFPR7UDWIT46GW6AFCAWUJTBLC3O55Q, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.132983844 (13h53m7.877040704s) [noise=true , address=104.36.82.16:4321 , piece_id=YTF4RWV2SD6Z4UV4XRYPBRGPCTSUTC6TQKL3DHVKU65ALTNA4TEQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.138277216 (13h53m7.882334067s) [noise=true , address=104.36.82.23:4477 , piece_id=HAF6424ZCIBGSR4US5LN5ZWE5JMNMLFGUMQQ4YP2ARP7PQXBH5GQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.142033326 (13h53m7.886090185s) [noise=true , address=104.36.82.34:4566 , piece_id=3YEAUSXEL7WNZ3O7WM24Z2DKY2TDVKMRZSHMEPMNEL2LR7BY7DBQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.151247206 (13h53m7.895304058s) [noise=true , address=104.36.82.22:4530 , piece_id=HMUUW5GYH6TRHYXP4YXBHD566FWQRILEHI53AVURA5QTQGG6USFA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.151981081 (13h53m7.896037932s) [noise=true , address=104.36.82.58:4293 , piece_id=I2ETFXYTHYWYJYSHTVOBQTO6ZPGNMA4D3NVKYDE2W34EH4SQE23A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.165022882 (13h53m7.909079743s) [noise=true , address=104.36.82.44:4881 , piece_id=XXVSXG3ILFXG4IVMELHL45TRRHDFVVJMJ5LGOHU5EDHG6XXCDH6A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.184418821 (13h53m7.928475667s) [noise=true , address=104.36.82.47:4523 , piece_id=MJIMHSO66V7OWFFK2UALPKSI7CGGCTHNPZINDXMVIHDLENIW5LFA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.184532669 (13h53m7.928589516s) [noise=true , address=45.91.116.5:4300 , piece_id=MADJCANN2O35MKXT2PIR5PGMJ3PNTAQAJNNGOK6ISJNVAH2L7YIA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.193375808 (13h53m7.937432656s) [noise=true , address=104.36.82.54:4160 , piece_id=GDBORRYX5QBQGH7ZMMUUAV7AAR5UHZOTK5FTQUPC7YKZC2KD7I3A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.204247501 (13h53m7.948304372s) [noise=true , address=104.36.82.38:4706 , piece_id=BIHIPQX6LRWBNFN4SDC3K3V22AMGL3RC5B4RVE7QXJFKB4XDN3BA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.235225782 (13h53m7.979282629s) [noise=true , address=104.36.82.62:4439 , piece_id=ILRD4TEFW4PSLV4EGPORG3FM6U2HB3WSL5KP3PN566FAOGWODLYQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.269129867 (13h53m8.013186721s) [noise=true , address=104.36.82.5:4861 , piece_id=GRDGO5LNCETQWRU3M6LACGRW5MNA2KFLXUBT5R2UH6SEGWP6RVRQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.278216912 (13h53m8.022273764s) [noise=true , address=104.36.82.19:4405 , piece_id=MHM3MWZJFTXSJT7PTOFSWHUYO3L6NIBUV6VU66XXZDH3MC3EFR2Q, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.278247847 (13h53m8.022304708s) [seg_pos={0 0}, upload=14294]: Segment reached optimal threshold of 14 pieces. Aug 7 23:32:46.278485612 (13h53m8.022542463s) [noise=true , address=104.36.82.11:4128 , piece_id=XRPULHI7BYDF2JOIASSN5QKGEAC6FJS2F5C22WIY7EKYILRHZ73A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wABkAtAjoYkKyNoAiaVCxSvY6WgvHvAYFPCjpNojtpiYiGtDS): protocol: expected piece hash; context canceled; EOF Aug 7 23:32:46.278511698 (13h53m8.022568566s) [noise=true , address=77.90.31.12:4593 , piece_id=C3RVQHFS4L7SKPFFIZ52MAQWGD7P4JNG3SNPCNY53SQUD72PZRUA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kSxoZb9nmG6oGNytpFTsoiGh3ZPmebVVe8TZxGzANaw7EfdeF): protocol: expected piece hash; context canceled; EOF Aug 7 23:32:46.278642807 (13h53m8.022699657s) [noise=true , address=104.36.82.61:4381 , piece_id=TTZOSRP6YPL6P4C6PY5S4K7Z6CIZJDPX7NUNBCK73CWZPCA3HQHA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zU35MWWaaC1npkP5qGcWBXAkfxGsd5UsdkM2nwX9x7wUGCDbY): protocol: write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection; write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection Aug 7 23:32:46.278650926 (13h53m8.022707786s) [noise=true , address=104.36.82.37:4665 , piece_id=RUKKGC4Y3FIVLZO2GRWJN3J6P36VWP5VSC2HCLEU3MZDDFXODJAQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jP9JrCmoQSdqrzXZxkNN1ihdFkaUMjGZss6Fh19zbL3YyWj5N): protocol: write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection; write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection Aug 7 23:32:46.278674050 (13h53m8.022730911s) [seg_pos={0 0}, upload=14294]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:32:46.278751154 (13h53m8.022808014s) [upload=14294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:32:46.278756168 (13h53m8.022813029s) [upload=14294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:32:46.498660020 (13h53m8.242716882s) [upload=14294]: Done uploading segments... Aug 7 23:32:46.498701263 (13h53m8.242758128s) [upload=14294]: Upload finished. err: <nil> Duration: 3.633735044s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 1.644146802s FAIL (2025-08-07 23:59:49 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== pload=14121]: Done uploading segments... Aug 7 23:22:28.505845197 (13h42m50.249902044s) [upload=14121]: Upload finished. err: <nil> Aug 7 23:23:10.377483605 (13h43m32.121540458s) [upload=14132]: Starting upload Aug 7 23:23:10.377535635 (13h43m32.121592495s) [upload=14132]: Done starting upload Aug 7 23:23:10.377603797 (13h43m32.121660647s) [upload=14132]: Uploading segments... Aug 7 23:23:10.377638935 (13h43m32.121695804s) [upload=14132]: Got next segment. Inline: true Aug 7 23:23:10.377643594 (13h43m32.121700454s) [upload=14132]: Waiting for error group managing segments... Aug 7 23:23:10.377688252 (13h43m32.121745104s) [upload=14132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:23:10.377693427 (13h43m32.121750286s) [upload=14132]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:23:10.377696805 (13h43m32.121753666s) [upload=14132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:23:10.562260724 (13h43m32.306317573s) [upload=14132]: Done uploading segments... Aug 7 23:23:10.562279228 (13h43m32.306336078s) [upload=14132]: Upload finished. err: <nil> Aug 7 23:24:07.896404858 (13h44m29.640461721s) [upload=14147]: Starting upload Aug 7 23:24:07.896490216 (13h44m29.640547079s) [upload=14147]: Done starting upload Aug 7 23:24:07.896580488 (13h44m29.640637339s) [upload=14147]: Uploading segments... Aug 7 23:24:07.896655006 (13h44m29.640711874s) [upload=14147]: Got next segment. Inline: true Aug 7 23:24:07.896660179 (13h44m29.640717039s) [upload=14147]: Waiting for error group managing segments... Aug 7 23:24:07.896714840 (13h44m29.6407717s) [upload=14147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:24:07.896719399 (13h44m29.64077626s) [upload=14147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:24:07.896722726 (13h44m29.640779586s) [upload=14147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:24:08.048898130 (13h44m29.792954991s) [upload=14147]: Done uploading segments... Aug 7 23:24:08.048952526 (13h44m29.793009386s) [upload=14147]: Upload finished. err: <nil> Aug 7 23:24:53.794754379 (13h45m15.53881125s) [upload=14162]: Starting upload Aug 7 23:24:53.794842317 (13h45m15.538899177s) [upload=14162]: Done starting upload Aug 7 23:24:53.794935964 (13h45m15.538992814s) [upload=14162]: Uploading segments... Aug 7 23:24:53.795003752 (13h45m15.539060623s) [upload=14162]: Got next segment. Inline: true Aug 7 23:24:53.795010814 (13h45m15.539067676s) [upload=14162]: Waiting for error group managing segments... Aug 7 23:24:53.795066607 (13h45m15.539123488s) [upload=14162]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:24:53.795158449 (13h45m15.5392153s) [upload=14162]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:24:53.795163166 (13h45m15.539220026s) [upload=14162]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:24:53.946994982 (13h45m15.691051833s) [upload=14162]: Done uploading segments... Aug 7 23:24:53.947013862 (13h45m15.691070711s) [upload=14162]: Upload finished. err: <nil> Aug 7 23:25:40.029856966 (13h46m1.773913836s) [upload=14177]: Starting upload Aug 7 23:25:40.029986938 (13h46m1.774043809s) [upload=14177]: Done starting upload Aug 7 23:25:40.030076796 (13h46m1.774133642s) [upload=14177]: Uploading segments... Aug 7 23:25:40.030132567 (13h46m1.774189438s) [upload=14177]: Got next segment. Inline: true Aug 7 23:25:40.030139885 (13h46m1.774196757s) [upload=14177]: Waiting for error group managing segments... Aug 7 23:25:40.030197588 (13h46m1.774254461s) [upload=14177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:25:40.030204146 (13h46m1.774261016s) [upload=14177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:25:40.030207984 (13h46m1.77426485s) [upload=14177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:25:40.184198742 (13h46m1.928255592s) [upload=14177]: Done uploading segments... Aug 7 23:25:40.184219017 (13h46m1.928275867s) [upload=14177]: Upload finished. err: <nil> Aug 7 23:26:54.625649110 (13h47m16.369705971s) [upload=14198]: Starting upload Aug 7 23:26:54.625738563 (13h47m16.369795424s) [upload=14198]: Done starting upload Aug 7 23:26:54.625842691 (13h47m16.369899539s) [upload=14198]: Uploading segments... Aug 7 23:26:54.626016839 (13h47m16.37007371s) [upload=14198]: Got next segment. Inline: true Aug 7 23:26:54.626052112 (13h47m16.370108973s) [upload=14198]: Waiting for error group managing segments... Aug 7 23:26:54.626129780 (13h47m16.370186645s) [upload=14198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:26:54.626137392 (13h47m16.370194264s) [upload=14198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:26:54.626143182 (13h47m16.370200048s) [upload=14198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:26:54.775804171 (13h47m16.519861021s) [upload=14198]: Done uploading segments... Aug 7 23:26:54.775824249 (13h47m16.519881102s) [upload=14198]: Upload finished. err: <nil> Aug 7 23:27:32.263746490 (13h47m54.007803361s) [upload=14208]: Starting upload Aug 7 23:27:32.263833655 (13h47m54.007890501s) [upload=14208]: Done starting upload Aug 7 23:27:32.263976536 (13h47m54.008033385s) [upload=14208]: Uploading segments... Aug 7 23:27:32.264070926 (13h47m54.008127796s) [upload=14208]: Got next segment. Inline: true Aug 7 23:27:32.264080356 (13h47m54.00813733s) [upload=14208]: Waiting for error group managing segments... Aug 7 23:27:32.264154657 (13h47m54.008211526s) [upload=14208]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:27:32.264163587 (13h47m54.008220454s) [upload=14208]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:27:32.264172017 (13h47m54.008228914s) [upload=14208]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:27:32.490188559 (13h47m54.23424541s) [upload=14208]: Done uploading segments... Aug 7 23:27:32.490207326 (13h47m54.234264172s) [upload=14208]: Upload finished. err: <nil> Aug 7 23:27:57.927940477 (13h48m19.671997323s) [upload=14217]: Starting upload Aug 7 23:27:57.928016423 (13h48m19.672073283s) [upload=14217]: Done starting upload Aug 7 23:27:57.928101290 (13h48m19.672158142s) [upload=14217]: Uploading segments... Aug 7 23:27:57.928155692 (13h48m19.672212566s) [upload=14217]: Got next segment. Inline: true Aug 7 23:27:57.928162009 (13h48m19.672218869s) [upload=14217]: Waiting for error group managing segments... Aug 7 23:27:57.928213294 (13h48m19.672270145s) [upload=14217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:27:57.928219212 (13h48m19.672276074s) [upload=14217]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:27:57.928223521 (13h48m19.672280383s) [upload=14217]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:27:58.087544005 (13h48m19.831600874s) [upload=14217]: Done uploading segments... Aug 7 23:27:58.087570226 (13h48m19.831627094s) [upload=14217]: Upload finished. err: <nil> Aug 7 23:29:12.765200935 (13h49m34.509257795s) [upload=14237]: Starting upload Aug 7 23:29:12.765253106 (13h49m34.509309977s) [upload=14237]: Done starting upload Aug 7 23:29:12.765324641 (13h49m34.509381499s) [upload=14237]: Uploading segments... Aug 7 23:29:12.765374187 (13h49m34.509431056s) [upload=14237]: Got next segment. Inline: true Aug 7 23:29:12.765378947 (13h49m34.509435807s) [upload=14237]: Waiting for error group managing segments... Aug 7 23:29:12.765430859 (13h49m34.50948772s) [upload=14237]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:29:12.765438106 (13h49m34.50949497s) [upload=14237]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:29:12.765443127 (13h49m34.509499988s) [upload=14237]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:29:12.926437674 (13h49m34.670494535s) [upload=14237]: Done uploading segments... Aug 7 23:29:12.926457418 (13h49m34.670514268s) [upload=14237]: Upload finished. err: <nil> Aug 7 23:29:42.225760316 (13h50m3.969817177s) [upload=14244]: Starting upload Aug 7 23:29:42.225855063 (13h50m3.969911929s) [upload=14244]: Done starting upload Aug 7 23:29:42.225962489 (13h50m3.970019355s) [upload=14244]: Uploading segments... Aug 7 23:29:42.226078484 (13h50m3.970135331s) [upload=14244]: Got next segment. Inline: true Aug 7 23:29:42.226083996 (13h50m3.970140842s) [upload=14244]: Waiting for error group managing segments... Aug 7 23:29:42.226143067 (13h50m3.970199914s) [upload=14244]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:29:42.226148511 (13h50m3.970205371s) [upload=14244]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:29:42.226153006 (13h50m3.970209865s) [upload=14244]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:29:42.401822947 (13h50m4.14587981s) [upload=14244]: Done uploading segments... Aug 7 23:29:42.401847556 (13h50m4.145904405s) [upload=14244]: Upload finished. err: <nil> Aug 7 23:30:31.529840301 (13h50m53.273897165s) [upload=14254]: Starting upload Aug 7 23:30:31.530002531 (13h50m53.274059378s) [upload=14254]: Done starting upload Aug 7 23:30:31.530201544 (13h50m53.274258391s) [upload=14254]: Uploading segments... Aug 7 23:30:31.530298120 (13h50m53.274354985s) [upload=14254]: Got next segment. Inline: true Aug 7 23:30:31.530303457 (13h50m53.274360317s) [upload=14254]: Waiting for error group managing segments... Aug 7 23:30:31.530353721 (13h50m53.274410567s) [upload=14254]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:30:31.530358927 (13h50m53.274415788s) [upload=14254]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:30:31.530363758 (13h50m53.274420619s) [upload=14254]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:30:31.696350967 (13h50m53.440407828s) [upload=14254]: Done uploading segments... Aug 7 23:30:31.696369460 (13h50m53.440426307s) [upload=14254]: Upload finished. err: <nil> Duration: 1.64417643s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 14 out of 60 is failed (threshold=3) [.XXXXXX..XX.X...................................X...XX.X...X] duration: 30.00082746s FAIL (2025-08-07 23:59:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== . err: <nil> Aug 7 23:52:25.641252610 (14h12m47.385309456s) [upload=14634]: Starting upload Aug 7 23:52:25.641347708 (14h12m47.385404571s) [upload=14634]: Done starting upload Aug 7 23:52:25.641412738 (14h12m47.385469584s) [upload=14634]: Uploading segments... Aug 7 23:52:25.641487429 (14h12m47.385544276s) [upload=14634]: Got next segment. Inline: true Aug 7 23:52:25.641494756 (14h12m47.385551602s) [upload=14634]: Waiting for error group managing segments... Aug 7 23:52:25.641535502 (14h12m47.385592365s) [upload=14634]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:25.641556088 (14h12m47.385612953s) [upload=14634]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:25.641559364 (14h12m47.385616224s) [upload=14634]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:25.852337418 (14h12m47.596394279s) [upload=14634]: Done uploading segments... Aug 7 23:52:25.852355736 (14h12m47.596412584s) [upload=14634]: Upload finished. err: <nil> Aug 7 23:52:51.292140355 (14h13m13.036197215s) [upload=14641]: Starting upload Aug 7 23:52:51.292216133 (14h13m13.036272985s) [upload=14641]: Done starting upload Aug 7 23:52:51.292307446 (14h13m13.036364307s) [upload=14641]: Uploading segments... Aug 7 23:52:51.292389087 (14h13m13.036445956s) [upload=14641]: Got next segment. Inline: true Aug 7 23:52:51.292394933 (14h13m13.036451793s) [upload=14641]: Waiting for error group managing segments... Aug 7 23:52:51.292456032 (14h13m13.036512883s) [upload=14641]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:51.292461787 (14h13m13.036518647s) [upload=14641]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:51.292467676 (14h13m13.036524537s) [upload=14641]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:51.436915828 (14h13m13.180972678s) [upload=14641]: Done uploading segments... Aug 7 23:52:51.436934458 (14h13m13.18099131s) [upload=14641]: Upload finished. err: <nil> Aug 7 23:53:10.946261153 (14h13m32.690318028s) [upload=14646]: Starting upload Aug 7 23:53:10.946350198 (14h13m32.690407065s) [upload=14646]: Done starting upload Aug 7 23:53:10.946435423 (14h13m32.690492276s) [upload=14646]: Uploading segments... Aug 7 23:53:10.946489255 (14h13m32.690546124s) [upload=14646]: Got next segment. Inline: true Aug 7 23:53:10.946497660 (14h13m32.690554519s) [upload=14646]: Waiting for error group managing segments... Aug 7 23:53:10.946542678 (14h13m32.690599529s) [upload=14646]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:10.946548267 (14h13m32.690605128s) [upload=14646]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:10.946553792 (14h13m32.690610652s) [upload=14646]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:14.554651436 (14h13m36.298708309s) [upload=14646]: Done uploading segments... Aug 7 23:53:14.554675183 (14h13m36.298732045s) [upload=14646]: Upload finished. err: <nil> Aug 7 23:53:53.584502029 (14h14m15.328558897s) [upload=14658]: Starting upload Aug 7 23:53:53.584599757 (14h14m15.328656616s) [upload=14658]: Done starting upload Aug 7 23:53:53.584702364 (14h14m15.328759218s) [upload=14658]: Uploading segments... Aug 7 23:53:53.584798610 (14h14m15.32885548s) [upload=14658]: Got next segment. Inline: true Aug 7 23:53:53.584806594 (14h14m15.328863461s) [upload=14658]: Waiting for error group managing segments... Aug 7 23:53:53.584875193 (14h14m15.328932062s) [upload=14658]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:53.584882324 (14h14m15.328939193s) [upload=14658]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:53.584901657 (14h14m15.328958528s) [upload=14658]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:23.318821682 (14h14m45.062878542s) [upload=14658]: Done uploading segments... Aug 7 23:54:23.318863033 (14h14m45.062919884s) [upload=14658]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:36.775209359 (14h14m58.519266209s) [upload=14671]: Starting upload Aug 7 23:54:36.775292235 (14h14m58.519349112s) [upload=14671]: Done starting upload Aug 7 23:54:36.775412031 (14h14m58.519468883s) [upload=14671]: Uploading segments... Aug 7 23:54:36.775480767 (14h14m58.519537636s) [upload=14671]: Got next segment. Inline: true Aug 7 23:54:36.775487320 (14h14m58.519544181s) [upload=14671]: Waiting for error group managing segments... Aug 7 23:54:36.775539752 (14h14m58.519596612s) [upload=14671]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:36.775545893 (14h14m58.519602753s) [upload=14671]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:36.775549619 (14h14m58.519606479s) [upload=14671]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:36.926276427 (14h14m58.670333277s) [upload=14671]: Done uploading segments... Aug 7 23:54:36.926295206 (14h14m58.670352057s) [upload=14671]: Upload finished. err: <nil> Aug 7 23:54:57.675347843 (14h15m19.419404694s) [upload=14676]: Starting upload Aug 7 23:54:57.675412352 (14h15m19.419469212s) [upload=14676]: Done starting upload Aug 7 23:54:57.675475670 (14h15m19.41953252s) [upload=14676]: Uploading segments... Aug 7 23:54:57.675516336 (14h15m19.419573205s) [upload=14676]: Got next segment. Inline: true Aug 7 23:54:57.675521071 (14h15m19.419577931s) [upload=14676]: Waiting for error group managing segments... Aug 7 23:54:57.675565512 (14h15m19.419622375s) [upload=14676]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:57.675574255 (14h15m19.419631202s) [upload=14676]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:57.675580694 (14h15m19.419637563s) [upload=14676]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:27.407552485 (14h15m49.151609344s) [upload=14676]: Done uploading segments... Aug 7 23:55:27.407592978 (14h15m49.151649829s) [upload=14676]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:56:06.851646232 (14h16m28.595703082s) [upload=14696]: Starting upload Aug 7 23:56:06.851717330 (14h16m28.595774191s) [upload=14696]: Done starting upload Aug 7 23:56:06.851786008 (14h16m28.59584286s) [upload=14696]: Uploading segments... Aug 7 23:56:06.851834232 (14h16m28.595891101s) [upload=14696]: Got next segment. Inline: true Aug 7 23:56:06.851839675 (14h16m28.595896535s) [upload=14696]: Waiting for error group managing segments... Aug 7 23:56:06.851896275 (14h16m28.595953125s) [upload=14696]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:06.851901659 (14h16m28.59595852s) [upload=14696]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:06.851905505 (14h16m28.595962364s) [upload=14696]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:07.006202008 (14h16m28.750258868s) [upload=14696]: Done uploading segments... Aug 7 23:56:07.006222000 (14h16m28.750278861s) [upload=14696]: Upload finished. err: <nil> Aug 7 23:56:59.020763574 (14h17m20.764820421s) [upload=14712]: Starting upload Aug 7 23:56:59.020858955 (14h17m20.764915806s) [upload=14712]: Done starting upload Aug 7 23:56:59.020935444 (14h17m20.764992295s) [upload=14712]: Uploading segments... Aug 7 23:56:59.021007501 (14h17m20.76506437s) [upload=14712]: Got next segment. Inline: true Aug 7 23:56:59.021013631 (14h17m20.765070491s) [upload=14712]: Waiting for error group managing segments... Aug 7 23:56:59.021070471 (14h17m20.76512733s) [upload=14712]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:59.021076966 (14h17m20.765133821s) [upload=14712]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:59.021081305 (14h17m20.76513816s) [upload=14712]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:59.154820708 (14h17m20.898877569s) [upload=14712]: Done uploading segments... Aug 7 23:56:59.154839835 (14h17m20.898896685s) [upload=14712]: Upload finished. err: <nil> Aug 7 23:58:10.963677657 (14h18m32.707734508s) [upload=14728]: Starting upload Aug 7 23:58:10.963770332 (14h18m32.70782718s) [upload=14728]: Done starting upload Aug 7 23:58:10.963851725 (14h18m32.707908586s) [upload=14728]: Uploading segments... Aug 7 23:58:10.963901559 (14h18m32.707958431s) [upload=14728]: Got next segment. Inline: true Aug 7 23:58:10.963907663 (14h18m32.707964525s) [upload=14728]: Waiting for error group managing segments... Aug 7 23:58:10.963960167 (14h18m32.708017032s) [upload=14728]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:10.963966034 (14h18m32.708022896s) [upload=14728]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:10.963969944 (14h18m32.708026806s) [upload=14728]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:11.106069715 (14h18m32.850126571s) [upload=14728]: Done uploading segments... Aug 7 23:58:11.106112099 (14h18m32.850168961s) [upload=14728]: Upload finished. err: <nil> Aug 7 23:59:19.078128070 (14h19m40.822184931s) [upload=14749]: Starting upload Aug 7 23:59:19.078217499 (14h19m40.822274359s) [upload=14749]: Done starting upload Aug 7 23:59:19.078291040 (14h19m40.822347895s) [upload=14749]: Uploading segments... Aug 7 23:59:19.078333863 (14h19m40.822390732s) [upload=14749]: Got next segment. Inline: true Aug 7 23:59:19.078339134 (14h19m40.822395984s) [upload=14749]: Waiting for error group managing segments... Aug 7 23:59:19.078391868 (14h19m40.822448717s) [upload=14749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:19.078397213 (14h19m40.822454064s) [upload=14749]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:19.078401054 (14h19m40.822457905s) [upload=14749]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:59:48.808547234 (14h20m10.552604081s) [upload=14749]: Done uploading segments... Aug 7 23:59:48.808568563 (14h20m10.552625409s) [upload=14749]: Upload finished. err: metaclient: context deadline exceeded Duration: 30.00085624s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 14 out of 60 is failed (threshold=3) [XXXXXX.XX........................................X.XX...XX.X] duration: 30.000582272s FAIL (2025-08-07 23:59:48 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 30.000612345s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [XXXXXXXX...........X............................XX.X.X...XX.] duration: 2.631626767s FAIL (2025-08-07 23:59:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== 6]: Uploading piece... Aug 7 23:57:10.893816491 (14h17m32.637873354s) [noise=true , address=45.91.116.12:4225 , piece_id=NTUY2VEJFWSFOAGWUPTJYNYNWL3XMK4C3FJUE6MSTMTS2TZJ7VUA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.894285524 (14h17m32.638342391s) [noise=true , address=104.36.82.16:4294 , piece_id=SIMVCA63FAX5EQZJO5TBOSGPVYLFOAICTABKMXS5SCQGNOUPFBIA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.895348932 (14h17m32.639405787s) [noise=true , address=77.90.31.14:4661 , piece_id=WFDXKM726LBL4MH4CJ4AQIUSXTQCQ3BJV34QBUGWPHNFFPZYGZKA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.895932061 (14h17m32.639988921s) [noise=true , address=104.36.82.10:4078 , piece_id=3DPLCXZLQWMZSYSSO3JPLZFP2VXJ5NKYO4FDVFYOWJLT43C7XSTQ, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.896545320 (14h17m32.640602184s) [noise=true , address=77.90.31.10:4518 , piece_id=LU6UTDDO6HUIGXV2QXCL45CIBFQMOQGS7CTQHBRKCBE2JJKLI6FA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.897117472 (14h17m32.641174341s) [noise=true , address=77.90.31.21:4877 , piece_id=SF7XDMNH7KGS26UE6MKUK3VLFZ6IOP6WZKMVO4H6SIAULM623GLA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.897621325 (14h17m32.641678192s) [noise=true , address=77.90.31.12:4589 , piece_id=5I5BMAQARX67CB2RRMCXG2U7KELJ6HSHRT3573EH3ZX6NJ257R4A, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.898101338 (14h17m32.642158204s) [noise=true , address=77.90.31.21:4878 , piece_id=XAXEYDOX54GUKMI22UKTA6BSTSY5WYATTITTGSRQ3BBYDGDCO5DQ, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.898693607 (14h17m32.642750474s) [noise=true , address=45.91.116.6:4975 , piece_id=VDEP57UE6G4CBU7DOYMMOIRXJUTTWPJKDJDGCGUOHBGVPAYCM6PA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.900512475 (14h17m32.644569335s) [noise=true , address=45.91.116.12:4227 , piece_id=J2FM3FAWY5HYXX3VFC6CSHUSWDQDNKUUB6NL6MWSF2ZG3BZFWBEA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.902337804 (14h17m32.646394669s) [noise=true , address=77.90.31.13:4626 , piece_id=PR6GY2Y7HDUORSLYZXU4IVLVA6RIMBFSRFJW4OTLXUZM7MJZ4KOQ, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.904215667 (14h17m32.648272537s) [noise=true , address=77.90.31.19:4841 , piece_id=Q3XFTIVGOSRQARSFONAKX26VAFTJF7THPIQV6USKI7N2LN3JBTLQ, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.906034481 (14h17m32.650091365s) [noise=true , address=104.36.82.5:4839 , piece_id=AV3ZBQ4W44IZEDAPHJGHKK5D5VRNU5XGOQGDWEARNGIKWRQAGYBQ, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.907857426 (14h17m32.651914295s) [noise=true , address=77.90.31.15:4698 , piece_id=WBU562MADMX3KSCRCD2H5R7MQN5IPFBLKYSKU47WMGYYBKZGPYJA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.908666188 (14h17m32.652723069s) [noise=true , address=104.36.82.14:4222 , piece_id=JFWGS6VCDTFOQYG32ZFTWNYLFGVVQPXD7K6TZXC4FR2SGEOUS7FA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:11.510238622 (14h17m33.254295484s) [upload=14716]: Next returned nil segment Aug 7 23:57:11.510261464 (14h17m33.254318324s) [upload=14716]: Waiting for error group managing segments... Aug 7 23:57:11.657094388 (14h17m33.401151236s) [noise=true , address=77.90.31.10:4518 , piece_id=LU6UTDDO6HUIGXV2QXCL45CIBFQMOQGS7CTQHBRKCBE2JJKLI6FA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.674408374 (14h17m33.418465234s) [noise=true , address=77.90.31.15:4698 , piece_id=WBU562MADMX3KSCRCD2H5R7MQN5IPFBLKYSKU47WMGYYBKZGPYJA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.681203586 (14h17m33.425260449s) [noise=true , address=77.90.31.12:4589 , piece_id=5I5BMAQARX67CB2RRMCXG2U7KELJ6HSHRT3573EH3ZX6NJ257R4A, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.682780850 (14h17m33.4268377s) [noise=true , address=77.90.31.21:4877 , piece_id=SF7XDMNH7KGS26UE6MKUK3VLFZ6IOP6WZKMVO4H6SIAULM623GLA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.684042315 (14h17m33.428099176s) [noise=true , address=104.36.82.16:4294 , piece_id=SIMVCA63FAX5EQZJO5TBOSGPVYLFOAICTABKMXS5SCQGNOUPFBIA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.719658547 (14h17m33.463715401s) [noise=true , address=45.91.116.12:4225 , piece_id=NTUY2VEJFWSFOAGWUPTJYNYNWL3XMK4C3FJUE6MSTMTS2TZJ7VUA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.730348391 (14h17m33.474405242s) [noise=true , address=77.90.31.21:4878 , piece_id=XAXEYDOX54GUKMI22UKTA6BSTSY5WYATTITTGSRQ3BBYDGDCO5DQ, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.748264399 (14h17m33.492321249s) [noise=true , address=77.90.31.19:4841 , piece_id=Q3XFTIVGOSRQARSFONAKX26VAFTJF7THPIQV6USKI7N2LN3JBTLQ, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.769764028 (14h17m33.513820878s) [noise=true , address=45.91.116.6:4975 , piece_id=VDEP57UE6G4CBU7DOYMMOIRXJUTTWPJKDJDGCGUOHBGVPAYCM6PA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.774177874 (14h17m33.518234733s) [noise=true , address=77.90.31.14:4661 , piece_id=WFDXKM726LBL4MH4CJ4AQIUSXTQCQ3BJV34QBUGWPHNFFPZYGZKA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.789864472 (14h17m33.533921326s) [noise=true , address=45.91.116.13:4153 , piece_id=WBFFQT5I6ZR5ZKUJRFULNOJZ455KIJMA5EEATDDOCVFUKRD67NFA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.799519597 (14h17m33.543576447s) [noise=true , address=45.91.116.12:4227 , piece_id=J2FM3FAWY5HYXX3VFC6CSHUSWDQDNKUUB6NL6MWSF2ZG3BZFWBEA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.800753821 (14h17m33.544810667s) [noise=true , address=77.90.31.13:4626 , piece_id=PR6GY2Y7HDUORSLYZXU4IVLVA6RIMBFSRFJW4OTLXUZM7MJZ4KOQ, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.800785983 (14h17m33.544842842s) [seg_pos={0 0}, upload=14716]: Segment reached optimal threshold of 13 pieces. Aug 7 23:57:11.801026052 (14h17m33.545082898s) [noise=true , address=104.36.82.14:4222 , piece_id=JFWGS6VCDTFOQYG32ZFTWNYLFGVVQPXD7K6TZXC4FR2SGEOUS7FA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1): protocol: expected piece hash; context canceled; EOF Aug 7 23:57:11.801098020 (14h17m33.545154867s) [noise=true , address=104.36.82.10:4078 , piece_id=3DPLCXZLQWMZSYSSO3JPLZFP2VXJ5NKYO4FDVFYOWJLT43C7XSTQ, seg_pos={0 0}, upload=14716]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Aug 7 23:57:11.801174944 (14h17m33.54523179s) [noise=true , address=104.36.82.5:4839 , piece_id=AV3ZBQ4W44IZEDAPHJGHKK5D5VRNU5XGOQGDWEARNGIKWRQAGYBQ, seg_pos={0 0}, upload=14716]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 139.59.144.22:48474->104.36.82.5:4839: use of closed network connection; write tcp 139.59.144.22:48474->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48474->104.36.82.5:4839: use of closed network connection Aug 7 23:57:11.801184926 (14h17m33.545241773s) [seg_pos={0 0}, upload=14716]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:57:11.801257096 (14h17m33.545313956s) [upload=14716]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:57:11.801261009 (14h17m33.545317869s) [upload=14716]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:11.942784613 (14h17m33.686841513s) [upload=14716]: Done uploading segments... Aug 7 23:57:11.942815424 (14h17m33.686872279s) [upload=14716]: Upload finished. err: <nil> Aug 7 23:58:20.561350310 (14h18m42.305407184s) [upload=14732]: Starting upload Aug 7 23:58:20.561483402 (14h18m42.30554027s) [upload=14732]: Done starting upload Aug 7 23:58:20.561650297 (14h18m42.305707159s) [upload=14732]: Uploading segments... Aug 7 23:58:20.562196341 (14h18m42.306253212s) [upload=14732]: Got next segment. Inline: false Aug 7 23:58:20.562334166 (14h18m42.306391031s) [upload=14732]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:20.562343255 (14h18m42.306400125s) [upload=14732]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:58:50.281195862 (14h19m12.025252712s) [upload=14732]: Next segment err: context deadline exceeded Aug 7 23:58:50.281383705 (14h19m12.025440555s) [upload=14732]: Done uploading segments... Aug 7 23:58:50.281405600 (14h19m12.02546245s) [upload=14732]: Upload finished. err: context deadline exceeded Aug 7 23:58:50.716777496 (14h19m12.460834346s) [upload=14742]: Starting upload Aug 7 23:58:50.716857905 (14h19m12.460914751s) [upload=14742]: Done starting upload Aug 7 23:58:50.716999972 (14h19m12.461056825s) [upload=14742]: Uploading segments... Aug 7 23:58:50.717186124 (14h19m12.461242976s) [upload=14742]: Got next segment. Inline: false Aug 7 23:58:50.717245601 (14h19m12.461302466s) [upload=14742]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:50.717252185 (14h19m12.461309053s) [upload=14742]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:59:20.436529435 (14h19m42.180586305s) [upload=14742]: Next segment err: context deadline exceeded Aug 7 23:59:20.436734882 (14h19m42.180791733s) [upload=14742]: Done uploading segments... Aug 7 23:59:20.436756200 (14h19m42.180813054s) [upload=14742]: Upload finished. err: context deadline exceeded Duration: 2.631731966s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [XX..X.......................................................] duration: 2.731116216s FAIL (2025-08-07 23:59:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.73142107s edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 304.036366ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 23:59:31 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 304.065981ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 106.509355ms remote: "tcp:185.244.226.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 23:59:30 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 106.537695ms edge gateway vhost bgp us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 7 out of 60 is failed (threshold=3) [.XXX...XX.X..............................................X..] duration: 1.058649623s FAIL (2025-08-07 23:59:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.058675542s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [XXXXXXXXX...........X............................XX.X.X...XX] duration: 30.006711251s FAIL (2025-08-07 23:59:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== 6]: Uploading piece... Aug 7 23:57:10.893816491 (14h17m32.637873354s) [noise=true , address=45.91.116.12:4225 , piece_id=NTUY2VEJFWSFOAGWUPTJYNYNWL3XMK4C3FJUE6MSTMTS2TZJ7VUA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.894285524 (14h17m32.638342391s) [noise=true , address=104.36.82.16:4294 , piece_id=SIMVCA63FAX5EQZJO5TBOSGPVYLFOAICTABKMXS5SCQGNOUPFBIA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.895348932 (14h17m32.639405787s) [noise=true , address=77.90.31.14:4661 , piece_id=WFDXKM726LBL4MH4CJ4AQIUSXTQCQ3BJV34QBUGWPHNFFPZYGZKA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.895932061 (14h17m32.639988921s) [noise=true , address=104.36.82.10:4078 , piece_id=3DPLCXZLQWMZSYSSO3JPLZFP2VXJ5NKYO4FDVFYOWJLT43C7XSTQ, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.896545320 (14h17m32.640602184s) [noise=true , address=77.90.31.10:4518 , piece_id=LU6UTDDO6HUIGXV2QXCL45CIBFQMOQGS7CTQHBRKCBE2JJKLI6FA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.897117472 (14h17m32.641174341s) [noise=true , address=77.90.31.21:4877 , piece_id=SF7XDMNH7KGS26UE6MKUK3VLFZ6IOP6WZKMVO4H6SIAULM623GLA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.897621325 (14h17m32.641678192s) [noise=true , address=77.90.31.12:4589 , piece_id=5I5BMAQARX67CB2RRMCXG2U7KELJ6HSHRT3573EH3ZX6NJ257R4A, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.898101338 (14h17m32.642158204s) [noise=true , address=77.90.31.21:4878 , piece_id=XAXEYDOX54GUKMI22UKTA6BSTSY5WYATTITTGSRQ3BBYDGDCO5DQ, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.898693607 (14h17m32.642750474s) [noise=true , address=45.91.116.6:4975 , piece_id=VDEP57UE6G4CBU7DOYMMOIRXJUTTWPJKDJDGCGUOHBGVPAYCM6PA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.900512475 (14h17m32.644569335s) [noise=true , address=45.91.116.12:4227 , piece_id=J2FM3FAWY5HYXX3VFC6CSHUSWDQDNKUUB6NL6MWSF2ZG3BZFWBEA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.902337804 (14h17m32.646394669s) [noise=true , address=77.90.31.13:4626 , piece_id=PR6GY2Y7HDUORSLYZXU4IVLVA6RIMBFSRFJW4OTLXUZM7MJZ4KOQ, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.904215667 (14h17m32.648272537s) [noise=true , address=77.90.31.19:4841 , piece_id=Q3XFTIVGOSRQARSFONAKX26VAFTJF7THPIQV6USKI7N2LN3JBTLQ, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.906034481 (14h17m32.650091365s) [noise=true , address=104.36.82.5:4839 , piece_id=AV3ZBQ4W44IZEDAPHJGHKK5D5VRNU5XGOQGDWEARNGIKWRQAGYBQ, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.907857426 (14h17m32.651914295s) [noise=true , address=77.90.31.15:4698 , piece_id=WBU562MADMX3KSCRCD2H5R7MQN5IPFBLKYSKU47WMGYYBKZGPYJA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.908666188 (14h17m32.652723069s) [noise=true , address=104.36.82.14:4222 , piece_id=JFWGS6VCDTFOQYG32ZFTWNYLFGVVQPXD7K6TZXC4FR2SGEOUS7FA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:11.510238622 (14h17m33.254295484s) [upload=14716]: Next returned nil segment Aug 7 23:57:11.510261464 (14h17m33.254318324s) [upload=14716]: Waiting for error group managing segments... Aug 7 23:57:11.657094388 (14h17m33.401151236s) [noise=true , address=77.90.31.10:4518 , piece_id=LU6UTDDO6HUIGXV2QXCL45CIBFQMOQGS7CTQHBRKCBE2JJKLI6FA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.674408374 (14h17m33.418465234s) [noise=true , address=77.90.31.15:4698 , piece_id=WBU562MADMX3KSCRCD2H5R7MQN5IPFBLKYSKU47WMGYYBKZGPYJA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.681203586 (14h17m33.425260449s) [noise=true , address=77.90.31.12:4589 , piece_id=5I5BMAQARX67CB2RRMCXG2U7KELJ6HSHRT3573EH3ZX6NJ257R4A, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.682780850 (14h17m33.4268377s) [noise=true , address=77.90.31.21:4877 , piece_id=SF7XDMNH7KGS26UE6MKUK3VLFZ6IOP6WZKMVO4H6SIAULM623GLA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.684042315 (14h17m33.428099176s) [noise=true , address=104.36.82.16:4294 , piece_id=SIMVCA63FAX5EQZJO5TBOSGPVYLFOAICTABKMXS5SCQGNOUPFBIA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.719658547 (14h17m33.463715401s) [noise=true , address=45.91.116.12:4225 , piece_id=NTUY2VEJFWSFOAGWUPTJYNYNWL3XMK4C3FJUE6MSTMTS2TZJ7VUA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.730348391 (14h17m33.474405242s) [noise=true , address=77.90.31.21:4878 , piece_id=XAXEYDOX54GUKMI22UKTA6BSTSY5WYATTITTGSRQ3BBYDGDCO5DQ, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.748264399 (14h17m33.492321249s) [noise=true , address=77.90.31.19:4841 , piece_id=Q3XFTIVGOSRQARSFONAKX26VAFTJF7THPIQV6USKI7N2LN3JBTLQ, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.769764028 (14h17m33.513820878s) [noise=true , address=45.91.116.6:4975 , piece_id=VDEP57UE6G4CBU7DOYMMOIRXJUTTWPJKDJDGCGUOHBGVPAYCM6PA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.774177874 (14h17m33.518234733s) [noise=true , address=77.90.31.14:4661 , piece_id=WFDXKM726LBL4MH4CJ4AQIUSXTQCQ3BJV34QBUGWPHNFFPZYGZKA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.789864472 (14h17m33.533921326s) [noise=true , address=45.91.116.13:4153 , piece_id=WBFFQT5I6ZR5ZKUJRFULNOJZ455KIJMA5EEATDDOCVFUKRD67NFA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.799519597 (14h17m33.543576447s) [noise=true , address=45.91.116.12:4227 , piece_id=J2FM3FAWY5HYXX3VFC6CSHUSWDQDNKUUB6NL6MWSF2ZG3BZFWBEA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.800753821 (14h17m33.544810667s) [noise=true , address=77.90.31.13:4626 , piece_id=PR6GY2Y7HDUORSLYZXU4IVLVA6RIMBFSRFJW4OTLXUZM7MJZ4KOQ, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.800785983 (14h17m33.544842842s) [seg_pos={0 0}, upload=14716]: Segment reached optimal threshold of 13 pieces. Aug 7 23:57:11.801026052 (14h17m33.545082898s) [noise=true , address=104.36.82.14:4222 , piece_id=JFWGS6VCDTFOQYG32ZFTWNYLFGVVQPXD7K6TZXC4FR2SGEOUS7FA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1): protocol: expected piece hash; context canceled; EOF Aug 7 23:57:11.801098020 (14h17m33.545154867s) [noise=true , address=104.36.82.10:4078 , piece_id=3DPLCXZLQWMZSYSSO3JPLZFP2VXJ5NKYO4FDVFYOWJLT43C7XSTQ, seg_pos={0 0}, upload=14716]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Aug 7 23:57:11.801174944 (14h17m33.54523179s) [noise=true , address=104.36.82.5:4839 , piece_id=AV3ZBQ4W44IZEDAPHJGHKK5D5VRNU5XGOQGDWEARNGIKWRQAGYBQ, seg_pos={0 0}, upload=14716]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 139.59.144.22:48474->104.36.82.5:4839: use of closed network connection; write tcp 139.59.144.22:48474->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48474->104.36.82.5:4839: use of closed network connection Aug 7 23:57:11.801184926 (14h17m33.545241773s) [seg_pos={0 0}, upload=14716]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:57:11.801257096 (14h17m33.545313956s) [upload=14716]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:57:11.801261009 (14h17m33.545317869s) [upload=14716]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:11.942784613 (14h17m33.686841513s) [upload=14716]: Done uploading segments... Aug 7 23:57:11.942815424 (14h17m33.686872279s) [upload=14716]: Upload finished. err: <nil> Aug 7 23:58:20.561350310 (14h18m42.305407184s) [upload=14732]: Starting upload Aug 7 23:58:20.561483402 (14h18m42.30554027s) [upload=14732]: Done starting upload Aug 7 23:58:20.561650297 (14h18m42.305707159s) [upload=14732]: Uploading segments... Aug 7 23:58:20.562196341 (14h18m42.306253212s) [upload=14732]: Got next segment. Inline: false Aug 7 23:58:20.562334166 (14h18m42.306391031s) [upload=14732]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:20.562343255 (14h18m42.306400125s) [upload=14732]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:58:50.281195862 (14h19m12.025252712s) [upload=14732]: Next segment err: context deadline exceeded Aug 7 23:58:50.281383705 (14h19m12.025440555s) [upload=14732]: Done uploading segments... Aug 7 23:58:50.281405600 (14h19m12.02546245s) [upload=14732]: Upload finished. err: context deadline exceeded Aug 7 23:58:50.716777496 (14h19m12.460834346s) [upload=14742]: Starting upload Aug 7 23:58:50.716857905 (14h19m12.460914751s) [upload=14742]: Done starting upload Aug 7 23:58:50.716999972 (14h19m12.461056825s) [upload=14742]: Uploading segments... Aug 7 23:58:50.717186124 (14h19m12.461242976s) [upload=14742]: Got next segment. Inline: false Aug 7 23:58:50.717245601 (14h19m12.461302466s) [upload=14742]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:50.717252185 (14h19m12.461309053s) [upload=14742]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:59:20.436529435 (14h19m42.180586305s) [upload=14742]: Next segment err: context deadline exceeded Aug 7 23:59:20.436734882 (14h19m42.180791733s) [upload=14742]: Done uploading segments... Aug 7 23:59:20.436756200 (14h19m42.180813054s) [upload=14742]: Upload finished. err: context deadline exceeded Duration: 30.006727599s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 3.607329317s FAIL (2025-08-07 23:59:20 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== 294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:32:45.172127356 (13h53m6.916184241s) [upload=14294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:32:45.390149151 (13h53m7.134206011s) [seg_pos={0 0}, upload=14294]: Begin upload segment... Aug 7 23:32:45.390215910 (13h53m7.134272761s) [seg_pos={0 0}, upload=14294]: Done begin upload segment. Aug 7 23:32:45.390249193 (13h53m7.134306059s) [noise=true , address=104.36.82.19:4405 , piece_id=MHM3MWZJFTXSJT7PTOFSWHUYO3L6NIBUV6VU66XXZDH3MC3EFR2Q, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390272919 (13h53m7.134329788s) [noise=true , address=77.90.31.12:4593 , piece_id=C3RVQHFS4L7SKPFFIZ52MAQWGD7P4JNG3SNPCNY53SQUD72PZRUA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390658970 (13h53m7.13471583s) [noise=true , address=104.36.82.16:4321 , piece_id=YTF4RWV2SD6Z4UV4XRYPBRGPCTSUTC6TQKL3DHVKU65ALTNA4TEQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390852777 (13h53m7.134909659s) [noise=true , address=104.36.82.34:4566 , piece_id=3YEAUSXEL7WNZ3O7WM24Z2DKY2TDVKMRZSHMEPMNEL2LR7BY7DBQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390957991 (13h53m7.135014851s) [noise=true , address=104.36.82.47:4523 , piece_id=MJIMHSO66V7OWFFK2UALPKSI7CGGCTHNPZINDXMVIHDLENIW5LFA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.391483291 (13h53m7.135540162s) [noise=true , address=104.36.82.5:4861 , piece_id=GRDGO5LNCETQWRU3M6LACGRW5MNA2KFLXUBT5R2UH6SEGWP6RVRQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.391931094 (13h53m7.135987964s) [noise=true , address=104.36.82.10:4090 , piece_id=V6NPPVIYEH6WLIW7OMT3NDFPR7UDWIT46GW6AFCAWUJTBLC3O55Q, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392250266 (13h53m7.136307136s) [noise=true , address=104.36.82.62:4439 , piece_id=ILRD4TEFW4PSLV4EGPORG3FM6U2HB3WSL5KP3PN566FAOGWODLYQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392611689 (13h53m7.136668559s) [noise=true , address=104.36.82.54:4160 , piece_id=GDBORRYX5QBQGH7ZMMUUAV7AAR5UHZOTK5FTQUPC7YKZC2KD7I3A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392917410 (13h53m7.136974282s) [noise=true , address=104.36.82.23:4477 , piece_id=HAF6424ZCIBGSR4US5LN5ZWE5JMNMLFGUMQQ4YP2ARP7PQXBH5GQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393270536 (13h53m7.137327398s) [noise=true , address=104.36.82.44:4881 , piece_id=XXVSXG3ILFXG4IVMELHL45TRRHDFVVJMJ5LGOHU5EDHG6XXCDH6A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393582635 (13h53m7.137639498s) [noise=true , address=104.36.82.37:4665 , piece_id=RUKKGC4Y3FIVLZO2GRWJN3J6P36VWP5VSC2HCLEU3MZDDFXODJAQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393856926 (13h53m7.137913788s) [noise=true , address=104.36.82.11:4128 , piece_id=XRPULHI7BYDF2JOIASSN5QKGEAC6FJS2F5C22WIY7EKYILRHZ73A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394196364 (13h53m7.138253227s) [noise=true , address=104.36.82.38:4706 , piece_id=BIHIPQX6LRWBNFN4SDC3K3V22AMGL3RC5B4RVE7QXJFKB4XDN3BA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394484397 (13h53m7.138541257s) [noise=true , address=104.36.82.61:4381 , piece_id=TTZOSRP6YPL6P4C6PY5S4K7Z6CIZJDPX7NUNBCK73CWZPCA3HQHA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394593709 (13h53m7.138650578s) [noise=true , address=45.91.116.5:4300 , piece_id=MADJCANN2O35MKXT2PIR5PGMJ3PNTAQAJNNGOK6ISJNVAH2L7YIA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.395093013 (13h53m7.139149879s) [noise=true , address=104.36.82.22:4530 , piece_id=HMUUW5GYH6TRHYXP4YXBHD566FWQRILEHI53AVURA5QTQGG6USFA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.395447945 (13h53m7.139504805s) [noise=true , address=104.36.82.58:4293 , piece_id=I2ETFXYTHYWYJYSHTVOBQTO6ZPGNMA4D3NVKYDE2W34EH4SQE23A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.954469844 (13h53m7.698526694s) [upload=14294]: Next returned nil segment Aug 7 23:32:45.954487953 (13h53m7.69854487s) [upload=14294]: Waiting for error group managing segments... Aug 7 23:32:46.126888627 (13h53m7.870945474s) [noise=true , address=104.36.82.10:4090 , piece_id=V6NPPVIYEH6WLIW7OMT3NDFPR7UDWIT46GW6AFCAWUJTBLC3O55Q, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.132983844 (13h53m7.877040704s) [noise=true , address=104.36.82.16:4321 , piece_id=YTF4RWV2SD6Z4UV4XRYPBRGPCTSUTC6TQKL3DHVKU65ALTNA4TEQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.138277216 (13h53m7.882334067s) [noise=true , address=104.36.82.23:4477 , piece_id=HAF6424ZCIBGSR4US5LN5ZWE5JMNMLFGUMQQ4YP2ARP7PQXBH5GQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.142033326 (13h53m7.886090185s) [noise=true , address=104.36.82.34:4566 , piece_id=3YEAUSXEL7WNZ3O7WM24Z2DKY2TDVKMRZSHMEPMNEL2LR7BY7DBQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.151247206 (13h53m7.895304058s) [noise=true , address=104.36.82.22:4530 , piece_id=HMUUW5GYH6TRHYXP4YXBHD566FWQRILEHI53AVURA5QTQGG6USFA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.151981081 (13h53m7.896037932s) [noise=true , address=104.36.82.58:4293 , piece_id=I2ETFXYTHYWYJYSHTVOBQTO6ZPGNMA4D3NVKYDE2W34EH4SQE23A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.165022882 (13h53m7.909079743s) [noise=true , address=104.36.82.44:4881 , piece_id=XXVSXG3ILFXG4IVMELHL45TRRHDFVVJMJ5LGOHU5EDHG6XXCDH6A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.184418821 (13h53m7.928475667s) [noise=true , address=104.36.82.47:4523 , piece_id=MJIMHSO66V7OWFFK2UALPKSI7CGGCTHNPZINDXMVIHDLENIW5LFA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.184532669 (13h53m7.928589516s) [noise=true , address=45.91.116.5:4300 , piece_id=MADJCANN2O35MKXT2PIR5PGMJ3PNTAQAJNNGOK6ISJNVAH2L7YIA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.193375808 (13h53m7.937432656s) [noise=true , address=104.36.82.54:4160 , piece_id=GDBORRYX5QBQGH7ZMMUUAV7AAR5UHZOTK5FTQUPC7YKZC2KD7I3A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.204247501 (13h53m7.948304372s) [noise=true , address=104.36.82.38:4706 , piece_id=BIHIPQX6LRWBNFN4SDC3K3V22AMGL3RC5B4RVE7QXJFKB4XDN3BA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.235225782 (13h53m7.979282629s) [noise=true , address=104.36.82.62:4439 , piece_id=ILRD4TEFW4PSLV4EGPORG3FM6U2HB3WSL5KP3PN566FAOGWODLYQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.269129867 (13h53m8.013186721s) [noise=true , address=104.36.82.5:4861 , piece_id=GRDGO5LNCETQWRU3M6LACGRW5MNA2KFLXUBT5R2UH6SEGWP6RVRQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.278216912 (13h53m8.022273764s) [noise=true , address=104.36.82.19:4405 , piece_id=MHM3MWZJFTXSJT7PTOFSWHUYO3L6NIBUV6VU66XXZDH3MC3EFR2Q, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.278247847 (13h53m8.022304708s) [seg_pos={0 0}, upload=14294]: Segment reached optimal threshold of 14 pieces. Aug 7 23:32:46.278485612 (13h53m8.022542463s) [noise=true , address=104.36.82.11:4128 , piece_id=XRPULHI7BYDF2JOIASSN5QKGEAC6FJS2F5C22WIY7EKYILRHZ73A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wABkAtAjoYkKyNoAiaVCxSvY6WgvHvAYFPCjpNojtpiYiGtDS): protocol: expected piece hash; context canceled; EOF Aug 7 23:32:46.278511698 (13h53m8.022568566s) [noise=true , address=77.90.31.12:4593 , piece_id=C3RVQHFS4L7SKPFFIZ52MAQWGD7P4JNG3SNPCNY53SQUD72PZRUA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kSxoZb9nmG6oGNytpFTsoiGh3ZPmebVVe8TZxGzANaw7EfdeF): protocol: expected piece hash; context canceled; EOF Aug 7 23:32:46.278642807 (13h53m8.022699657s) [noise=true , address=104.36.82.61:4381 , piece_id=TTZOSRP6YPL6P4C6PY5S4K7Z6CIZJDPX7NUNBCK73CWZPCA3HQHA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zU35MWWaaC1npkP5qGcWBXAkfxGsd5UsdkM2nwX9x7wUGCDbY): protocol: write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection; write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection Aug 7 23:32:46.278650926 (13h53m8.022707786s) [noise=true , address=104.36.82.37:4665 , piece_id=RUKKGC4Y3FIVLZO2GRWJN3J6P36VWP5VSC2HCLEU3MZDDFXODJAQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jP9JrCmoQSdqrzXZxkNN1ihdFkaUMjGZss6Fh19zbL3YyWj5N): protocol: write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection; write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection Aug 7 23:32:46.278674050 (13h53m8.022730911s) [seg_pos={0 0}, upload=14294]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:32:46.278751154 (13h53m8.022808014s) [upload=14294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:32:46.278756168 (13h53m8.022813029s) [upload=14294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:32:46.498660020 (13h53m8.242716882s) [upload=14294]: Done uploading segments... Aug 7 23:32:46.498701263 (13h53m8.242758128s) [upload=14294]: Upload finished. err: <nil> Duration: 3.60738695s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.XX..X......................................................] duration: 2.602321654s FAIL (2025-08-07 23:59:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.602356247s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 7 out of 60 is failed (threshold=3) [..XXX...XX.X..............................................X.] duration: 1.085546364s FAIL (2025-08-07 23:59:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.0855694s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 7 out of 60 is failed (threshold=3) [...XXX...XX.X..............................................X] duration: 30.001109075s FAIL (2025-08-07 23:59:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001133602s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 12 out of 60 is failed (threshold=3) [X.XXXXX.X............................................XXX.X.X] duration: 1m0.000893563s FAIL (2025-08-07 23:59:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000944142s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 998.347565ms FAIL (2025-08-07 23:58:54 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== pload=14121]: Done uploading segments... Aug 7 23:22:28.505845197 (13h42m50.249902044s) [upload=14121]: Upload finished. err: <nil> Aug 7 23:23:10.377483605 (13h43m32.121540458s) [upload=14132]: Starting upload Aug 7 23:23:10.377535635 (13h43m32.121592495s) [upload=14132]: Done starting upload Aug 7 23:23:10.377603797 (13h43m32.121660647s) [upload=14132]: Uploading segments... Aug 7 23:23:10.377638935 (13h43m32.121695804s) [upload=14132]: Got next segment. Inline: true Aug 7 23:23:10.377643594 (13h43m32.121700454s) [upload=14132]: Waiting for error group managing segments... Aug 7 23:23:10.377688252 (13h43m32.121745104s) [upload=14132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:23:10.377693427 (13h43m32.121750286s) [upload=14132]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:23:10.377696805 (13h43m32.121753666s) [upload=14132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:23:10.562260724 (13h43m32.306317573s) [upload=14132]: Done uploading segments... Aug 7 23:23:10.562279228 (13h43m32.306336078s) [upload=14132]: Upload finished. err: <nil> Aug 7 23:24:07.896404858 (13h44m29.640461721s) [upload=14147]: Starting upload Aug 7 23:24:07.896490216 (13h44m29.640547079s) [upload=14147]: Done starting upload Aug 7 23:24:07.896580488 (13h44m29.640637339s) [upload=14147]: Uploading segments... Aug 7 23:24:07.896655006 (13h44m29.640711874s) [upload=14147]: Got next segment. Inline: true Aug 7 23:24:07.896660179 (13h44m29.640717039s) [upload=14147]: Waiting for error group managing segments... Aug 7 23:24:07.896714840 (13h44m29.6407717s) [upload=14147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:24:07.896719399 (13h44m29.64077626s) [upload=14147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:24:07.896722726 (13h44m29.640779586s) [upload=14147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:24:08.048898130 (13h44m29.792954991s) [upload=14147]: Done uploading segments... Aug 7 23:24:08.048952526 (13h44m29.793009386s) [upload=14147]: Upload finished. err: <nil> Aug 7 23:24:53.794754379 (13h45m15.53881125s) [upload=14162]: Starting upload Aug 7 23:24:53.794842317 (13h45m15.538899177s) [upload=14162]: Done starting upload Aug 7 23:24:53.794935964 (13h45m15.538992814s) [upload=14162]: Uploading segments... Aug 7 23:24:53.795003752 (13h45m15.539060623s) [upload=14162]: Got next segment. Inline: true Aug 7 23:24:53.795010814 (13h45m15.539067676s) [upload=14162]: Waiting for error group managing segments... Aug 7 23:24:53.795066607 (13h45m15.539123488s) [upload=14162]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:24:53.795158449 (13h45m15.5392153s) [upload=14162]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:24:53.795163166 (13h45m15.539220026s) [upload=14162]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:24:53.946994982 (13h45m15.691051833s) [upload=14162]: Done uploading segments... Aug 7 23:24:53.947013862 (13h45m15.691070711s) [upload=14162]: Upload finished. err: <nil> Aug 7 23:25:40.029856966 (13h46m1.773913836s) [upload=14177]: Starting upload Aug 7 23:25:40.029986938 (13h46m1.774043809s) [upload=14177]: Done starting upload Aug 7 23:25:40.030076796 (13h46m1.774133642s) [upload=14177]: Uploading segments... Aug 7 23:25:40.030132567 (13h46m1.774189438s) [upload=14177]: Got next segment. Inline: true Aug 7 23:25:40.030139885 (13h46m1.774196757s) [upload=14177]: Waiting for error group managing segments... Aug 7 23:25:40.030197588 (13h46m1.774254461s) [upload=14177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:25:40.030204146 (13h46m1.774261016s) [upload=14177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:25:40.030207984 (13h46m1.77426485s) [upload=14177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:25:40.184198742 (13h46m1.928255592s) [upload=14177]: Done uploading segments... Aug 7 23:25:40.184219017 (13h46m1.928275867s) [upload=14177]: Upload finished. err: <nil> Aug 7 23:26:54.625649110 (13h47m16.369705971s) [upload=14198]: Starting upload Aug 7 23:26:54.625738563 (13h47m16.369795424s) [upload=14198]: Done starting upload Aug 7 23:26:54.625842691 (13h47m16.369899539s) [upload=14198]: Uploading segments... Aug 7 23:26:54.626016839 (13h47m16.37007371s) [upload=14198]: Got next segment. Inline: true Aug 7 23:26:54.626052112 (13h47m16.370108973s) [upload=14198]: Waiting for error group managing segments... Aug 7 23:26:54.626129780 (13h47m16.370186645s) [upload=14198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:26:54.626137392 (13h47m16.370194264s) [upload=14198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:26:54.626143182 (13h47m16.370200048s) [upload=14198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:26:54.775804171 (13h47m16.519861021s) [upload=14198]: Done uploading segments... Aug 7 23:26:54.775824249 (13h47m16.519881102s) [upload=14198]: Upload finished. err: <nil> Aug 7 23:27:32.263746490 (13h47m54.007803361s) [upload=14208]: Starting upload Aug 7 23:27:32.263833655 (13h47m54.007890501s) [upload=14208]: Done starting upload Aug 7 23:27:32.263976536 (13h47m54.008033385s) [upload=14208]: Uploading segments... Aug 7 23:27:32.264070926 (13h47m54.008127796s) [upload=14208]: Got next segment. Inline: true Aug 7 23:27:32.264080356 (13h47m54.00813733s) [upload=14208]: Waiting for error group managing segments... Aug 7 23:27:32.264154657 (13h47m54.008211526s) [upload=14208]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:27:32.264163587 (13h47m54.008220454s) [upload=14208]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:27:32.264172017 (13h47m54.008228914s) [upload=14208]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:27:32.490188559 (13h47m54.23424541s) [upload=14208]: Done uploading segments... Aug 7 23:27:32.490207326 (13h47m54.234264172s) [upload=14208]: Upload finished. err: <nil> Aug 7 23:27:57.927940477 (13h48m19.671997323s) [upload=14217]: Starting upload Aug 7 23:27:57.928016423 (13h48m19.672073283s) [upload=14217]: Done starting upload Aug 7 23:27:57.928101290 (13h48m19.672158142s) [upload=14217]: Uploading segments... Aug 7 23:27:57.928155692 (13h48m19.672212566s) [upload=14217]: Got next segment. Inline: true Aug 7 23:27:57.928162009 (13h48m19.672218869s) [upload=14217]: Waiting for error group managing segments... Aug 7 23:27:57.928213294 (13h48m19.672270145s) [upload=14217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:27:57.928219212 (13h48m19.672276074s) [upload=14217]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:27:57.928223521 (13h48m19.672280383s) [upload=14217]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:27:58.087544005 (13h48m19.831600874s) [upload=14217]: Done uploading segments... Aug 7 23:27:58.087570226 (13h48m19.831627094s) [upload=14217]: Upload finished. err: <nil> Aug 7 23:29:12.765200935 (13h49m34.509257795s) [upload=14237]: Starting upload Aug 7 23:29:12.765253106 (13h49m34.509309977s) [upload=14237]: Done starting upload Aug 7 23:29:12.765324641 (13h49m34.509381499s) [upload=14237]: Uploading segments... Aug 7 23:29:12.765374187 (13h49m34.509431056s) [upload=14237]: Got next segment. Inline: true Aug 7 23:29:12.765378947 (13h49m34.509435807s) [upload=14237]: Waiting for error group managing segments... Aug 7 23:29:12.765430859 (13h49m34.50948772s) [upload=14237]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:29:12.765438106 (13h49m34.50949497s) [upload=14237]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:29:12.765443127 (13h49m34.509499988s) [upload=14237]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:29:12.926437674 (13h49m34.670494535s) [upload=14237]: Done uploading segments... Aug 7 23:29:12.926457418 (13h49m34.670514268s) [upload=14237]: Upload finished. err: <nil> Aug 7 23:29:42.225760316 (13h50m3.969817177s) [upload=14244]: Starting upload Aug 7 23:29:42.225855063 (13h50m3.969911929s) [upload=14244]: Done starting upload Aug 7 23:29:42.225962489 (13h50m3.970019355s) [upload=14244]: Uploading segments... Aug 7 23:29:42.226078484 (13h50m3.970135331s) [upload=14244]: Got next segment. Inline: true Aug 7 23:29:42.226083996 (13h50m3.970140842s) [upload=14244]: Waiting for error group managing segments... Aug 7 23:29:42.226143067 (13h50m3.970199914s) [upload=14244]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:29:42.226148511 (13h50m3.970205371s) [upload=14244]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:29:42.226153006 (13h50m3.970209865s) [upload=14244]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:29:42.401822947 (13h50m4.14587981s) [upload=14244]: Done uploading segments... Aug 7 23:29:42.401847556 (13h50m4.145904405s) [upload=14244]: Upload finished. err: <nil> Aug 7 23:30:31.529840301 (13h50m53.273897165s) [upload=14254]: Starting upload Aug 7 23:30:31.530002531 (13h50m53.274059378s) [upload=14254]: Done starting upload Aug 7 23:30:31.530201544 (13h50m53.274258391s) [upload=14254]: Uploading segments... Aug 7 23:30:31.530298120 (13h50m53.274354985s) [upload=14254]: Got next segment. Inline: true Aug 7 23:30:31.530303457 (13h50m53.274360317s) [upload=14254]: Waiting for error group managing segments... Aug 7 23:30:31.530353721 (13h50m53.274410567s) [upload=14254]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:30:31.530358927 (13h50m53.274415788s) [upload=14254]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:30:31.530363758 (13h50m53.274420619s) [upload=14254]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:30:31.696350967 (13h50m53.440407828s) [upload=14254]: Done uploading segments... Aug 7 23:30:31.696369460 (13h50m53.440426307s) [upload=14254]: Upload finished. err: <nil> Duration: 998.392402ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [XXXXXXXXXX...........X............................XX.X.X...X] duration: 30.001192425s FAIL (2025-08-07 23:58:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== d Aug 7 23:57:10.782817930 (14h17m32.52687478s) [upload=14716]: Done starting upload Aug 7 23:57:10.782927377 (14h17m32.526984224s) [upload=14716]: Uploading segments... Aug 7 23:57:10.783451386 (14h17m32.527508235s) [upload=14716]: Got next segment. Inline: false Aug 7 23:57:10.783532501 (14h17m32.527589366s) [upload=14716]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:57:10.783538852 (14h17m32.527595711s) [upload=14716]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:57:10.893553776 (14h17m32.63761064s) [seg_pos={0 0}, upload=14716]: Begin upload segment... Aug 7 23:57:10.893622767 (14h17m32.637679627s) [seg_pos={0 0}, upload=14716]: Done begin upload segment. Aug 7 23:57:10.893756688 (14h17m32.637813551s) [noise=true , address=45.91.116.13:4153 , piece_id=WBFFQT5I6ZR5ZKUJRFULNOJZ455KIJMA5EEATDDOCVFUKRD67NFA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.893816491 (14h17m32.637873354s) [noise=true , address=45.91.116.12:4225 , piece_id=NTUY2VEJFWSFOAGWUPTJYNYNWL3XMK4C3FJUE6MSTMTS2TZJ7VUA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.894285524 (14h17m32.638342391s) [noise=true , address=104.36.82.16:4294 , piece_id=SIMVCA63FAX5EQZJO5TBOSGPVYLFOAICTABKMXS5SCQGNOUPFBIA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.895348932 (14h17m32.639405787s) [noise=true , address=77.90.31.14:4661 , piece_id=WFDXKM726LBL4MH4CJ4AQIUSXTQCQ3BJV34QBUGWPHNFFPZYGZKA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.895932061 (14h17m32.639988921s) [noise=true , address=104.36.82.10:4078 , piece_id=3DPLCXZLQWMZSYSSO3JPLZFP2VXJ5NKYO4FDVFYOWJLT43C7XSTQ, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.896545320 (14h17m32.640602184s) [noise=true , address=77.90.31.10:4518 , piece_id=LU6UTDDO6HUIGXV2QXCL45CIBFQMOQGS7CTQHBRKCBE2JJKLI6FA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.897117472 (14h17m32.641174341s) [noise=true , address=77.90.31.21:4877 , piece_id=SF7XDMNH7KGS26UE6MKUK3VLFZ6IOP6WZKMVO4H6SIAULM623GLA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.897621325 (14h17m32.641678192s) [noise=true , address=77.90.31.12:4589 , piece_id=5I5BMAQARX67CB2RRMCXG2U7KELJ6HSHRT3573EH3ZX6NJ257R4A, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.898101338 (14h17m32.642158204s) [noise=true , address=77.90.31.21:4878 , piece_id=XAXEYDOX54GUKMI22UKTA6BSTSY5WYATTITTGSRQ3BBYDGDCO5DQ, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.898693607 (14h17m32.642750474s) [noise=true , address=45.91.116.6:4975 , piece_id=VDEP57UE6G4CBU7DOYMMOIRXJUTTWPJKDJDGCGUOHBGVPAYCM6PA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.900512475 (14h17m32.644569335s) [noise=true , address=45.91.116.12:4227 , piece_id=J2FM3FAWY5HYXX3VFC6CSHUSWDQDNKUUB6NL6MWSF2ZG3BZFWBEA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.902337804 (14h17m32.646394669s) [noise=true , address=77.90.31.13:4626 , piece_id=PR6GY2Y7HDUORSLYZXU4IVLVA6RIMBFSRFJW4OTLXUZM7MJZ4KOQ, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.904215667 (14h17m32.648272537s) [noise=true , address=77.90.31.19:4841 , piece_id=Q3XFTIVGOSRQARSFONAKX26VAFTJF7THPIQV6USKI7N2LN3JBTLQ, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.906034481 (14h17m32.650091365s) [noise=true , address=104.36.82.5:4839 , piece_id=AV3ZBQ4W44IZEDAPHJGHKK5D5VRNU5XGOQGDWEARNGIKWRQAGYBQ, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.907857426 (14h17m32.651914295s) [noise=true , address=77.90.31.15:4698 , piece_id=WBU562MADMX3KSCRCD2H5R7MQN5IPFBLKYSKU47WMGYYBKZGPYJA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:10.908666188 (14h17m32.652723069s) [noise=true , address=104.36.82.14:4222 , piece_id=JFWGS6VCDTFOQYG32ZFTWNYLFGVVQPXD7K6TZXC4FR2SGEOUS7FA, seg_pos={0 0}, upload=14716]: Uploading piece... Aug 7 23:57:11.510238622 (14h17m33.254295484s) [upload=14716]: Next returned nil segment Aug 7 23:57:11.510261464 (14h17m33.254318324s) [upload=14716]: Waiting for error group managing segments... Aug 7 23:57:11.657094388 (14h17m33.401151236s) [noise=true , address=77.90.31.10:4518 , piece_id=LU6UTDDO6HUIGXV2QXCL45CIBFQMOQGS7CTQHBRKCBE2JJKLI6FA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.674408374 (14h17m33.418465234s) [noise=true , address=77.90.31.15:4698 , piece_id=WBU562MADMX3KSCRCD2H5R7MQN5IPFBLKYSKU47WMGYYBKZGPYJA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.681203586 (14h17m33.425260449s) [noise=true , address=77.90.31.12:4589 , piece_id=5I5BMAQARX67CB2RRMCXG2U7KELJ6HSHRT3573EH3ZX6NJ257R4A, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.682780850 (14h17m33.4268377s) [noise=true , address=77.90.31.21:4877 , piece_id=SF7XDMNH7KGS26UE6MKUK3VLFZ6IOP6WZKMVO4H6SIAULM623GLA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.684042315 (14h17m33.428099176s) [noise=true , address=104.36.82.16:4294 , piece_id=SIMVCA63FAX5EQZJO5TBOSGPVYLFOAICTABKMXS5SCQGNOUPFBIA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.719658547 (14h17m33.463715401s) [noise=true , address=45.91.116.12:4225 , piece_id=NTUY2VEJFWSFOAGWUPTJYNYNWL3XMK4C3FJUE6MSTMTS2TZJ7VUA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.730348391 (14h17m33.474405242s) [noise=true , address=77.90.31.21:4878 , piece_id=XAXEYDOX54GUKMI22UKTA6BSTSY5WYATTITTGSRQ3BBYDGDCO5DQ, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.748264399 (14h17m33.492321249s) [noise=true , address=77.90.31.19:4841 , piece_id=Q3XFTIVGOSRQARSFONAKX26VAFTJF7THPIQV6USKI7N2LN3JBTLQ, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.769764028 (14h17m33.513820878s) [noise=true , address=45.91.116.6:4975 , piece_id=VDEP57UE6G4CBU7DOYMMOIRXJUTTWPJKDJDGCGUOHBGVPAYCM6PA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.774177874 (14h17m33.518234733s) [noise=true , address=77.90.31.14:4661 , piece_id=WFDXKM726LBL4MH4CJ4AQIUSXTQCQ3BJV34QBUGWPHNFFPZYGZKA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.789864472 (14h17m33.533921326s) [noise=true , address=45.91.116.13:4153 , piece_id=WBFFQT5I6ZR5ZKUJRFULNOJZ455KIJMA5EEATDDOCVFUKRD67NFA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.799519597 (14h17m33.543576447s) [noise=true , address=45.91.116.12:4227 , piece_id=J2FM3FAWY5HYXX3VFC6CSHUSWDQDNKUUB6NL6MWSF2ZG3BZFWBEA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.800753821 (14h17m33.544810667s) [noise=true , address=77.90.31.13:4626 , piece_id=PR6GY2Y7HDUORSLYZXU4IVLVA6RIMBFSRFJW4OTLXUZM7MJZ4KOQ, seg_pos={0 0}, upload=14716]: Done uploading piece. err: <nil> Aug 7 23:57:11.800785983 (14h17m33.544842842s) [seg_pos={0 0}, upload=14716]: Segment reached optimal threshold of 13 pieces. Aug 7 23:57:11.801026052 (14h17m33.545082898s) [noise=true , address=104.36.82.14:4222 , piece_id=JFWGS6VCDTFOQYG32ZFTWNYLFGVVQPXD7K6TZXC4FR2SGEOUS7FA, seg_pos={0 0}, upload=14716]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1): protocol: expected piece hash; context canceled; EOF Aug 7 23:57:11.801098020 (14h17m33.545154867s) [noise=true , address=104.36.82.10:4078 , piece_id=3DPLCXZLQWMZSYSSO3JPLZFP2VXJ5NKYO4FDVFYOWJLT43C7XSTQ, seg_pos={0 0}, upload=14716]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Aug 7 23:57:11.801174944 (14h17m33.54523179s) [noise=true , address=104.36.82.5:4839 , piece_id=AV3ZBQ4W44IZEDAPHJGHKK5D5VRNU5XGOQGDWEARNGIKWRQAGYBQ, seg_pos={0 0}, upload=14716]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 139.59.144.22:48474->104.36.82.5:4839: use of closed network connection; write tcp 139.59.144.22:48474->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48474->104.36.82.5:4839: use of closed network connection Aug 7 23:57:11.801184926 (14h17m33.545241773s) [seg_pos={0 0}, upload=14716]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:57:11.801257096 (14h17m33.545313956s) [upload=14716]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:57:11.801261009 (14h17m33.545317869s) [upload=14716]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:11.942784613 (14h17m33.686841513s) [upload=14716]: Done uploading segments... Aug 7 23:57:11.942815424 (14h17m33.686872279s) [upload=14716]: Upload finished. err: <nil> Aug 7 23:58:20.561350310 (14h18m42.305407184s) [upload=14732]: Starting upload Aug 7 23:58:20.561483402 (14h18m42.30554027s) [upload=14732]: Done starting upload Aug 7 23:58:20.561650297 (14h18m42.305707159s) [upload=14732]: Uploading segments... Aug 7 23:58:20.562196341 (14h18m42.306253212s) [upload=14732]: Got next segment. Inline: false Aug 7 23:58:20.562334166 (14h18m42.306391031s) [upload=14732]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:20.562343255 (14h18m42.306400125s) [upload=14732]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:58:50.281195862 (14h19m12.025252712s) [upload=14732]: Next segment err: context deadline exceeded Aug 7 23:58:50.281383705 (14h19m12.025440555s) [upload=14732]: Done uploading segments... Aug 7 23:58:50.281405600 (14h19m12.02546245s) [upload=14732]: Upload finished. err: context deadline exceeded Duration: 30.001215092s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 9 out of 60 is failed (threshold=3) [XXXXX................................................XX..XX.] duration: 831.258709ms FAIL (2025-08-07 23:58:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== > Aug 7 23:50:24.097841430 (14h10m45.841898277s) [upload=14602]: Starting upload Aug 7 23:50:24.097929459 (14h10m45.841986306s) [upload=14602]: Done starting upload Aug 7 23:50:24.097991265 (14h10m45.842048125s) [upload=14602]: Uploading segments... Aug 7 23:50:24.098029893 (14h10m45.842086762s) [upload=14602]: Got next segment. Inline: true Aug 7 23:50:24.098034701 (14h10m45.84209156s) [upload=14602]: Waiting for error group managing segments... Aug 7 23:50:24.098071586 (14h10m45.842128432s) [upload=14602]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:24.098076409 (14h10m45.842133268s) [upload=14602]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:24.098079284 (14h10m45.842136144s) [upload=14602]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:24.238728801 (14h10m45.982785662s) [upload=14602]: Done uploading segments... Aug 7 23:50:24.238753595 (14h10m45.982810442s) [upload=14602]: Upload finished. err: <nil> Aug 7 23:51:03.108197922 (14h11m24.852254772s) [upload=14612]: Starting upload Aug 7 23:51:03.108300837 (14h11m24.852357707s) [upload=14612]: Done starting upload Aug 7 23:51:03.108453294 (14h11m24.852510146s) [upload=14612]: Uploading segments... Aug 7 23:51:03.108972593 (14h11m24.853029491s) [upload=14612]: Got next segment. Inline: true Aug 7 23:51:03.108988857 (14h11m24.85304573s) [upload=14612]: Waiting for error group managing segments... Aug 7 23:51:03.109073931 (14h11m24.853130786s) [upload=14612]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:51:03.109080082 (14h11m24.853136933s) [upload=14612]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:51:03.109083894 (14h11m24.853140746s) [upload=14612]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:03.446817008 (14h11m25.190873858s) [upload=14612]: Done uploading segments... Aug 7 23:51:03.446842582 (14h11m25.190899436s) [upload=14612]: Upload finished. err: <nil> Aug 7 23:52:09.742356648 (14h12m31.486413495s) [upload=14628]: Starting upload Aug 7 23:52:09.742413553 (14h12m31.486470415s) [upload=14628]: Done starting upload Aug 7 23:52:09.742476702 (14h12m31.486533548s) [upload=14628]: Uploading segments... Aug 7 23:52:09.742511425 (14h12m31.486568294s) [upload=14628]: Got next segment. Inline: true Aug 7 23:52:09.742515305 (14h12m31.486572152s) [upload=14628]: Waiting for error group managing segments... Aug 7 23:52:09.742550700 (14h12m31.486607545s) [upload=14628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:09.742555041 (14h12m31.486611889s) [upload=14628]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:09.742557771 (14h12m31.486614632s) [upload=14628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:09.895389395 (14h12m31.639446255s) [upload=14628]: Done uploading segments... Aug 7 23:52:09.895421188 (14h12m31.639478039s) [upload=14628]: Upload finished. err: <nil> Aug 7 23:52:28.792030311 (14h12m50.536087163s) [upload=14635]: Starting upload Aug 7 23:52:28.792092644 (14h12m50.536149508s) [upload=14635]: Done starting upload Aug 7 23:52:28.792152087 (14h12m50.53620894s) [upload=14635]: Uploading segments... Aug 7 23:52:28.792190462 (14h12m50.536247332s) [upload=14635]: Got next segment. Inline: true Aug 7 23:52:28.792195687 (14h12m50.536252548s) [upload=14635]: Waiting for error group managing segments... Aug 7 23:52:28.792239958 (14h12m50.536296808s) [upload=14635]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:28.792245830 (14h12m50.536302691s) [upload=14635]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:28.792249165 (14h12m50.536306026s) [upload=14635]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:28.947698658 (14h12m50.691755509s) [upload=14635]: Done uploading segments... Aug 7 23:52:28.947717721 (14h12m50.691774571s) [upload=14635]: Upload finished. err: <nil> Aug 7 23:53:17.152926358 (14h13m38.896983229s) [upload=14650]: Starting upload Aug 7 23:53:17.153036870 (14h13m38.89709373s) [upload=14650]: Done starting upload Aug 7 23:53:17.153154678 (14h13m38.897211544s) [upload=14650]: Uploading segments... Aug 7 23:53:17.153233089 (14h13m38.897289957s) [upload=14650]: Got next segment. Inline: true Aug 7 23:53:17.153240256 (14h13m38.897297124s) [upload=14650]: Waiting for error group managing segments... Aug 7 23:53:17.153318892 (14h13m38.897375757s) [upload=14650]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:17.153325816 (14h13m38.89738268s) [upload=14650]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:17.153330453 (14h13m38.897387314s) [upload=14650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:16.875757982 (14h14m38.619814845s) [upload=14650]: Done uploading segments... Aug 7 23:54:16.875781849 (14h14m38.619838701s) [upload=14650]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:24.158448119 (14h14m45.90250498s) [upload=14667]: Starting upload Aug 7 23:54:24.158525946 (14h14m45.902582806s) [upload=14667]: Done starting upload Aug 7 23:54:24.158598972 (14h14m45.902655824s) [upload=14667]: Uploading segments... Aug 7 23:54:24.158654148 (14h14m45.902711017s) [upload=14667]: Got next segment. Inline: true Aug 7 23:54:24.158659995 (14h14m45.902716856s) [upload=14667]: Waiting for error group managing segments... Aug 7 23:54:24.158714896 (14h14m45.902771747s) [upload=14667]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:24.158720357 (14h14m45.902777209s) [upload=14667]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:24.158724105 (14h14m45.902780965s) [upload=14667]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:23.888400190 (14h15m45.63245704s) [upload=14667]: Done uploading segments... Aug 7 23:55:23.888419087 (14h15m45.632475938s) [upload=14667]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:24.297146309 (14h15m46.041203174s) [upload=14687]: Starting upload Aug 7 23:55:24.297232144 (14h15m46.04128901s) [upload=14687]: Done starting upload Aug 7 23:55:24.297315772 (14h15m46.041372619s) [upload=14687]: Uploading segments... Aug 7 23:55:24.297396238 (14h15m46.041453151s) [upload=14687]: Got next segment. Inline: true Aug 7 23:55:24.297403729 (14h15m46.04146059s) [upload=14687]: Waiting for error group managing segments... Aug 7 23:55:24.297444116 (14h15m46.041500976s) [upload=14687]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:24.297450971 (14h15m46.041507831s) [upload=14687]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:24.297453883 (14h15m46.041510749s) [upload=14687]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:24.439126204 (14h15m46.183183055s) [upload=14687]: Done uploading segments... Aug 7 23:55:24.439141121 (14h15m46.183197972s) [upload=14687]: Upload finished. err: <nil> Aug 7 23:56:03.068174127 (14h16m24.812230977s) [upload=14695]: Starting upload Aug 7 23:56:03.068258755 (14h16m24.812315617s) [upload=14695]: Done starting upload Aug 7 23:56:03.068368961 (14h16m24.812425812s) [upload=14695]: Uploading segments... Aug 7 23:56:03.068437851 (14h16m24.812494719s) [upload=14695]: Got next segment. Inline: true Aug 7 23:56:03.068445058 (14h16m24.812501927s) [upload=14695]: Waiting for error group managing segments... Aug 7 23:56:03.068516175 (14h16m24.812573049s) [upload=14695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:03.068523275 (14h16m24.812580137s) [upload=14695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:03.068527513 (14h16m24.812584488s) [upload=14695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:03.370378590 (14h16m25.114435452s) [upload=14695]: Done uploading segments... Aug 7 23:56:03.370401742 (14h16m25.114458656s) [upload=14695]: Upload finished. err: <nil> Aug 7 23:56:41.636734874 (14h17m3.380791742s) [upload=14707]: Starting upload Aug 7 23:56:41.636844337 (14h17m3.380901201s) [upload=14707]: Done starting upload Aug 7 23:56:41.636938361 (14h17m3.380995223s) [upload=14707]: Uploading segments... Aug 7 23:56:41.637004566 (14h17m3.381061436s) [upload=14707]: Got next segment. Inline: true Aug 7 23:56:41.637011720 (14h17m3.381068581s) [upload=14707]: Waiting for error group managing segments... Aug 7 23:56:41.637073380 (14h17m3.381130242s) [upload=14707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:41.637079684 (14h17m3.381136546s) [upload=14707]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:41.637083892 (14h17m3.381140753s) [upload=14707]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:41.369074470 (14h18m3.113131318s) [upload=14707]: Done uploading segments... Aug 7 23:57:41.369096313 (14h18m3.11315316s) [upload=14707]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:57:41.785857243 (14h18m3.529914104s) [upload=14726]: Starting upload Aug 7 23:57:41.785973295 (14h18m3.530030155s) [upload=14726]: Done starting upload Aug 7 23:57:41.786053418 (14h18m3.530110268s) [upload=14726]: Uploading segments... Aug 7 23:57:41.786115407 (14h18m3.530172273s) [upload=14726]: Got next segment. Inline: true Aug 7 23:57:41.786121268 (14h18m3.530178127s) [upload=14726]: Waiting for error group managing segments... Aug 7 23:57:41.786162041 (14h18m3.530218901s) [upload=14726]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:57:41.786167247 (14h18m3.530224108s) [upload=14726]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:57:41.786170583 (14h18m3.530227443s) [upload=14726]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:41.518215592 (14h19m3.262272452s) [upload=14726]: Done uploading segments... Aug 7 23:58:41.518237426 (14h19m3.262294275s) [upload=14726]: Upload finished. err: metaclient: context deadline exceeded Duration: 831.295026ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [X.XX..X.....................................................] duration: 2.799881985s FAIL (2025-08-07 23:58:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.799909421s edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 326.029159ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 23:58:43 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 326.060327ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 9 out of 60 is failed (threshold=3) [.XXXXX................................................XX..XX] duration: 1m0.053087674s FAIL (2025-08-07 23:58:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== > Aug 7 23:50:24.097841430 (14h10m45.841898277s) [upload=14602]: Starting upload Aug 7 23:50:24.097929459 (14h10m45.841986306s) [upload=14602]: Done starting upload Aug 7 23:50:24.097991265 (14h10m45.842048125s) [upload=14602]: Uploading segments... Aug 7 23:50:24.098029893 (14h10m45.842086762s) [upload=14602]: Got next segment. Inline: true Aug 7 23:50:24.098034701 (14h10m45.84209156s) [upload=14602]: Waiting for error group managing segments... Aug 7 23:50:24.098071586 (14h10m45.842128432s) [upload=14602]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:24.098076409 (14h10m45.842133268s) [upload=14602]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:24.098079284 (14h10m45.842136144s) [upload=14602]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:24.238728801 (14h10m45.982785662s) [upload=14602]: Done uploading segments... Aug 7 23:50:24.238753595 (14h10m45.982810442s) [upload=14602]: Upload finished. err: <nil> Aug 7 23:51:03.108197922 (14h11m24.852254772s) [upload=14612]: Starting upload Aug 7 23:51:03.108300837 (14h11m24.852357707s) [upload=14612]: Done starting upload Aug 7 23:51:03.108453294 (14h11m24.852510146s) [upload=14612]: Uploading segments... Aug 7 23:51:03.108972593 (14h11m24.853029491s) [upload=14612]: Got next segment. Inline: true Aug 7 23:51:03.108988857 (14h11m24.85304573s) [upload=14612]: Waiting for error group managing segments... Aug 7 23:51:03.109073931 (14h11m24.853130786s) [upload=14612]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:51:03.109080082 (14h11m24.853136933s) [upload=14612]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:51:03.109083894 (14h11m24.853140746s) [upload=14612]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:03.446817008 (14h11m25.190873858s) [upload=14612]: Done uploading segments... Aug 7 23:51:03.446842582 (14h11m25.190899436s) [upload=14612]: Upload finished. err: <nil> Aug 7 23:52:09.742356648 (14h12m31.486413495s) [upload=14628]: Starting upload Aug 7 23:52:09.742413553 (14h12m31.486470415s) [upload=14628]: Done starting upload Aug 7 23:52:09.742476702 (14h12m31.486533548s) [upload=14628]: Uploading segments... Aug 7 23:52:09.742511425 (14h12m31.486568294s) [upload=14628]: Got next segment. Inline: true Aug 7 23:52:09.742515305 (14h12m31.486572152s) [upload=14628]: Waiting for error group managing segments... Aug 7 23:52:09.742550700 (14h12m31.486607545s) [upload=14628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:09.742555041 (14h12m31.486611889s) [upload=14628]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:09.742557771 (14h12m31.486614632s) [upload=14628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:09.895389395 (14h12m31.639446255s) [upload=14628]: Done uploading segments... Aug 7 23:52:09.895421188 (14h12m31.639478039s) [upload=14628]: Upload finished. err: <nil> Aug 7 23:52:28.792030311 (14h12m50.536087163s) [upload=14635]: Starting upload Aug 7 23:52:28.792092644 (14h12m50.536149508s) [upload=14635]: Done starting upload Aug 7 23:52:28.792152087 (14h12m50.53620894s) [upload=14635]: Uploading segments... Aug 7 23:52:28.792190462 (14h12m50.536247332s) [upload=14635]: Got next segment. Inline: true Aug 7 23:52:28.792195687 (14h12m50.536252548s) [upload=14635]: Waiting for error group managing segments... Aug 7 23:52:28.792239958 (14h12m50.536296808s) [upload=14635]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:28.792245830 (14h12m50.536302691s) [upload=14635]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:28.792249165 (14h12m50.536306026s) [upload=14635]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:28.947698658 (14h12m50.691755509s) [upload=14635]: Done uploading segments... Aug 7 23:52:28.947717721 (14h12m50.691774571s) [upload=14635]: Upload finished. err: <nil> Aug 7 23:53:17.152926358 (14h13m38.896983229s) [upload=14650]: Starting upload Aug 7 23:53:17.153036870 (14h13m38.89709373s) [upload=14650]: Done starting upload Aug 7 23:53:17.153154678 (14h13m38.897211544s) [upload=14650]: Uploading segments... Aug 7 23:53:17.153233089 (14h13m38.897289957s) [upload=14650]: Got next segment. Inline: true Aug 7 23:53:17.153240256 (14h13m38.897297124s) [upload=14650]: Waiting for error group managing segments... Aug 7 23:53:17.153318892 (14h13m38.897375757s) [upload=14650]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:17.153325816 (14h13m38.89738268s) [upload=14650]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:17.153330453 (14h13m38.897387314s) [upload=14650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:16.875757982 (14h14m38.619814845s) [upload=14650]: Done uploading segments... Aug 7 23:54:16.875781849 (14h14m38.619838701s) [upload=14650]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:24.158448119 (14h14m45.90250498s) [upload=14667]: Starting upload Aug 7 23:54:24.158525946 (14h14m45.902582806s) [upload=14667]: Done starting upload Aug 7 23:54:24.158598972 (14h14m45.902655824s) [upload=14667]: Uploading segments... Aug 7 23:54:24.158654148 (14h14m45.902711017s) [upload=14667]: Got next segment. Inline: true Aug 7 23:54:24.158659995 (14h14m45.902716856s) [upload=14667]: Waiting for error group managing segments... Aug 7 23:54:24.158714896 (14h14m45.902771747s) [upload=14667]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:24.158720357 (14h14m45.902777209s) [upload=14667]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:24.158724105 (14h14m45.902780965s) [upload=14667]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:23.888400190 (14h15m45.63245704s) [upload=14667]: Done uploading segments... Aug 7 23:55:23.888419087 (14h15m45.632475938s) [upload=14667]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:24.297146309 (14h15m46.041203174s) [upload=14687]: Starting upload Aug 7 23:55:24.297232144 (14h15m46.04128901s) [upload=14687]: Done starting upload Aug 7 23:55:24.297315772 (14h15m46.041372619s) [upload=14687]: Uploading segments... Aug 7 23:55:24.297396238 (14h15m46.041453151s) [upload=14687]: Got next segment. Inline: true Aug 7 23:55:24.297403729 (14h15m46.04146059s) [upload=14687]: Waiting for error group managing segments... Aug 7 23:55:24.297444116 (14h15m46.041500976s) [upload=14687]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:24.297450971 (14h15m46.041507831s) [upload=14687]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:24.297453883 (14h15m46.041510749s) [upload=14687]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:24.439126204 (14h15m46.183183055s) [upload=14687]: Done uploading segments... Aug 7 23:55:24.439141121 (14h15m46.183197972s) [upload=14687]: Upload finished. err: <nil> Aug 7 23:56:03.068174127 (14h16m24.812230977s) [upload=14695]: Starting upload Aug 7 23:56:03.068258755 (14h16m24.812315617s) [upload=14695]: Done starting upload Aug 7 23:56:03.068368961 (14h16m24.812425812s) [upload=14695]: Uploading segments... Aug 7 23:56:03.068437851 (14h16m24.812494719s) [upload=14695]: Got next segment. Inline: true Aug 7 23:56:03.068445058 (14h16m24.812501927s) [upload=14695]: Waiting for error group managing segments... Aug 7 23:56:03.068516175 (14h16m24.812573049s) [upload=14695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:03.068523275 (14h16m24.812580137s) [upload=14695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:03.068527513 (14h16m24.812584488s) [upload=14695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:03.370378590 (14h16m25.114435452s) [upload=14695]: Done uploading segments... Aug 7 23:56:03.370401742 (14h16m25.114458656s) [upload=14695]: Upload finished. err: <nil> Aug 7 23:56:41.636734874 (14h17m3.380791742s) [upload=14707]: Starting upload Aug 7 23:56:41.636844337 (14h17m3.380901201s) [upload=14707]: Done starting upload Aug 7 23:56:41.636938361 (14h17m3.380995223s) [upload=14707]: Uploading segments... Aug 7 23:56:41.637004566 (14h17m3.381061436s) [upload=14707]: Got next segment. Inline: true Aug 7 23:56:41.637011720 (14h17m3.381068581s) [upload=14707]: Waiting for error group managing segments... Aug 7 23:56:41.637073380 (14h17m3.381130242s) [upload=14707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:41.637079684 (14h17m3.381136546s) [upload=14707]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:41.637083892 (14h17m3.381140753s) [upload=14707]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:41.369074470 (14h18m3.113131318s) [upload=14707]: Done uploading segments... Aug 7 23:57:41.369096313 (14h18m3.11315316s) [upload=14707]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:57:41.785857243 (14h18m3.529914104s) [upload=14726]: Starting upload Aug 7 23:57:41.785973295 (14h18m3.530030155s) [upload=14726]: Done starting upload Aug 7 23:57:41.786053418 (14h18m3.530110268s) [upload=14726]: Uploading segments... Aug 7 23:57:41.786115407 (14h18m3.530172273s) [upload=14726]: Got next segment. Inline: true Aug 7 23:57:41.786121268 (14h18m3.530178127s) [upload=14726]: Waiting for error group managing segments... Aug 7 23:57:41.786162041 (14h18m3.530218901s) [upload=14726]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:57:41.786167247 (14h18m3.530224108s) [upload=14726]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:57:41.786170583 (14h18m3.530227443s) [upload=14726]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:41.518215592 (14h19m3.262272452s) [upload=14726]: Done uploading segments... Aug 7 23:58:41.518237426 (14h19m3.262294275s) [upload=14726]: Upload finished. err: metaclient: context deadline exceeded Duration: 1m0.053109367s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 3.010762691s FAIL (2025-08-07 23:58:41 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== 294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:32:45.172127356 (13h53m6.916184241s) [upload=14294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:32:45.390149151 (13h53m7.134206011s) [seg_pos={0 0}, upload=14294]: Begin upload segment... Aug 7 23:32:45.390215910 (13h53m7.134272761s) [seg_pos={0 0}, upload=14294]: Done begin upload segment. Aug 7 23:32:45.390249193 (13h53m7.134306059s) [noise=true , address=104.36.82.19:4405 , piece_id=MHM3MWZJFTXSJT7PTOFSWHUYO3L6NIBUV6VU66XXZDH3MC3EFR2Q, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390272919 (13h53m7.134329788s) [noise=true , address=77.90.31.12:4593 , piece_id=C3RVQHFS4L7SKPFFIZ52MAQWGD7P4JNG3SNPCNY53SQUD72PZRUA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390658970 (13h53m7.13471583s) [noise=true , address=104.36.82.16:4321 , piece_id=YTF4RWV2SD6Z4UV4XRYPBRGPCTSUTC6TQKL3DHVKU65ALTNA4TEQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390852777 (13h53m7.134909659s) [noise=true , address=104.36.82.34:4566 , piece_id=3YEAUSXEL7WNZ3O7WM24Z2DKY2TDVKMRZSHMEPMNEL2LR7BY7DBQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390957991 (13h53m7.135014851s) [noise=true , address=104.36.82.47:4523 , piece_id=MJIMHSO66V7OWFFK2UALPKSI7CGGCTHNPZINDXMVIHDLENIW5LFA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.391483291 (13h53m7.135540162s) [noise=true , address=104.36.82.5:4861 , piece_id=GRDGO5LNCETQWRU3M6LACGRW5MNA2KFLXUBT5R2UH6SEGWP6RVRQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.391931094 (13h53m7.135987964s) [noise=true , address=104.36.82.10:4090 , piece_id=V6NPPVIYEH6WLIW7OMT3NDFPR7UDWIT46GW6AFCAWUJTBLC3O55Q, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392250266 (13h53m7.136307136s) [noise=true , address=104.36.82.62:4439 , piece_id=ILRD4TEFW4PSLV4EGPORG3FM6U2HB3WSL5KP3PN566FAOGWODLYQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392611689 (13h53m7.136668559s) [noise=true , address=104.36.82.54:4160 , piece_id=GDBORRYX5QBQGH7ZMMUUAV7AAR5UHZOTK5FTQUPC7YKZC2KD7I3A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392917410 (13h53m7.136974282s) [noise=true , address=104.36.82.23:4477 , piece_id=HAF6424ZCIBGSR4US5LN5ZWE5JMNMLFGUMQQ4YP2ARP7PQXBH5GQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393270536 (13h53m7.137327398s) [noise=true , address=104.36.82.44:4881 , piece_id=XXVSXG3ILFXG4IVMELHL45TRRHDFVVJMJ5LGOHU5EDHG6XXCDH6A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393582635 (13h53m7.137639498s) [noise=true , address=104.36.82.37:4665 , piece_id=RUKKGC4Y3FIVLZO2GRWJN3J6P36VWP5VSC2HCLEU3MZDDFXODJAQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393856926 (13h53m7.137913788s) [noise=true , address=104.36.82.11:4128 , piece_id=XRPULHI7BYDF2JOIASSN5QKGEAC6FJS2F5C22WIY7EKYILRHZ73A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394196364 (13h53m7.138253227s) [noise=true , address=104.36.82.38:4706 , piece_id=BIHIPQX6LRWBNFN4SDC3K3V22AMGL3RC5B4RVE7QXJFKB4XDN3BA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394484397 (13h53m7.138541257s) [noise=true , address=104.36.82.61:4381 , piece_id=TTZOSRP6YPL6P4C6PY5S4K7Z6CIZJDPX7NUNBCK73CWZPCA3HQHA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394593709 (13h53m7.138650578s) [noise=true , address=45.91.116.5:4300 , piece_id=MADJCANN2O35MKXT2PIR5PGMJ3PNTAQAJNNGOK6ISJNVAH2L7YIA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.395093013 (13h53m7.139149879s) [noise=true , address=104.36.82.22:4530 , piece_id=HMUUW5GYH6TRHYXP4YXBHD566FWQRILEHI53AVURA5QTQGG6USFA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.395447945 (13h53m7.139504805s) [noise=true , address=104.36.82.58:4293 , piece_id=I2ETFXYTHYWYJYSHTVOBQTO6ZPGNMA4D3NVKYDE2W34EH4SQE23A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.954469844 (13h53m7.698526694s) [upload=14294]: Next returned nil segment Aug 7 23:32:45.954487953 (13h53m7.69854487s) [upload=14294]: Waiting for error group managing segments... Aug 7 23:32:46.126888627 (13h53m7.870945474s) [noise=true , address=104.36.82.10:4090 , piece_id=V6NPPVIYEH6WLIW7OMT3NDFPR7UDWIT46GW6AFCAWUJTBLC3O55Q, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.132983844 (13h53m7.877040704s) [noise=true , address=104.36.82.16:4321 , piece_id=YTF4RWV2SD6Z4UV4XRYPBRGPCTSUTC6TQKL3DHVKU65ALTNA4TEQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.138277216 (13h53m7.882334067s) [noise=true , address=104.36.82.23:4477 , piece_id=HAF6424ZCIBGSR4US5LN5ZWE5JMNMLFGUMQQ4YP2ARP7PQXBH5GQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.142033326 (13h53m7.886090185s) [noise=true , address=104.36.82.34:4566 , piece_id=3YEAUSXEL7WNZ3O7WM24Z2DKY2TDVKMRZSHMEPMNEL2LR7BY7DBQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.151247206 (13h53m7.895304058s) [noise=true , address=104.36.82.22:4530 , piece_id=HMUUW5GYH6TRHYXP4YXBHD566FWQRILEHI53AVURA5QTQGG6USFA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.151981081 (13h53m7.896037932s) [noise=true , address=104.36.82.58:4293 , piece_id=I2ETFXYTHYWYJYSHTVOBQTO6ZPGNMA4D3NVKYDE2W34EH4SQE23A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.165022882 (13h53m7.909079743s) [noise=true , address=104.36.82.44:4881 , piece_id=XXVSXG3ILFXG4IVMELHL45TRRHDFVVJMJ5LGOHU5EDHG6XXCDH6A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.184418821 (13h53m7.928475667s) [noise=true , address=104.36.82.47:4523 , piece_id=MJIMHSO66V7OWFFK2UALPKSI7CGGCTHNPZINDXMVIHDLENIW5LFA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.184532669 (13h53m7.928589516s) [noise=true , address=45.91.116.5:4300 , piece_id=MADJCANN2O35MKXT2PIR5PGMJ3PNTAQAJNNGOK6ISJNVAH2L7YIA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.193375808 (13h53m7.937432656s) [noise=true , address=104.36.82.54:4160 , piece_id=GDBORRYX5QBQGH7ZMMUUAV7AAR5UHZOTK5FTQUPC7YKZC2KD7I3A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.204247501 (13h53m7.948304372s) [noise=true , address=104.36.82.38:4706 , piece_id=BIHIPQX6LRWBNFN4SDC3K3V22AMGL3RC5B4RVE7QXJFKB4XDN3BA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.235225782 (13h53m7.979282629s) [noise=true , address=104.36.82.62:4439 , piece_id=ILRD4TEFW4PSLV4EGPORG3FM6U2HB3WSL5KP3PN566FAOGWODLYQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.269129867 (13h53m8.013186721s) [noise=true , address=104.36.82.5:4861 , piece_id=GRDGO5LNCETQWRU3M6LACGRW5MNA2KFLXUBT5R2UH6SEGWP6RVRQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.278216912 (13h53m8.022273764s) [noise=true , address=104.36.82.19:4405 , piece_id=MHM3MWZJFTXSJT7PTOFSWHUYO3L6NIBUV6VU66XXZDH3MC3EFR2Q, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.278247847 (13h53m8.022304708s) [seg_pos={0 0}, upload=14294]: Segment reached optimal threshold of 14 pieces. Aug 7 23:32:46.278485612 (13h53m8.022542463s) [noise=true , address=104.36.82.11:4128 , piece_id=XRPULHI7BYDF2JOIASSN5QKGEAC6FJS2F5C22WIY7EKYILRHZ73A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wABkAtAjoYkKyNoAiaVCxSvY6WgvHvAYFPCjpNojtpiYiGtDS): protocol: expected piece hash; context canceled; EOF Aug 7 23:32:46.278511698 (13h53m8.022568566s) [noise=true , address=77.90.31.12:4593 , piece_id=C3RVQHFS4L7SKPFFIZ52MAQWGD7P4JNG3SNPCNY53SQUD72PZRUA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kSxoZb9nmG6oGNytpFTsoiGh3ZPmebVVe8TZxGzANaw7EfdeF): protocol: expected piece hash; context canceled; EOF Aug 7 23:32:46.278642807 (13h53m8.022699657s) [noise=true , address=104.36.82.61:4381 , piece_id=TTZOSRP6YPL6P4C6PY5S4K7Z6CIZJDPX7NUNBCK73CWZPCA3HQHA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zU35MWWaaC1npkP5qGcWBXAkfxGsd5UsdkM2nwX9x7wUGCDbY): protocol: write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection; write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection Aug 7 23:32:46.278650926 (13h53m8.022707786s) [noise=true , address=104.36.82.37:4665 , piece_id=RUKKGC4Y3FIVLZO2GRWJN3J6P36VWP5VSC2HCLEU3MZDDFXODJAQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jP9JrCmoQSdqrzXZxkNN1ihdFkaUMjGZss6Fh19zbL3YyWj5N): protocol: write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection; write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection Aug 7 23:32:46.278674050 (13h53m8.022730911s) [seg_pos={0 0}, upload=14294]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:32:46.278751154 (13h53m8.022808014s) [upload=14294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:32:46.278756168 (13h53m8.022813029s) [upload=14294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:32:46.498660020 (13h53m8.242716882s) [upload=14294]: Done uploading segments... Aug 7 23:32:46.498701263 (13h53m8.242758128s) [upload=14294]: Upload finished. err: <nil> Duration: 3.010792998s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.XXXXXX.XX........................................X.XX...XX.] duration: 2.334634894s FAIL (2025-08-07 23:58:31 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.334679638s core uplink us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 110.1833ms remote: "tcp:185.244.226.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 23:58:31 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 110.203627ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 13 out of 60 is failed (threshold=3) [XXX.XX.XX.X.........................................X.XXX.X.] duration: 1.004624765s FAIL (2025-08-07 23:58:29 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.00466528s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 1.03798698s FAIL (2025-08-07 23:58:17 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== pload=14121]: Done uploading segments... Aug 7 23:22:28.505845197 (13h42m50.249902044s) [upload=14121]: Upload finished. err: <nil> Aug 7 23:23:10.377483605 (13h43m32.121540458s) [upload=14132]: Starting upload Aug 7 23:23:10.377535635 (13h43m32.121592495s) [upload=14132]: Done starting upload Aug 7 23:23:10.377603797 (13h43m32.121660647s) [upload=14132]: Uploading segments... Aug 7 23:23:10.377638935 (13h43m32.121695804s) [upload=14132]: Got next segment. Inline: true Aug 7 23:23:10.377643594 (13h43m32.121700454s) [upload=14132]: Waiting for error group managing segments... Aug 7 23:23:10.377688252 (13h43m32.121745104s) [upload=14132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:23:10.377693427 (13h43m32.121750286s) [upload=14132]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:23:10.377696805 (13h43m32.121753666s) [upload=14132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:23:10.562260724 (13h43m32.306317573s) [upload=14132]: Done uploading segments... Aug 7 23:23:10.562279228 (13h43m32.306336078s) [upload=14132]: Upload finished. err: <nil> Aug 7 23:24:07.896404858 (13h44m29.640461721s) [upload=14147]: Starting upload Aug 7 23:24:07.896490216 (13h44m29.640547079s) [upload=14147]: Done starting upload Aug 7 23:24:07.896580488 (13h44m29.640637339s) [upload=14147]: Uploading segments... Aug 7 23:24:07.896655006 (13h44m29.640711874s) [upload=14147]: Got next segment. Inline: true Aug 7 23:24:07.896660179 (13h44m29.640717039s) [upload=14147]: Waiting for error group managing segments... Aug 7 23:24:07.896714840 (13h44m29.6407717s) [upload=14147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:24:07.896719399 (13h44m29.64077626s) [upload=14147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:24:07.896722726 (13h44m29.640779586s) [upload=14147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:24:08.048898130 (13h44m29.792954991s) [upload=14147]: Done uploading segments... Aug 7 23:24:08.048952526 (13h44m29.793009386s) [upload=14147]: Upload finished. err: <nil> Aug 7 23:24:53.794754379 (13h45m15.53881125s) [upload=14162]: Starting upload Aug 7 23:24:53.794842317 (13h45m15.538899177s) [upload=14162]: Done starting upload Aug 7 23:24:53.794935964 (13h45m15.538992814s) [upload=14162]: Uploading segments... Aug 7 23:24:53.795003752 (13h45m15.539060623s) [upload=14162]: Got next segment. Inline: true Aug 7 23:24:53.795010814 (13h45m15.539067676s) [upload=14162]: Waiting for error group managing segments... Aug 7 23:24:53.795066607 (13h45m15.539123488s) [upload=14162]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:24:53.795158449 (13h45m15.5392153s) [upload=14162]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:24:53.795163166 (13h45m15.539220026s) [upload=14162]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:24:53.946994982 (13h45m15.691051833s) [upload=14162]: Done uploading segments... Aug 7 23:24:53.947013862 (13h45m15.691070711s) [upload=14162]: Upload finished. err: <nil> Aug 7 23:25:40.029856966 (13h46m1.773913836s) [upload=14177]: Starting upload Aug 7 23:25:40.029986938 (13h46m1.774043809s) [upload=14177]: Done starting upload Aug 7 23:25:40.030076796 (13h46m1.774133642s) [upload=14177]: Uploading segments... Aug 7 23:25:40.030132567 (13h46m1.774189438s) [upload=14177]: Got next segment. Inline: true Aug 7 23:25:40.030139885 (13h46m1.774196757s) [upload=14177]: Waiting for error group managing segments... Aug 7 23:25:40.030197588 (13h46m1.774254461s) [upload=14177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:25:40.030204146 (13h46m1.774261016s) [upload=14177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:25:40.030207984 (13h46m1.77426485s) [upload=14177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:25:40.184198742 (13h46m1.928255592s) [upload=14177]: Done uploading segments... Aug 7 23:25:40.184219017 (13h46m1.928275867s) [upload=14177]: Upload finished. err: <nil> Aug 7 23:26:54.625649110 (13h47m16.369705971s) [upload=14198]: Starting upload Aug 7 23:26:54.625738563 (13h47m16.369795424s) [upload=14198]: Done starting upload Aug 7 23:26:54.625842691 (13h47m16.369899539s) [upload=14198]: Uploading segments... Aug 7 23:26:54.626016839 (13h47m16.37007371s) [upload=14198]: Got next segment. Inline: true Aug 7 23:26:54.626052112 (13h47m16.370108973s) [upload=14198]: Waiting for error group managing segments... Aug 7 23:26:54.626129780 (13h47m16.370186645s) [upload=14198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:26:54.626137392 (13h47m16.370194264s) [upload=14198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:26:54.626143182 (13h47m16.370200048s) [upload=14198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:26:54.775804171 (13h47m16.519861021s) [upload=14198]: Done uploading segments... Aug 7 23:26:54.775824249 (13h47m16.519881102s) [upload=14198]: Upload finished. err: <nil> Aug 7 23:27:32.263746490 (13h47m54.007803361s) [upload=14208]: Starting upload Aug 7 23:27:32.263833655 (13h47m54.007890501s) [upload=14208]: Done starting upload Aug 7 23:27:32.263976536 (13h47m54.008033385s) [upload=14208]: Uploading segments... Aug 7 23:27:32.264070926 (13h47m54.008127796s) [upload=14208]: Got next segment. Inline: true Aug 7 23:27:32.264080356 (13h47m54.00813733s) [upload=14208]: Waiting for error group managing segments... Aug 7 23:27:32.264154657 (13h47m54.008211526s) [upload=14208]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:27:32.264163587 (13h47m54.008220454s) [upload=14208]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:27:32.264172017 (13h47m54.008228914s) [upload=14208]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:27:32.490188559 (13h47m54.23424541s) [upload=14208]: Done uploading segments... Aug 7 23:27:32.490207326 (13h47m54.234264172s) [upload=14208]: Upload finished. err: <nil> Aug 7 23:27:57.927940477 (13h48m19.671997323s) [upload=14217]: Starting upload Aug 7 23:27:57.928016423 (13h48m19.672073283s) [upload=14217]: Done starting upload Aug 7 23:27:57.928101290 (13h48m19.672158142s) [upload=14217]: Uploading segments... Aug 7 23:27:57.928155692 (13h48m19.672212566s) [upload=14217]: Got next segment. Inline: true Aug 7 23:27:57.928162009 (13h48m19.672218869s) [upload=14217]: Waiting for error group managing segments... Aug 7 23:27:57.928213294 (13h48m19.672270145s) [upload=14217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:27:57.928219212 (13h48m19.672276074s) [upload=14217]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:27:57.928223521 (13h48m19.672280383s) [upload=14217]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:27:58.087544005 (13h48m19.831600874s) [upload=14217]: Done uploading segments... Aug 7 23:27:58.087570226 (13h48m19.831627094s) [upload=14217]: Upload finished. err: <nil> Aug 7 23:29:12.765200935 (13h49m34.509257795s) [upload=14237]: Starting upload Aug 7 23:29:12.765253106 (13h49m34.509309977s) [upload=14237]: Done starting upload Aug 7 23:29:12.765324641 (13h49m34.509381499s) [upload=14237]: Uploading segments... Aug 7 23:29:12.765374187 (13h49m34.509431056s) [upload=14237]: Got next segment. Inline: true Aug 7 23:29:12.765378947 (13h49m34.509435807s) [upload=14237]: Waiting for error group managing segments... Aug 7 23:29:12.765430859 (13h49m34.50948772s) [upload=14237]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:29:12.765438106 (13h49m34.50949497s) [upload=14237]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:29:12.765443127 (13h49m34.509499988s) [upload=14237]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:29:12.926437674 (13h49m34.670494535s) [upload=14237]: Done uploading segments... Aug 7 23:29:12.926457418 (13h49m34.670514268s) [upload=14237]: Upload finished. err: <nil> Aug 7 23:29:42.225760316 (13h50m3.969817177s) [upload=14244]: Starting upload Aug 7 23:29:42.225855063 (13h50m3.969911929s) [upload=14244]: Done starting upload Aug 7 23:29:42.225962489 (13h50m3.970019355s) [upload=14244]: Uploading segments... Aug 7 23:29:42.226078484 (13h50m3.970135331s) [upload=14244]: Got next segment. Inline: true Aug 7 23:29:42.226083996 (13h50m3.970140842s) [upload=14244]: Waiting for error group managing segments... Aug 7 23:29:42.226143067 (13h50m3.970199914s) [upload=14244]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:29:42.226148511 (13h50m3.970205371s) [upload=14244]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:29:42.226153006 (13h50m3.970209865s) [upload=14244]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:29:42.401822947 (13h50m4.14587981s) [upload=14244]: Done uploading segments... Aug 7 23:29:42.401847556 (13h50m4.145904405s) [upload=14244]: Upload finished. err: <nil> Aug 7 23:30:31.529840301 (13h50m53.273897165s) [upload=14254]: Starting upload Aug 7 23:30:31.530002531 (13h50m53.274059378s) [upload=14254]: Done starting upload Aug 7 23:30:31.530201544 (13h50m53.274258391s) [upload=14254]: Uploading segments... Aug 7 23:30:31.530298120 (13h50m53.274354985s) [upload=14254]: Got next segment. Inline: true Aug 7 23:30:31.530303457 (13h50m53.274360317s) [upload=14254]: Waiting for error group managing segments... Aug 7 23:30:31.530353721 (13h50m53.274410567s) [upload=14254]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:30:31.530358927 (13h50m53.274415788s) [upload=14254]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:30:31.530363758 (13h50m53.274420619s) [upload=14254]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:30:31.696350967 (13h50m53.440407828s) [upload=14254]: Done uploading segments... Aug 7 23:30:31.696369460 (13h50m53.440426307s) [upload=14254]: Upload finished. err: <nil> Duration: 1.038040834s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 306.890523ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 23:58:17 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 306.916233ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 14 out of 60 is failed (threshold=3) [X.XXXXXX..XX.X...................................X...XX.X...] duration: 878.903413ms FAIL (2025-08-07 23:58:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ad finished. err: <nil> Aug 7 23:49:47.788743937 (14h10m9.532800783s) [upload=14591]: Starting upload Aug 7 23:49:47.788822734 (14h10m9.5328796s) [upload=14591]: Done starting upload Aug 7 23:49:47.788889290 (14h10m9.532946138s) [upload=14591]: Uploading segments... Aug 7 23:49:47.788950938 (14h10m9.533007798s) [upload=14591]: Got next segment. Inline: true Aug 7 23:49:47.788955912 (14h10m9.533012775s) [upload=14591]: Waiting for error group managing segments... Aug 7 23:49:47.789005080 (14h10m9.533061926s) [upload=14591]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:49:47.789009334 (14h10m9.533066194s) [upload=14591]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:49:47.789014285 (14h10m9.53307113s) [upload=14591]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:49:47.932333015 (14h10m9.676389863s) [upload=14591]: Done uploading segments... Aug 7 23:49:47.932362673 (14h10m9.676419519s) [upload=14591]: Upload finished. err: <nil> Aug 7 23:50:23.531847225 (14h10m45.275904072s) [upload=14601]: Starting upload Aug 7 23:50:23.531926184 (14h10m45.275983047s) [upload=14601]: Done starting upload Aug 7 23:50:23.531993110 (14h10m45.27604996s) [upload=14601]: Uploading segments... Aug 7 23:50:23.532035359 (14h10m45.276092227s) [upload=14601]: Got next segment. Inline: true Aug 7 23:50:23.532041373 (14h10m45.276098223s) [upload=14601]: Waiting for error group managing segments... Aug 7 23:50:23.532083061 (14h10m45.276139912s) [upload=14601]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:23.532088436 (14h10m45.276145285s) [upload=14601]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:23.532091688 (14h10m45.276148539s) [upload=14601]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:23.673405160 (14h10m45.417462011s) [upload=14601]: Done uploading segments... Aug 7 23:50:23.673423409 (14h10m45.417480259s) [upload=14601]: Upload finished. err: <nil> Aug 7 23:51:32.204614635 (14h11m53.948671502s) [upload=14620]: Starting upload Aug 7 23:51:32.204700450 (14h11m53.948757313s) [upload=14620]: Done starting upload Aug 7 23:51:32.204781575 (14h11m53.948838427s) [upload=14620]: Uploading segments... Aug 7 23:51:32.204839741 (14h11m53.94889661s) [upload=14620]: Got next segment. Inline: true Aug 7 23:51:32.204847624 (14h11m53.948904474s) [upload=14620]: Waiting for error group managing segments... Aug 7 23:51:32.204902472 (14h11m53.948959327s) [upload=14620]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:51:32.204907775 (14h11m53.948964625s) [upload=14620]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:51:32.204911177 (14h11m53.948968027s) [upload=14620]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:01.937570352 (14h12m23.681627223s) [upload=14620]: Done uploading segments... Aug 7 23:52:01.937615027 (14h12m23.681671888s) [upload=14620]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:52:03.484443714 (14h12m25.228500575s) [upload=14625]: Starting upload Aug 7 23:52:03.484557174 (14h12m25.228614037s) [upload=14625]: Done starting upload Aug 7 23:52:03.484649660 (14h12m25.22870651s) [upload=14625]: Uploading segments... Aug 7 23:52:03.484705970 (14h12m25.22876283s) [upload=14625]: Got next segment. Inline: true Aug 7 23:52:03.484711540 (14h12m25.228768402s) [upload=14625]: Waiting for error group managing segments... Aug 7 23:52:03.484758149 (14h12m25.228815009s) [upload=14625]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:03.484763263 (14h12m25.228820122s) [upload=14625]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:03.484766746 (14h12m25.228823606s) [upload=14625]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:03.643631412 (14h12m25.387688267s) [upload=14625]: Done uploading segments... Aug 7 23:52:03.643670624 (14h12m25.387727482s) [upload=14625]: Upload finished. err: <nil> Aug 7 23:52:25.641252610 (14h12m47.385309456s) [upload=14634]: Starting upload Aug 7 23:52:25.641347708 (14h12m47.385404571s) [upload=14634]: Done starting upload Aug 7 23:52:25.641412738 (14h12m47.385469584s) [upload=14634]: Uploading segments... Aug 7 23:52:25.641487429 (14h12m47.385544276s) [upload=14634]: Got next segment. Inline: true Aug 7 23:52:25.641494756 (14h12m47.385551602s) [upload=14634]: Waiting for error group managing segments... Aug 7 23:52:25.641535502 (14h12m47.385592365s) [upload=14634]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:25.641556088 (14h12m47.385612953s) [upload=14634]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:25.641559364 (14h12m47.385616224s) [upload=14634]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:25.852337418 (14h12m47.596394279s) [upload=14634]: Done uploading segments... Aug 7 23:52:25.852355736 (14h12m47.596412584s) [upload=14634]: Upload finished. err: <nil> Aug 7 23:52:51.292140355 (14h13m13.036197215s) [upload=14641]: Starting upload Aug 7 23:52:51.292216133 (14h13m13.036272985s) [upload=14641]: Done starting upload Aug 7 23:52:51.292307446 (14h13m13.036364307s) [upload=14641]: Uploading segments... Aug 7 23:52:51.292389087 (14h13m13.036445956s) [upload=14641]: Got next segment. Inline: true Aug 7 23:52:51.292394933 (14h13m13.036451793s) [upload=14641]: Waiting for error group managing segments... Aug 7 23:52:51.292456032 (14h13m13.036512883s) [upload=14641]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:51.292461787 (14h13m13.036518647s) [upload=14641]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:51.292467676 (14h13m13.036524537s) [upload=14641]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:51.436915828 (14h13m13.180972678s) [upload=14641]: Done uploading segments... Aug 7 23:52:51.436934458 (14h13m13.18099131s) [upload=14641]: Upload finished. err: <nil> Aug 7 23:53:10.946261153 (14h13m32.690318028s) [upload=14646]: Starting upload Aug 7 23:53:10.946350198 (14h13m32.690407065s) [upload=14646]: Done starting upload Aug 7 23:53:10.946435423 (14h13m32.690492276s) [upload=14646]: Uploading segments... Aug 7 23:53:10.946489255 (14h13m32.690546124s) [upload=14646]: Got next segment. Inline: true Aug 7 23:53:10.946497660 (14h13m32.690554519s) [upload=14646]: Waiting for error group managing segments... Aug 7 23:53:10.946542678 (14h13m32.690599529s) [upload=14646]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:10.946548267 (14h13m32.690605128s) [upload=14646]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:10.946553792 (14h13m32.690610652s) [upload=14646]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:14.554651436 (14h13m36.298708309s) [upload=14646]: Done uploading segments... Aug 7 23:53:14.554675183 (14h13m36.298732045s) [upload=14646]: Upload finished. err: <nil> Aug 7 23:53:53.584502029 (14h14m15.328558897s) [upload=14658]: Starting upload Aug 7 23:53:53.584599757 (14h14m15.328656616s) [upload=14658]: Done starting upload Aug 7 23:53:53.584702364 (14h14m15.328759218s) [upload=14658]: Uploading segments... Aug 7 23:53:53.584798610 (14h14m15.32885548s) [upload=14658]: Got next segment. Inline: true Aug 7 23:53:53.584806594 (14h14m15.328863461s) [upload=14658]: Waiting for error group managing segments... Aug 7 23:53:53.584875193 (14h14m15.328932062s) [upload=14658]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:53.584882324 (14h14m15.328939193s) [upload=14658]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:53.584901657 (14h14m15.328958528s) [upload=14658]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:23.318821682 (14h14m45.062878542s) [upload=14658]: Done uploading segments... Aug 7 23:54:23.318863033 (14h14m45.062919884s) [upload=14658]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:36.775209359 (14h14m58.519266209s) [upload=14671]: Starting upload Aug 7 23:54:36.775292235 (14h14m58.519349112s) [upload=14671]: Done starting upload Aug 7 23:54:36.775412031 (14h14m58.519468883s) [upload=14671]: Uploading segments... Aug 7 23:54:36.775480767 (14h14m58.519537636s) [upload=14671]: Got next segment. Inline: true Aug 7 23:54:36.775487320 (14h14m58.519544181s) [upload=14671]: Waiting for error group managing segments... Aug 7 23:54:36.775539752 (14h14m58.519596612s) [upload=14671]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:36.775545893 (14h14m58.519602753s) [upload=14671]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:36.775549619 (14h14m58.519606479s) [upload=14671]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:36.926276427 (14h14m58.670333277s) [upload=14671]: Done uploading segments... Aug 7 23:54:36.926295206 (14h14m58.670352057s) [upload=14671]: Upload finished. err: <nil> Aug 7 23:54:57.675347843 (14h15m19.419404694s) [upload=14676]: Starting upload Aug 7 23:54:57.675412352 (14h15m19.419469212s) [upload=14676]: Done starting upload Aug 7 23:54:57.675475670 (14h15m19.41953252s) [upload=14676]: Uploading segments... Aug 7 23:54:57.675516336 (14h15m19.419573205s) [upload=14676]: Got next segment. Inline: true Aug 7 23:54:57.675521071 (14h15m19.419577931s) [upload=14676]: Waiting for error group managing segments... Aug 7 23:54:57.675565512 (14h15m19.419622375s) [upload=14676]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:57.675574255 (14h15m19.419631202s) [upload=14676]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:57.675580694 (14h15m19.419637563s) [upload=14676]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:27.407552485 (14h15m49.151609344s) [upload=14676]: Done uploading segments... Aug 7 23:55:27.407592978 (14h15m49.151649829s) [upload=14676]: Upload finished. err: metaclient: context deadline exceeded Duration: 878.927687ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.X.XX..X....................................................] duration: 2.717949516s FAIL (2025-08-07 23:58:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.718013714s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 14 out of 60 is failed (threshold=3) [XXXX.XX.XX.X.........................................X.XXX.X] duration: 30.000642743s FAIL (2025-08-07 23:58:06 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.000668201s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 13 out of 60 is failed (threshold=3) [..XXXXXX.XX........................................X.XX...XX] duration: 30.000661837s FAIL (2025-08-07 23:57:50 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 30.000685699s core uplink us1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)