Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 8 out of 60 is failed (threshold=3) [..XXXXX................................................XX..X] duration: 1m0.000575122s FAIL (2025-08-07 23:57:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 4570]: Upload finished. err: <nil> Aug 7 23:49:25.777587368 (14h9m47.521644216s) [upload=14582]: Starting upload Aug 7 23:49:25.777649686 (14h9m47.521706547s) [upload=14582]: Done starting upload Aug 7 23:49:25.777703819 (14h9m47.521760666s) [upload=14582]: Uploading segments... Aug 7 23:49:25.777747152 (14h9m47.521804022s) [upload=14582]: Got next segment. Inline: true Aug 7 23:49:25.777751825 (14h9m47.521808685s) [upload=14582]: Waiting for error group managing segments... Aug 7 23:49:25.777787950 (14h9m47.521844813s) [upload=14582]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:49:25.777792452 (14h9m47.521849312s) [upload=14582]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:49:25.777795064 (14h9m47.521851924s) [upload=14582]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:49:25.918566855 (14h9m47.662623707s) [upload=14582]: Done uploading segments... Aug 7 23:49:25.918602989 (14h9m47.66265984s) [upload=14582]: Upload finished. err: <nil> Aug 7 23:50:24.097841430 (14h10m45.841898277s) [upload=14602]: Starting upload Aug 7 23:50:24.097929459 (14h10m45.841986306s) [upload=14602]: Done starting upload Aug 7 23:50:24.097991265 (14h10m45.842048125s) [upload=14602]: Uploading segments... Aug 7 23:50:24.098029893 (14h10m45.842086762s) [upload=14602]: Got next segment. Inline: true Aug 7 23:50:24.098034701 (14h10m45.84209156s) [upload=14602]: Waiting for error group managing segments... Aug 7 23:50:24.098071586 (14h10m45.842128432s) [upload=14602]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:24.098076409 (14h10m45.842133268s) [upload=14602]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:24.098079284 (14h10m45.842136144s) [upload=14602]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:24.238728801 (14h10m45.982785662s) [upload=14602]: Done uploading segments... Aug 7 23:50:24.238753595 (14h10m45.982810442s) [upload=14602]: Upload finished. err: <nil> Aug 7 23:51:03.108197922 (14h11m24.852254772s) [upload=14612]: Starting upload Aug 7 23:51:03.108300837 (14h11m24.852357707s) [upload=14612]: Done starting upload Aug 7 23:51:03.108453294 (14h11m24.852510146s) [upload=14612]: Uploading segments... Aug 7 23:51:03.108972593 (14h11m24.853029491s) [upload=14612]: Got next segment. Inline: true Aug 7 23:51:03.108988857 (14h11m24.85304573s) [upload=14612]: Waiting for error group managing segments... Aug 7 23:51:03.109073931 (14h11m24.853130786s) [upload=14612]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:51:03.109080082 (14h11m24.853136933s) [upload=14612]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:51:03.109083894 (14h11m24.853140746s) [upload=14612]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:03.446817008 (14h11m25.190873858s) [upload=14612]: Done uploading segments... Aug 7 23:51:03.446842582 (14h11m25.190899436s) [upload=14612]: Upload finished. err: <nil> Aug 7 23:52:09.742356648 (14h12m31.486413495s) [upload=14628]: Starting upload Aug 7 23:52:09.742413553 (14h12m31.486470415s) [upload=14628]: Done starting upload Aug 7 23:52:09.742476702 (14h12m31.486533548s) [upload=14628]: Uploading segments... Aug 7 23:52:09.742511425 (14h12m31.486568294s) [upload=14628]: Got next segment. Inline: true Aug 7 23:52:09.742515305 (14h12m31.486572152s) [upload=14628]: Waiting for error group managing segments... Aug 7 23:52:09.742550700 (14h12m31.486607545s) [upload=14628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:09.742555041 (14h12m31.486611889s) [upload=14628]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:09.742557771 (14h12m31.486614632s) [upload=14628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:09.895389395 (14h12m31.639446255s) [upload=14628]: Done uploading segments... Aug 7 23:52:09.895421188 (14h12m31.639478039s) [upload=14628]: Upload finished. err: <nil> Aug 7 23:52:28.792030311 (14h12m50.536087163s) [upload=14635]: Starting upload Aug 7 23:52:28.792092644 (14h12m50.536149508s) [upload=14635]: Done starting upload Aug 7 23:52:28.792152087 (14h12m50.53620894s) [upload=14635]: Uploading segments... Aug 7 23:52:28.792190462 (14h12m50.536247332s) [upload=14635]: Got next segment. Inline: true Aug 7 23:52:28.792195687 (14h12m50.536252548s) [upload=14635]: Waiting for error group managing segments... Aug 7 23:52:28.792239958 (14h12m50.536296808s) [upload=14635]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:28.792245830 (14h12m50.536302691s) [upload=14635]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:28.792249165 (14h12m50.536306026s) [upload=14635]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:28.947698658 (14h12m50.691755509s) [upload=14635]: Done uploading segments... Aug 7 23:52:28.947717721 (14h12m50.691774571s) [upload=14635]: Upload finished. err: <nil> Aug 7 23:53:17.152926358 (14h13m38.896983229s) [upload=14650]: Starting upload Aug 7 23:53:17.153036870 (14h13m38.89709373s) [upload=14650]: Done starting upload Aug 7 23:53:17.153154678 (14h13m38.897211544s) [upload=14650]: Uploading segments... Aug 7 23:53:17.153233089 (14h13m38.897289957s) [upload=14650]: Got next segment. Inline: true Aug 7 23:53:17.153240256 (14h13m38.897297124s) [upload=14650]: Waiting for error group managing segments... Aug 7 23:53:17.153318892 (14h13m38.897375757s) [upload=14650]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:17.153325816 (14h13m38.89738268s) [upload=14650]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:17.153330453 (14h13m38.897387314s) [upload=14650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:16.875757982 (14h14m38.619814845s) [upload=14650]: Done uploading segments... Aug 7 23:54:16.875781849 (14h14m38.619838701s) [upload=14650]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:24.158448119 (14h14m45.90250498s) [upload=14667]: Starting upload Aug 7 23:54:24.158525946 (14h14m45.902582806s) [upload=14667]: Done starting upload Aug 7 23:54:24.158598972 (14h14m45.902655824s) [upload=14667]: Uploading segments... Aug 7 23:54:24.158654148 (14h14m45.902711017s) [upload=14667]: Got next segment. Inline: true Aug 7 23:54:24.158659995 (14h14m45.902716856s) [upload=14667]: Waiting for error group managing segments... Aug 7 23:54:24.158714896 (14h14m45.902771747s) [upload=14667]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:24.158720357 (14h14m45.902777209s) [upload=14667]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:24.158724105 (14h14m45.902780965s) [upload=14667]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:23.888400190 (14h15m45.63245704s) [upload=14667]: Done uploading segments... Aug 7 23:55:23.888419087 (14h15m45.632475938s) [upload=14667]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:24.297146309 (14h15m46.041203174s) [upload=14687]: Starting upload Aug 7 23:55:24.297232144 (14h15m46.04128901s) [upload=14687]: Done starting upload Aug 7 23:55:24.297315772 (14h15m46.041372619s) [upload=14687]: Uploading segments... Aug 7 23:55:24.297396238 (14h15m46.041453151s) [upload=14687]: Got next segment. Inline: true Aug 7 23:55:24.297403729 (14h15m46.04146059s) [upload=14687]: Waiting for error group managing segments... Aug 7 23:55:24.297444116 (14h15m46.041500976s) [upload=14687]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:24.297450971 (14h15m46.041507831s) [upload=14687]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:24.297453883 (14h15m46.041510749s) [upload=14687]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:24.439126204 (14h15m46.183183055s) [upload=14687]: Done uploading segments... Aug 7 23:55:24.439141121 (14h15m46.183197972s) [upload=14687]: Upload finished. err: <nil> Aug 7 23:56:03.068174127 (14h16m24.812230977s) [upload=14695]: Starting upload Aug 7 23:56:03.068258755 (14h16m24.812315617s) [upload=14695]: Done starting upload Aug 7 23:56:03.068368961 (14h16m24.812425812s) [upload=14695]: Uploading segments... Aug 7 23:56:03.068437851 (14h16m24.812494719s) [upload=14695]: Got next segment. Inline: true Aug 7 23:56:03.068445058 (14h16m24.812501927s) [upload=14695]: Waiting for error group managing segments... Aug 7 23:56:03.068516175 (14h16m24.812573049s) [upload=14695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:03.068523275 (14h16m24.812580137s) [upload=14695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:03.068527513 (14h16m24.812584488s) [upload=14695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:03.370378590 (14h16m25.114435452s) [upload=14695]: Done uploading segments... Aug 7 23:56:03.370401742 (14h16m25.114458656s) [upload=14695]: Upload finished. err: <nil> Aug 7 23:56:41.636734874 (14h17m3.380791742s) [upload=14707]: Starting upload Aug 7 23:56:41.636844337 (14h17m3.380901201s) [upload=14707]: Done starting upload Aug 7 23:56:41.636938361 (14h17m3.380995223s) [upload=14707]: Uploading segments... Aug 7 23:56:41.637004566 (14h17m3.381061436s) [upload=14707]: Got next segment. Inline: true Aug 7 23:56:41.637011720 (14h17m3.381068581s) [upload=14707]: Waiting for error group managing segments... Aug 7 23:56:41.637073380 (14h17m3.381130242s) [upload=14707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:41.637079684 (14h17m3.381136546s) [upload=14707]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:41.637083892 (14h17m3.381140753s) [upload=14707]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:41.369074470 (14h18m3.113131318s) [upload=14707]: Done uploading segments... Aug 7 23:57:41.369096313 (14h18m3.11315316s) [upload=14707]: Upload finished. err: metaclient: context deadline exceeded Duration: 1m0.000611343s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 112.708832ms remote: "tcp:185.244.226.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 23:57:38 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 112.734706ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 303.421801ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 23:57:37 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 303.499439ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 3.286142481s FAIL (2025-08-07 23:57:34 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== 294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:32:45.172127356 (13h53m6.916184241s) [upload=14294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:32:45.390149151 (13h53m7.134206011s) [seg_pos={0 0}, upload=14294]: Begin upload segment... Aug 7 23:32:45.390215910 (13h53m7.134272761s) [seg_pos={0 0}, upload=14294]: Done begin upload segment. Aug 7 23:32:45.390249193 (13h53m7.134306059s) [noise=true , address=104.36.82.19:4405 , piece_id=MHM3MWZJFTXSJT7PTOFSWHUYO3L6NIBUV6VU66XXZDH3MC3EFR2Q, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390272919 (13h53m7.134329788s) [noise=true , address=77.90.31.12:4593 , piece_id=C3RVQHFS4L7SKPFFIZ52MAQWGD7P4JNG3SNPCNY53SQUD72PZRUA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390658970 (13h53m7.13471583s) [noise=true , address=104.36.82.16:4321 , piece_id=YTF4RWV2SD6Z4UV4XRYPBRGPCTSUTC6TQKL3DHVKU65ALTNA4TEQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390852777 (13h53m7.134909659s) [noise=true , address=104.36.82.34:4566 , piece_id=3YEAUSXEL7WNZ3O7WM24Z2DKY2TDVKMRZSHMEPMNEL2LR7BY7DBQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390957991 (13h53m7.135014851s) [noise=true , address=104.36.82.47:4523 , piece_id=MJIMHSO66V7OWFFK2UALPKSI7CGGCTHNPZINDXMVIHDLENIW5LFA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.391483291 (13h53m7.135540162s) [noise=true , address=104.36.82.5:4861 , piece_id=GRDGO5LNCETQWRU3M6LACGRW5MNA2KFLXUBT5R2UH6SEGWP6RVRQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.391931094 (13h53m7.135987964s) [noise=true , address=104.36.82.10:4090 , piece_id=V6NPPVIYEH6WLIW7OMT3NDFPR7UDWIT46GW6AFCAWUJTBLC3O55Q, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392250266 (13h53m7.136307136s) [noise=true , address=104.36.82.62:4439 , piece_id=ILRD4TEFW4PSLV4EGPORG3FM6U2HB3WSL5KP3PN566FAOGWODLYQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392611689 (13h53m7.136668559s) [noise=true , address=104.36.82.54:4160 , piece_id=GDBORRYX5QBQGH7ZMMUUAV7AAR5UHZOTK5FTQUPC7YKZC2KD7I3A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392917410 (13h53m7.136974282s) [noise=true , address=104.36.82.23:4477 , piece_id=HAF6424ZCIBGSR4US5LN5ZWE5JMNMLFGUMQQ4YP2ARP7PQXBH5GQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393270536 (13h53m7.137327398s) [noise=true , address=104.36.82.44:4881 , piece_id=XXVSXG3ILFXG4IVMELHL45TRRHDFVVJMJ5LGOHU5EDHG6XXCDH6A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393582635 (13h53m7.137639498s) [noise=true , address=104.36.82.37:4665 , piece_id=RUKKGC4Y3FIVLZO2GRWJN3J6P36VWP5VSC2HCLEU3MZDDFXODJAQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393856926 (13h53m7.137913788s) [noise=true , address=104.36.82.11:4128 , piece_id=XRPULHI7BYDF2JOIASSN5QKGEAC6FJS2F5C22WIY7EKYILRHZ73A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394196364 (13h53m7.138253227s) [noise=true , address=104.36.82.38:4706 , piece_id=BIHIPQX6LRWBNFN4SDC3K3V22AMGL3RC5B4RVE7QXJFKB4XDN3BA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394484397 (13h53m7.138541257s) [noise=true , address=104.36.82.61:4381 , piece_id=TTZOSRP6YPL6P4C6PY5S4K7Z6CIZJDPX7NUNBCK73CWZPCA3HQHA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394593709 (13h53m7.138650578s) [noise=true , address=45.91.116.5:4300 , piece_id=MADJCANN2O35MKXT2PIR5PGMJ3PNTAQAJNNGOK6ISJNVAH2L7YIA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.395093013 (13h53m7.139149879s) [noise=true , address=104.36.82.22:4530 , piece_id=HMUUW5GYH6TRHYXP4YXBHD566FWQRILEHI53AVURA5QTQGG6USFA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.395447945 (13h53m7.139504805s) [noise=true , address=104.36.82.58:4293 , piece_id=I2ETFXYTHYWYJYSHTVOBQTO6ZPGNMA4D3NVKYDE2W34EH4SQE23A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.954469844 (13h53m7.698526694s) [upload=14294]: Next returned nil segment Aug 7 23:32:45.954487953 (13h53m7.69854487s) [upload=14294]: Waiting for error group managing segments... Aug 7 23:32:46.126888627 (13h53m7.870945474s) [noise=true , address=104.36.82.10:4090 , piece_id=V6NPPVIYEH6WLIW7OMT3NDFPR7UDWIT46GW6AFCAWUJTBLC3O55Q, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.132983844 (13h53m7.877040704s) [noise=true , address=104.36.82.16:4321 , piece_id=YTF4RWV2SD6Z4UV4XRYPBRGPCTSUTC6TQKL3DHVKU65ALTNA4TEQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.138277216 (13h53m7.882334067s) [noise=true , address=104.36.82.23:4477 , piece_id=HAF6424ZCIBGSR4US5LN5ZWE5JMNMLFGUMQQ4YP2ARP7PQXBH5GQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.142033326 (13h53m7.886090185s) [noise=true , address=104.36.82.34:4566 , piece_id=3YEAUSXEL7WNZ3O7WM24Z2DKY2TDVKMRZSHMEPMNEL2LR7BY7DBQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.151247206 (13h53m7.895304058s) [noise=true , address=104.36.82.22:4530 , piece_id=HMUUW5GYH6TRHYXP4YXBHD566FWQRILEHI53AVURA5QTQGG6USFA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.151981081 (13h53m7.896037932s) [noise=true , address=104.36.82.58:4293 , piece_id=I2ETFXYTHYWYJYSHTVOBQTO6ZPGNMA4D3NVKYDE2W34EH4SQE23A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.165022882 (13h53m7.909079743s) [noise=true , address=104.36.82.44:4881 , piece_id=XXVSXG3ILFXG4IVMELHL45TRRHDFVVJMJ5LGOHU5EDHG6XXCDH6A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.184418821 (13h53m7.928475667s) [noise=true , address=104.36.82.47:4523 , piece_id=MJIMHSO66V7OWFFK2UALPKSI7CGGCTHNPZINDXMVIHDLENIW5LFA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.184532669 (13h53m7.928589516s) [noise=true , address=45.91.116.5:4300 , piece_id=MADJCANN2O35MKXT2PIR5PGMJ3PNTAQAJNNGOK6ISJNVAH2L7YIA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.193375808 (13h53m7.937432656s) [noise=true , address=104.36.82.54:4160 , piece_id=GDBORRYX5QBQGH7ZMMUUAV7AAR5UHZOTK5FTQUPC7YKZC2KD7I3A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.204247501 (13h53m7.948304372s) [noise=true , address=104.36.82.38:4706 , piece_id=BIHIPQX6LRWBNFN4SDC3K3V22AMGL3RC5B4RVE7QXJFKB4XDN3BA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.235225782 (13h53m7.979282629s) [noise=true , address=104.36.82.62:4439 , piece_id=ILRD4TEFW4PSLV4EGPORG3FM6U2HB3WSL5KP3PN566FAOGWODLYQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.269129867 (13h53m8.013186721s) [noise=true , address=104.36.82.5:4861 , piece_id=GRDGO5LNCETQWRU3M6LACGRW5MNA2KFLXUBT5R2UH6SEGWP6RVRQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.278216912 (13h53m8.022273764s) [noise=true , address=104.36.82.19:4405 , piece_id=MHM3MWZJFTXSJT7PTOFSWHUYO3L6NIBUV6VU66XXZDH3MC3EFR2Q, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.278247847 (13h53m8.022304708s) [seg_pos={0 0}, upload=14294]: Segment reached optimal threshold of 14 pieces. Aug 7 23:32:46.278485612 (13h53m8.022542463s) [noise=true , address=104.36.82.11:4128 , piece_id=XRPULHI7BYDF2JOIASSN5QKGEAC6FJS2F5C22WIY7EKYILRHZ73A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wABkAtAjoYkKyNoAiaVCxSvY6WgvHvAYFPCjpNojtpiYiGtDS): protocol: expected piece hash; context canceled; EOF Aug 7 23:32:46.278511698 (13h53m8.022568566s) [noise=true , address=77.90.31.12:4593 , piece_id=C3RVQHFS4L7SKPFFIZ52MAQWGD7P4JNG3SNPCNY53SQUD72PZRUA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kSxoZb9nmG6oGNytpFTsoiGh3ZPmebVVe8TZxGzANaw7EfdeF): protocol: expected piece hash; context canceled; EOF Aug 7 23:32:46.278642807 (13h53m8.022699657s) [noise=true , address=104.36.82.61:4381 , piece_id=TTZOSRP6YPL6P4C6PY5S4K7Z6CIZJDPX7NUNBCK73CWZPCA3HQHA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zU35MWWaaC1npkP5qGcWBXAkfxGsd5UsdkM2nwX9x7wUGCDbY): protocol: write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection; write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection Aug 7 23:32:46.278650926 (13h53m8.022707786s) [noise=true , address=104.36.82.37:4665 , piece_id=RUKKGC4Y3FIVLZO2GRWJN3J6P36VWP5VSC2HCLEU3MZDDFXODJAQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jP9JrCmoQSdqrzXZxkNN1ihdFkaUMjGZss6Fh19zbL3YyWj5N): protocol: write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection; write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection Aug 7 23:32:46.278674050 (13h53m8.022730911s) [seg_pos={0 0}, upload=14294]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:32:46.278751154 (13h53m8.022808014s) [upload=14294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:32:46.278756168 (13h53m8.022813029s) [upload=14294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:32:46.498660020 (13h53m8.242716882s) [upload=14294]: Done uploading segments... Aug 7 23:32:46.498701263 (13h53m8.242758128s) [upload=14294]: Upload finished. err: <nil> Duration: 3.286178533s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..X.XX..X...................................................] duration: 2.771086652s FAIL (2025-08-07 23:57:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.771118573s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 6 out of 60 is failed (threshold=3) [....XXX...XX.X..............................................] duration: 1.122242328s FAIL (2025-08-07 23:57:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.122269364s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 14 out of 60 is failed (threshold=3) [XXXXX.XX.XX.X.........................................X.XXX.] duration: 890.314368ms FAIL (2025-08-07 23:57:22 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 890.34152ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [.XXXXXXXXXX...........X............................XX.X.X...] duration: 2.411654494s FAIL (2025-08-07 23:57:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== exceeded Aug 7 23:53:43.167705717 (14h14m4.911762564s) [upload=14648]: Done uploading segments... Aug 7 23:53:43.167737922 (14h14m4.911794768s) [upload=14648]: Upload finished. err: context deadline exceeded Aug 7 23:53:43.606184070 (14h14m5.350240917s) [upload=14655]: Starting upload Aug 7 23:53:43.606251699 (14h14m5.350308565s) [upload=14655]: Done starting upload Aug 7 23:53:43.606378812 (14h14m5.350435674s) [upload=14655]: Uploading segments... Aug 7 23:53:43.606613336 (14h14m5.350670197s) [upload=14655]: Got next segment. Inline: false Aug 7 23:53:43.606652700 (14h14m5.35070955s) [upload=14655]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:43.606683541 (14h14m5.350740402s) [upload=14655]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:53:43.711705294 (14h14m5.455762157s) [seg_pos={0 0}, upload=14655]: Begin upload segment... Aug 7 23:53:43.711752671 (14h14m5.455809532s) [seg_pos={0 0}, upload=14655]: Done begin upload segment. Aug 7 23:53:43.711786432 (14h14m5.455843291s) [noise=true , address=104.36.82.10:4077 , piece_id=XOE3BBCSJ64O7A7LC7HAAYEBF3F6CIU77UVI5HY3RI2FWSXEHJXA, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.711812017 (14h14m5.455868889s) [noise=true , address=77.90.31.16:4735 , piece_id=OE7TL5NB6UH3A4JE2BDYFA4FZKHCRWT6YK6KJZHD6JJ55FOKXIEQ, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.712247313 (14h14m5.456304173s) [noise=true , address=45.91.116.10:4081 , piece_id=MMI5DUXMAD27VJI7VRQHBBEC2MJUDT5KWX2OGWLUPMTZLBIRWSAA, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.712257082 (14h14m5.456313945s) [noise=true , address=77.90.31.13:4626 , piece_id=74QG6PW6CJDJRESXUF534PQ463VRJQF33PMQRX6W6VVQCQMX2SUA, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.712579895 (14h14m5.456636755s) [noise=true , address=104.36.82.15:4258 , piece_id=JWNJIQN5WUWHVOQOQASGTGXOZWJVZZICABZCFOGZQVCDAVZ7WNVQ, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.712592145 (14h14m5.456649005s) [noise=true , address=45.91.116.4:4939 , piece_id=YLQB6XZJ6FL7SBJQ6FE4VIGKE5QYA3XPFU4E37OQKF5O2A7ZV4DQ, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.712863854 (14h14m5.456920715s) [noise=true , address=104.36.82.9:4007 , piece_id=XHRH7RHA6VKCFN7TDCMKIP6EH5EQWW3ZWVP7QE67X6LS7IR4J2MQ, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.712991802 (14h14m5.457048663s) [noise=true , address=77.90.31.13:4628 , piece_id=ZLWT3WANPKHZ6XFV7PC3KZULY73ZEOYD3E3EVY3OMIQI3OYM3LKA, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.713196040 (14h14m5.4572529s) [noise=true , address=104.36.82.14:4222 , piece_id=AAQ6HQOYMMNNHXABB2MOCCJHF43PXWJZH4SDYF6ZNSLQWMKY32AA, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.713340669 (14h14m5.457397532s) [noise=true , address=45.91.116.6:4975 , piece_id=BBULAO6EEFXHYP2AKMUF63YRSQJGVKXLIWGCKWUM5JYUVLZAB25Q, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.713490538 (14h14m5.457547399s) [noise=true , address=104.36.82.11:4114 , piece_id=IKJQ5SDEBNDZGJ6TUDTQBA2A74BQC6WV3SC4UKE7AGWEENDBI57Q, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.713730980 (14h14m5.45778784s) [noise=true , address=45.91.116.8:4045 , piece_id=7PDSFMO4MC4SMKYW3MN6OIU33MIKCLFBSJ6LYFQAGUFMOZQCLVXQ, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.713818259 (14h14m5.45787511s) [noise=true , address=77.90.31.19:4843 , piece_id=HLLIY2T4U45TYQ7L7RKY26MUTAA4KWAZ6LV7KP6HOTRJURY3WUSQ, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.714085345 (14h14m5.458142208s) [noise=true , address=45.91.116.5:4299 , piece_id=F5FM3VSMCDN3BSCGDQO5XGPZPQACVJAQG4Y2HWJEPQUF7BFGWNUA, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.714148553 (14h14m5.458205416s) [noise=true , address=77.90.31.14:4662 , piece_id=FDS6SHOXCIIUA4DKBYNMKY67RG6X5OLUKJLZ5M2M76UQB3ANWPLA, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.714360745 (14h14m5.458417608s) [noise=true , address=45.91.116.7:4010 , piece_id=RXF2GDUICLNB25V6S2CVFIKNTQHZOLWDZZWV3AEYWONFCFPZE2GA, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:44.327859573 (14h14m6.071916423s) [upload=14655]: Next returned nil segment Aug 7 23:53:44.327877823 (14h14m6.071934683s) [upload=14655]: Waiting for error group managing segments... Aug 7 23:53:44.492864996 (14h14m6.236921844s) [noise=true , address=104.36.82.11:4114 , piece_id=IKJQ5SDEBNDZGJ6TUDTQBA2A74BQC6WV3SC4UKE7AGWEENDBI57Q, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.496078923 (14h14m6.240135773s) [noise=true , address=104.36.82.9:4007 , piece_id=XHRH7RHA6VKCFN7TDCMKIP6EH5EQWW3ZWVP7QE67X6LS7IR4J2MQ, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.519818170 (14h14m6.263875019s) [noise=true , address=104.36.82.15:4258 , piece_id=JWNJIQN5WUWHVOQOQASGTGXOZWJVZZICABZCFOGZQVCDAVZ7WNVQ, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.540418125 (14h14m6.284474976s) [noise=true , address=45.91.116.7:4010 , piece_id=RXF2GDUICLNB25V6S2CVFIKNTQHZOLWDZZWV3AEYWONFCFPZE2GA, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.568753323 (14h14m6.312810176s) [noise=true , address=104.36.82.10:4077 , piece_id=XOE3BBCSJ64O7A7LC7HAAYEBF3F6CIU77UVI5HY3RI2FWSXEHJXA, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.577903595 (14h14m6.321960456s) [noise=true , address=77.90.31.14:4662 , piece_id=FDS6SHOXCIIUA4DKBYNMKY67RG6X5OLUKJLZ5M2M76UQB3ANWPLA, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.580925505 (14h14m6.324982356s) [noise=true , address=45.91.116.6:4975 , piece_id=BBULAO6EEFXHYP2AKMUF63YRSQJGVKXLIWGCKWUM5JYUVLZAB25Q, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.585723335 (14h14m6.329780193s) [noise=true , address=77.90.31.13:4628 , piece_id=ZLWT3WANPKHZ6XFV7PC3KZULY73ZEOYD3E3EVY3OMIQI3OYM3LKA, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.588082614 (14h14m6.332139464s) [noise=true , address=77.90.31.16:4735 , piece_id=OE7TL5NB6UH3A4JE2BDYFA4FZKHCRWT6YK6KJZHD6JJ55FOKXIEQ, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.626221649 (14h14m6.370278495s) [noise=true , address=45.91.116.8:4045 , piece_id=7PDSFMO4MC4SMKYW3MN6OIU33MIKCLFBSJ6LYFQAGUFMOZQCLVXQ, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.628668635 (14h14m6.372725499s) [noise=true , address=104.36.82.14:4222 , piece_id=AAQ6HQOYMMNNHXABB2MOCCJHF43PXWJZH4SDYF6ZNSLQWMKY32AA, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.636831349 (14h14m6.38088821s) [noise=true , address=77.90.31.19:4843 , piece_id=HLLIY2T4U45TYQ7L7RKY26MUTAA4KWAZ6LV7KP6HOTRJURY3WUSQ, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.645180863 (14h14m6.389237709s) [noise=true , address=45.91.116.10:4081 , piece_id=MMI5DUXMAD27VJI7VRQHBBEC2MJUDT5KWX2OGWLUPMTZLBIRWSAA, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.645221241 (14h14m6.389278101s) [seg_pos={0 0}, upload=14655]: Segment reached optimal threshold of 13 pieces. Aug 7 23:53:44.645393922 (14h14m6.389450768s) [noise=true , address=77.90.31.13:4626 , piece_id=74QG6PW6CJDJRESXUF534PQ463VRJQF33PMQRX6W6VVQCQMX2SUA, seg_pos={0 0}, upload=14655]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF Aug 7 23:53:44.645466730 (14h14m6.389523595s) [noise=true , address=45.91.116.4:4939 , piece_id=YLQB6XZJ6FL7SBJQ6FE4VIGKE5QYA3XPFU4E37OQKF5O2A7ZV4DQ, seg_pos={0 0}, upload=14655]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Aug 7 23:53:44.645496683 (14h14m6.389553539s) [noise=true , address=45.91.116.5:4299 , piece_id=F5FM3VSMCDN3BSCGDQO5XGPZPQACVJAQG4Y2HWJEPQUF7BFGWNUA, seg_pos={0 0}, upload=14655]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: expected piece hash; context canceled; EOF Aug 7 23:53:44.645513654 (14h14m6.389570515s) [seg_pos={0 0}, upload=14655]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:53:44.645605002 (14h14m6.389661865s) [upload=14655]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:53:44.645610128 (14h14m6.389666989s) [upload=14655]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:44.783671038 (14h14m6.527727892s) [upload=14655]: Done uploading segments... Aug 7 23:53:44.783719939 (14h14m6.5277768s) [upload=14655]: Upload finished. err: <nil> Aug 7 23:54:44.874516624 (14h15m6.618573476s) [upload=14672]: Starting upload Aug 7 23:54:44.874591338 (14h15m6.618648203s) [upload=14672]: Done starting upload Aug 7 23:54:44.874717241 (14h15m6.618774089s) [upload=14672]: Uploading segments... Aug 7 23:54:44.874890680 (14h15m6.61894753s) [upload=14672]: Got next segment. Inline: false Aug 7 23:54:44.874947399 (14h15m6.619004253s) [upload=14672]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:44.874953184 (14h15m6.619010035s) [upload=14672]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:55:14.597054629 (14h15m36.341111495s) [upload=14672]: Next segment err: context deadline exceeded Aug 7 23:55:14.597269046 (14h15m36.341325894s) [upload=14672]: Done uploading segments... Aug 7 23:55:14.597309239 (14h15m36.341366085s) [upload=14672]: Upload finished. err: context deadline exceeded Duration: 2.411689655s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 8 out of 60 is failed (threshold=3) [.......X...X.XX.......................................XXX..X] duration: 1m0.001565837s FAIL (2025-08-07 23:57:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== =14680]: Uploading piece... Aug 7 23:55:07.932345260 (14h15m29.676402111s) [noise=true , address=104.36.82.37:4674 , piece_id=YUF72BMFACW3E64RIROCMP3EQRL3A6CRXWQCCEPCQDHLUNUXGLYQ, seg_pos={0 0}, upload=14680]: Uploading piece... Aug 7 23:55:07.932701306 (14h15m29.676758166s) [noise=true , address=104.36.82.18:4393 , piece_id=4ARPCXISES6BDUFP2AMYT2LSDJATW54NWNJFPMKOUM7PJ4BGAN6A, seg_pos={0 0}, upload=14680]: Uploading piece... Aug 7 23:55:07.932782770 (14h15m29.676839633s) [noise=true , address=104.36.82.3:4781 , piece_id=KEPWKRTHG466LLID5E4KYOKTBJP4GQ5V533IWORMRZMDVIJ2FUFA, seg_pos={0 0}, upload=14680]: Uploading piece... Aug 7 23:55:07.932998473 (14h15m29.677055334s) [noise=true , address=104.36.82.38:4721 , piece_id=6IOH2Y34ZEPVIPO6AM5LQVXPBP4VYQZMPI2BNOTOGMQD6W3GDKVA, seg_pos={0 0}, upload=14680]: Uploading piece... Aug 7 23:55:07.933115267 (14h15m29.677172118s) [noise=true , address=79.127.201.215:4351 , piece_id=HEROPPGH3CMQ4CR2BRE6TZXDNHZREGJIIJ3JMFFI6VRINRCQG4VQ, seg_pos={0 0}, upload=14680]: Uploading piece... Aug 7 23:55:07.933318007 (14h15m29.677374867s) [noise=true , address=104.36.82.36:4619 , piece_id=LNOK6MDTPSCTWL3TL2JJKKUZ4PZVLYEWMLOTXCAN75BZCDLYE6QQ, seg_pos={0 0}, upload=14680]: Uploading piece... Aug 7 23:55:07.933483236 (14h15m29.677540099s) [noise=true , address=104.36.82.62:4417 , piece_id=6HMUER6F2LPN2KXG6ICWDSD4TCQAB5WCAXGV4FLZCLTVA23H6Y3A, seg_pos={0 0}, upload=14680]: Uploading piece... Aug 7 23:55:07.933611086 (14h15m29.677667949s) [noise=true , address=45.91.116.8:4055 , piece_id=ZBY2DCT3UBURSDD5XDMWP2D7BUJWKSQRHXP6G4MW5QY6TN7NVMQQ, seg_pos={0 0}, upload=14680]: Uploading piece... Aug 7 23:55:07.933787117 (14h15m29.677843963s) [noise=true , address=104.36.82.10:4098 , piece_id=UWSDGBAQPKXWUFKZFYMB2V7RILSFR47KY6YTA42HAA37ILY76MIQ, seg_pos={0 0}, upload=14680]: Uploading piece... Aug 7 23:55:07.933889490 (14h15m29.677946349s) [noise=true , address=193.149.164.143:4696 , piece_id=MFUOMLJTYOTYXX36TSEHVWKZRLKJGXN2L6ZA34YLXOGZ6DP4J4ZA, seg_pos={0 0}, upload=14680]: Uploading piece... Aug 7 23:55:07.934088677 (14h15m29.678145523s) [noise=true , address=104.36.82.9:4027 , piece_id=SGO6KBP2JEZBWLXKUSDIGHZYM6YFGQVABMAZYXGUE4ZWIUHPF6RA, seg_pos={0 0}, upload=14680]: Uploading piece... Aug 7 23:55:07.934234572 (14h15m29.678291435s) [noise=true , address=104.36.82.12:4157 , piece_id=SOBZMB7OFRD7H5AWLRACA4A4USTEFMYQUZTBH2AMWYAPS7WV3KZQ, seg_pos={0 0}, upload=14680]: Uploading piece... Aug 7 23:55:07.934370299 (14h15m29.678427164s) [noise=true , address=104.36.82.17:4344 , piece_id=RLCQLQ2DOPU3LZQHPXYIOFE53MUYR5BJQBUQTSO6YDFXW2QH7OAA, seg_pos={0 0}, upload=14680]: Uploading piece... Aug 7 23:55:07.934568442 (14h15m29.678625302s) [noise=true , address=104.36.82.46:4486 , piece_id=FY2HINNN2GR2N3FZC4LAAI2JHUC2S2T6QGVEZBBVZITXC36PLNHQ, seg_pos={0 0}, upload=14680]: Uploading piece... Aug 7 23:55:07.934733742 (14h15m29.678790602s) [noise=true , address=104.36.82.43:4849 , piece_id=XCGWSAXY46BZ6OBKEPGEZG23O5O6G7MXE7ZKDOJVQJTPYHC4N64Q, seg_pos={0 0}, upload=14680]: Uploading piece... Aug 7 23:55:07.934906595 (14h15m29.67896346s) [noise=true , address=104.36.82.16:4324 , piece_id=E5TG74PRUTKPSRG5N3OH5IVDG5OTKK3TTDJ57NZ6YV7ZCPEAWABQ, seg_pos={0 0}, upload=14680]: Uploading piece... Aug 7 23:55:07.936191471 (14h15m29.680248319s) [noise=true , address=104.36.82.47:4543 , piece_id=BMNUOAXVSJRIDWQH3TMSHHVQIS3NK4CYU5MBTSFO2HSFABLHXIDQ, seg_pos={0 0}, upload=14680]: Uploading piece... Aug 7 23:55:08.514337964 (14h15m30.258394823s) [upload=14680]: Next returned nil segment Aug 7 23:55:08.514356321 (14h15m30.258413185s) [upload=14680]: Waiting for error group managing segments... Aug 7 23:55:08.671755669 (14h15m30.415812521s) [noise=true , address=104.36.82.38:4721 , piece_id=6IOH2Y34ZEPVIPO6AM5LQVXPBP4VYQZMPI2BNOTOGMQD6W3GDKVA, seg_pos={0 0}, upload=14680]: Done uploading piece. err: <nil> Aug 7 23:55:08.678688910 (14h15m30.422745761s) [noise=true , address=104.36.82.46:4486 , piece_id=FY2HINNN2GR2N3FZC4LAAI2JHUC2S2T6QGVEZBBVZITXC36PLNHQ, seg_pos={0 0}, upload=14680]: Done uploading piece. err: <nil> Aug 7 23:55:08.684039396 (14h15m30.428096246s) [noise=true , address=104.36.82.61:4404 , piece_id=2ANUGQPCW55CGUY2GHSNVWDHWNCH6HIURXMECOFN7GF5ZTU2KCVQ, seg_pos={0 0}, upload=14680]: Done uploading piece. err: <nil> Aug 7 23:55:08.687477419 (14h15m30.43153427s) [noise=true , address=104.36.82.37:4674 , piece_id=YUF72BMFACW3E64RIROCMP3EQRL3A6CRXWQCCEPCQDHLUNUXGLYQ, seg_pos={0 0}, upload=14680]: Done uploading piece. err: <nil> Aug 7 23:55:08.699523124 (14h15m30.443579979s) [noise=true , address=104.36.82.9:4027 , piece_id=SGO6KBP2JEZBWLXKUSDIGHZYM6YFGQVABMAZYXGUE4ZWIUHPF6RA, seg_pos={0 0}, upload=14680]: Done uploading piece. err: <nil> Aug 7 23:55:08.705032951 (14h15m30.449089803s) [noise=true , address=104.36.82.10:4098 , piece_id=UWSDGBAQPKXWUFKZFYMB2V7RILSFR47KY6YTA42HAA37ILY76MIQ, seg_pos={0 0}, upload=14680]: Done uploading piece. err: <nil> Aug 7 23:55:08.720735560 (14h15m30.46479241s) [noise=true , address=45.91.116.8:4055 , piece_id=ZBY2DCT3UBURSDD5XDMWP2D7BUJWKSQRHXP6G4MW5QY6TN7NVMQQ, seg_pos={0 0}, upload=14680]: Done uploading piece. err: <nil> Aug 7 23:55:08.776398996 (14h15m30.520455843s) [noise=true , address=104.36.82.43:4849 , piece_id=XCGWSAXY46BZ6OBKEPGEZG23O5O6G7MXE7ZKDOJVQJTPYHC4N64Q, seg_pos={0 0}, upload=14680]: Done uploading piece. err: <nil> Aug 7 23:55:08.865380402 (14h15m30.609437248s) [noise=true , address=79.127.201.215:4351 , piece_id=HEROPPGH3CMQ4CR2BRE6TZXDNHZREGJIIJ3JMFFI6VRINRCQG4VQ, seg_pos={0 0}, upload=14680]: Done uploading piece. err: <nil> Aug 7 23:55:08.894196738 (14h15m30.638253585s) [noise=true , address=104.36.82.12:4157 , piece_id=SOBZMB7OFRD7H5AWLRACA4A4USTEFMYQUZTBH2AMWYAPS7WV3KZQ, seg_pos={0 0}, upload=14680]: Done uploading piece. err: <nil> Aug 7 23:55:08.903844740 (14h15m30.647901587s) [noise=true , address=104.36.82.17:4344 , piece_id=RLCQLQ2DOPU3LZQHPXYIOFE53MUYR5BJQBUQTSO6YDFXW2QH7OAA, seg_pos={0 0}, upload=14680]: Done uploading piece. err: <nil> Aug 7 23:55:08.943592253 (14h15m30.6876491s) [noise=true , address=104.36.82.36:4619 , piece_id=LNOK6MDTPSCTWL3TL2JJKKUZ4PZVLYEWMLOTXCAN75BZCDLYE6QQ, seg_pos={0 0}, upload=14680]: Done uploading piece. err: <nil> Aug 7 23:55:08.950728392 (14h15m30.694785242s) [noise=true , address=104.36.82.3:4781 , piece_id=KEPWKRTHG466LLID5E4KYOKTBJP4GQ5V533IWORMRZMDVIJ2FUFA, seg_pos={0 0}, upload=14680]: Done uploading piece. err: <nil> Aug 7 23:55:09.099651965 (14h15m30.843708815s) [noise=true , address=104.36.82.18:4393 , piece_id=4ARPCXISES6BDUFP2AMYT2LSDJATW54NWNJFPMKOUM7PJ4BGAN6A, seg_pos={0 0}, upload=14680]: Done uploading piece. err: <nil> Aug 7 23:55:09.099688781 (14h15m30.843745641s) [seg_pos={0 0}, upload=14680]: Segment reached optimal threshold of 14 pieces. Aug 7 23:55:09.099975619 (14h15m30.84403248s) [noise=true , address=104.36.82.47:4543 , piece_id=BMNUOAXVSJRIDWQH3TMSHHVQIS3NK4CYU5MBTSFO2HSFABLHXIDQ, seg_pos={0 0}, upload=14680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KxnBTVxhbwA79Gd8V7Mh6bWybtqttqgo5agHoHewXHAey5y7D): protocol: expected piece hash; context canceled; EOF Aug 7 23:55:09.100086986 (14h15m30.844143848s) [noise=true , address=193.149.164.143:4696 , piece_id=MFUOMLJTYOTYXX36TSEHVWKZRLKJGXN2L6ZA34YLXOGZ6DP4J4ZA, seg_pos={0 0}, upload=14680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: expected piece hash; context canceled; EOF Aug 7 23:55:09.100136402 (14h15m30.844193256s) [noise=true , address=104.36.82.62:4417 , piece_id=6HMUER6F2LPN2KXG6ICWDSD4TCQAB5WCAXGV4FLZCLTVA23H6Y3A, seg_pos={0 0}, upload=14680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127dfR4NEArFbQQDa7f7C2fWeKDzeEut9nhLogdG5aZk84FxXtL): protocol: read tcp 139.59.144.22:52154->104.36.82.62:4417: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:55:09.100179595 (14h15m30.844236456s) [noise=true , address=104.36.82.16:4324 , piece_id=E5TG74PRUTKPSRG5N3OH5IVDG5OTKK3TTDJ57NZ6YV7ZCPEAWABQ, seg_pos={0 0}, upload=14680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga): protocol: expected piece hash; context canceled; EOF Aug 7 23:55:09.100195155 (14h15m30.844252016s) [seg_pos={0 0}, upload=14680]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:55:09.100278249 (14h15m30.844335109s) [upload=14680]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:55:09.100283388 (14h15m30.844340248s) [upload=14680]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:09.252668155 (14h15m30.996725003s) [upload=14680]: Done uploading segments... Aug 7 23:55:09.252687494 (14h15m30.996744342s) [upload=14680]: Upload finished. err: <nil> Aug 7 23:56:10.697600025 (14h16m32.441656885s) [upload=14699]: Starting upload Aug 7 23:56:10.697687472 (14h16m32.441744333s) [upload=14699]: Done starting upload Aug 7 23:56:10.697956045 (14h16m32.44201291s) [upload=14699]: Uploading segments... Aug 7 23:56:10.703705277 (14h16m32.447762127s) [upload=14699]: Got next segment. Inline: false Aug 7 23:56:10.703752487 (14h16m32.447809337s) [upload=14699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:10.703758530 (14h16m32.44781539s) [upload=14699]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:57:10.395970422 (14h17m32.140027272s) [upload=14699]: Next segment err: context deadline exceeded Aug 7 23:57:10.396054113 (14h17m32.140110973s) [upload=14699]: Done uploading segments... Aug 7 23:57:10.396076384 (14h17m32.140133242s) [upload=14699]: Upload finished. err: context deadline exceeded Duration: 1m0.001582392s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 3.347269258s FAIL (2025-08-07 23:57:09 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== 294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:32:45.172127356 (13h53m6.916184241s) [upload=14294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:32:45.390149151 (13h53m7.134206011s) [seg_pos={0 0}, upload=14294]: Begin upload segment... Aug 7 23:32:45.390215910 (13h53m7.134272761s) [seg_pos={0 0}, upload=14294]: Done begin upload segment. Aug 7 23:32:45.390249193 (13h53m7.134306059s) [noise=true , address=104.36.82.19:4405 , piece_id=MHM3MWZJFTXSJT7PTOFSWHUYO3L6NIBUV6VU66XXZDH3MC3EFR2Q, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390272919 (13h53m7.134329788s) [noise=true , address=77.90.31.12:4593 , piece_id=C3RVQHFS4L7SKPFFIZ52MAQWGD7P4JNG3SNPCNY53SQUD72PZRUA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390658970 (13h53m7.13471583s) [noise=true , address=104.36.82.16:4321 , piece_id=YTF4RWV2SD6Z4UV4XRYPBRGPCTSUTC6TQKL3DHVKU65ALTNA4TEQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390852777 (13h53m7.134909659s) [noise=true , address=104.36.82.34:4566 , piece_id=3YEAUSXEL7WNZ3O7WM24Z2DKY2TDVKMRZSHMEPMNEL2LR7BY7DBQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390957991 (13h53m7.135014851s) [noise=true , address=104.36.82.47:4523 , piece_id=MJIMHSO66V7OWFFK2UALPKSI7CGGCTHNPZINDXMVIHDLENIW5LFA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.391483291 (13h53m7.135540162s) [noise=true , address=104.36.82.5:4861 , piece_id=GRDGO5LNCETQWRU3M6LACGRW5MNA2KFLXUBT5R2UH6SEGWP6RVRQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.391931094 (13h53m7.135987964s) [noise=true , address=104.36.82.10:4090 , piece_id=V6NPPVIYEH6WLIW7OMT3NDFPR7UDWIT46GW6AFCAWUJTBLC3O55Q, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392250266 (13h53m7.136307136s) [noise=true , address=104.36.82.62:4439 , piece_id=ILRD4TEFW4PSLV4EGPORG3FM6U2HB3WSL5KP3PN566FAOGWODLYQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392611689 (13h53m7.136668559s) [noise=true , address=104.36.82.54:4160 , piece_id=GDBORRYX5QBQGH7ZMMUUAV7AAR5UHZOTK5FTQUPC7YKZC2KD7I3A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392917410 (13h53m7.136974282s) [noise=true , address=104.36.82.23:4477 , piece_id=HAF6424ZCIBGSR4US5LN5ZWE5JMNMLFGUMQQ4YP2ARP7PQXBH5GQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393270536 (13h53m7.137327398s) [noise=true , address=104.36.82.44:4881 , piece_id=XXVSXG3ILFXG4IVMELHL45TRRHDFVVJMJ5LGOHU5EDHG6XXCDH6A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393582635 (13h53m7.137639498s) [noise=true , address=104.36.82.37:4665 , piece_id=RUKKGC4Y3FIVLZO2GRWJN3J6P36VWP5VSC2HCLEU3MZDDFXODJAQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393856926 (13h53m7.137913788s) [noise=true , address=104.36.82.11:4128 , piece_id=XRPULHI7BYDF2JOIASSN5QKGEAC6FJS2F5C22WIY7EKYILRHZ73A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394196364 (13h53m7.138253227s) [noise=true , address=104.36.82.38:4706 , piece_id=BIHIPQX6LRWBNFN4SDC3K3V22AMGL3RC5B4RVE7QXJFKB4XDN3BA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394484397 (13h53m7.138541257s) [noise=true , address=104.36.82.61:4381 , piece_id=TTZOSRP6YPL6P4C6PY5S4K7Z6CIZJDPX7NUNBCK73CWZPCA3HQHA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394593709 (13h53m7.138650578s) [noise=true , address=45.91.116.5:4300 , piece_id=MADJCANN2O35MKXT2PIR5PGMJ3PNTAQAJNNGOK6ISJNVAH2L7YIA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.395093013 (13h53m7.139149879s) [noise=true , address=104.36.82.22:4530 , piece_id=HMUUW5GYH6TRHYXP4YXBHD566FWQRILEHI53AVURA5QTQGG6USFA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.395447945 (13h53m7.139504805s) [noise=true , address=104.36.82.58:4293 , piece_id=I2ETFXYTHYWYJYSHTVOBQTO6ZPGNMA4D3NVKYDE2W34EH4SQE23A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.954469844 (13h53m7.698526694s) [upload=14294]: Next returned nil segment Aug 7 23:32:45.954487953 (13h53m7.69854487s) [upload=14294]: Waiting for error group managing segments... Aug 7 23:32:46.126888627 (13h53m7.870945474s) [noise=true , address=104.36.82.10:4090 , piece_id=V6NPPVIYEH6WLIW7OMT3NDFPR7UDWIT46GW6AFCAWUJTBLC3O55Q, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.132983844 (13h53m7.877040704s) [noise=true , address=104.36.82.16:4321 , piece_id=YTF4RWV2SD6Z4UV4XRYPBRGPCTSUTC6TQKL3DHVKU65ALTNA4TEQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.138277216 (13h53m7.882334067s) [noise=true , address=104.36.82.23:4477 , piece_id=HAF6424ZCIBGSR4US5LN5ZWE5JMNMLFGUMQQ4YP2ARP7PQXBH5GQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.142033326 (13h53m7.886090185s) [noise=true , address=104.36.82.34:4566 , piece_id=3YEAUSXEL7WNZ3O7WM24Z2DKY2TDVKMRZSHMEPMNEL2LR7BY7DBQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.151247206 (13h53m7.895304058s) [noise=true , address=104.36.82.22:4530 , piece_id=HMUUW5GYH6TRHYXP4YXBHD566FWQRILEHI53AVURA5QTQGG6USFA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.151981081 (13h53m7.896037932s) [noise=true , address=104.36.82.58:4293 , piece_id=I2ETFXYTHYWYJYSHTVOBQTO6ZPGNMA4D3NVKYDE2W34EH4SQE23A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.165022882 (13h53m7.909079743s) [noise=true , address=104.36.82.44:4881 , piece_id=XXVSXG3ILFXG4IVMELHL45TRRHDFVVJMJ5LGOHU5EDHG6XXCDH6A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.184418821 (13h53m7.928475667s) [noise=true , address=104.36.82.47:4523 , piece_id=MJIMHSO66V7OWFFK2UALPKSI7CGGCTHNPZINDXMVIHDLENIW5LFA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.184532669 (13h53m7.928589516s) [noise=true , address=45.91.116.5:4300 , piece_id=MADJCANN2O35MKXT2PIR5PGMJ3PNTAQAJNNGOK6ISJNVAH2L7YIA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.193375808 (13h53m7.937432656s) [noise=true , address=104.36.82.54:4160 , piece_id=GDBORRYX5QBQGH7ZMMUUAV7AAR5UHZOTK5FTQUPC7YKZC2KD7I3A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.204247501 (13h53m7.948304372s) [noise=true , address=104.36.82.38:4706 , piece_id=BIHIPQX6LRWBNFN4SDC3K3V22AMGL3RC5B4RVE7QXJFKB4XDN3BA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.235225782 (13h53m7.979282629s) [noise=true , address=104.36.82.62:4439 , piece_id=ILRD4TEFW4PSLV4EGPORG3FM6U2HB3WSL5KP3PN566FAOGWODLYQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.269129867 (13h53m8.013186721s) [noise=true , address=104.36.82.5:4861 , piece_id=GRDGO5LNCETQWRU3M6LACGRW5MNA2KFLXUBT5R2UH6SEGWP6RVRQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.278216912 (13h53m8.022273764s) [noise=true , address=104.36.82.19:4405 , piece_id=MHM3MWZJFTXSJT7PTOFSWHUYO3L6NIBUV6VU66XXZDH3MC3EFR2Q, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.278247847 (13h53m8.022304708s) [seg_pos={0 0}, upload=14294]: Segment reached optimal threshold of 14 pieces. Aug 7 23:32:46.278485612 (13h53m8.022542463s) [noise=true , address=104.36.82.11:4128 , piece_id=XRPULHI7BYDF2JOIASSN5QKGEAC6FJS2F5C22WIY7EKYILRHZ73A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wABkAtAjoYkKyNoAiaVCxSvY6WgvHvAYFPCjpNojtpiYiGtDS): protocol: expected piece hash; context canceled; EOF Aug 7 23:32:46.278511698 (13h53m8.022568566s) [noise=true , address=77.90.31.12:4593 , piece_id=C3RVQHFS4L7SKPFFIZ52MAQWGD7P4JNG3SNPCNY53SQUD72PZRUA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kSxoZb9nmG6oGNytpFTsoiGh3ZPmebVVe8TZxGzANaw7EfdeF): protocol: expected piece hash; context canceled; EOF Aug 7 23:32:46.278642807 (13h53m8.022699657s) [noise=true , address=104.36.82.61:4381 , piece_id=TTZOSRP6YPL6P4C6PY5S4K7Z6CIZJDPX7NUNBCK73CWZPCA3HQHA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zU35MWWaaC1npkP5qGcWBXAkfxGsd5UsdkM2nwX9x7wUGCDbY): protocol: write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection; write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection Aug 7 23:32:46.278650926 (13h53m8.022707786s) [noise=true , address=104.36.82.37:4665 , piece_id=RUKKGC4Y3FIVLZO2GRWJN3J6P36VWP5VSC2HCLEU3MZDDFXODJAQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jP9JrCmoQSdqrzXZxkNN1ihdFkaUMjGZss6Fh19zbL3YyWj5N): protocol: write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection; write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection Aug 7 23:32:46.278674050 (13h53m8.022730911s) [seg_pos={0 0}, upload=14294]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:32:46.278751154 (13h53m8.022808014s) [upload=14294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:32:46.278756168 (13h53m8.022813029s) [upload=14294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:32:46.498660020 (13h53m8.242716882s) [upload=14294]: Done uploading segments... Aug 7 23:32:46.498701263 (13h53m8.242758128s) [upload=14294]: Upload finished. err: <nil> Duration: 3.347303597s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 11 out of 60 is failed (threshold=3) [.X.XXXXX.X............................................XXX.X.] duration: 17.698428802s FAIL (2025-08-07 23:57:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 17.698452576s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 838.207446ms FAIL (2025-08-07 23:57:04 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== pload=14121]: Done uploading segments... Aug 7 23:22:28.505845197 (13h42m50.249902044s) [upload=14121]: Upload finished. err: <nil> Aug 7 23:23:10.377483605 (13h43m32.121540458s) [upload=14132]: Starting upload Aug 7 23:23:10.377535635 (13h43m32.121592495s) [upload=14132]: Done starting upload Aug 7 23:23:10.377603797 (13h43m32.121660647s) [upload=14132]: Uploading segments... Aug 7 23:23:10.377638935 (13h43m32.121695804s) [upload=14132]: Got next segment. Inline: true Aug 7 23:23:10.377643594 (13h43m32.121700454s) [upload=14132]: Waiting for error group managing segments... Aug 7 23:23:10.377688252 (13h43m32.121745104s) [upload=14132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:23:10.377693427 (13h43m32.121750286s) [upload=14132]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:23:10.377696805 (13h43m32.121753666s) [upload=14132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:23:10.562260724 (13h43m32.306317573s) [upload=14132]: Done uploading segments... Aug 7 23:23:10.562279228 (13h43m32.306336078s) [upload=14132]: Upload finished. err: <nil> Aug 7 23:24:07.896404858 (13h44m29.640461721s) [upload=14147]: Starting upload Aug 7 23:24:07.896490216 (13h44m29.640547079s) [upload=14147]: Done starting upload Aug 7 23:24:07.896580488 (13h44m29.640637339s) [upload=14147]: Uploading segments... Aug 7 23:24:07.896655006 (13h44m29.640711874s) [upload=14147]: Got next segment. Inline: true Aug 7 23:24:07.896660179 (13h44m29.640717039s) [upload=14147]: Waiting for error group managing segments... Aug 7 23:24:07.896714840 (13h44m29.6407717s) [upload=14147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:24:07.896719399 (13h44m29.64077626s) [upload=14147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:24:07.896722726 (13h44m29.640779586s) [upload=14147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:24:08.048898130 (13h44m29.792954991s) [upload=14147]: Done uploading segments... Aug 7 23:24:08.048952526 (13h44m29.793009386s) [upload=14147]: Upload finished. err: <nil> Aug 7 23:24:53.794754379 (13h45m15.53881125s) [upload=14162]: Starting upload Aug 7 23:24:53.794842317 (13h45m15.538899177s) [upload=14162]: Done starting upload Aug 7 23:24:53.794935964 (13h45m15.538992814s) [upload=14162]: Uploading segments... Aug 7 23:24:53.795003752 (13h45m15.539060623s) [upload=14162]: Got next segment. Inline: true Aug 7 23:24:53.795010814 (13h45m15.539067676s) [upload=14162]: Waiting for error group managing segments... Aug 7 23:24:53.795066607 (13h45m15.539123488s) [upload=14162]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:24:53.795158449 (13h45m15.5392153s) [upload=14162]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:24:53.795163166 (13h45m15.539220026s) [upload=14162]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:24:53.946994982 (13h45m15.691051833s) [upload=14162]: Done uploading segments... Aug 7 23:24:53.947013862 (13h45m15.691070711s) [upload=14162]: Upload finished. err: <nil> Aug 7 23:25:40.029856966 (13h46m1.773913836s) [upload=14177]: Starting upload Aug 7 23:25:40.029986938 (13h46m1.774043809s) [upload=14177]: Done starting upload Aug 7 23:25:40.030076796 (13h46m1.774133642s) [upload=14177]: Uploading segments... Aug 7 23:25:40.030132567 (13h46m1.774189438s) [upload=14177]: Got next segment. Inline: true Aug 7 23:25:40.030139885 (13h46m1.774196757s) [upload=14177]: Waiting for error group managing segments... Aug 7 23:25:40.030197588 (13h46m1.774254461s) [upload=14177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:25:40.030204146 (13h46m1.774261016s) [upload=14177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:25:40.030207984 (13h46m1.77426485s) [upload=14177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:25:40.184198742 (13h46m1.928255592s) [upload=14177]: Done uploading segments... Aug 7 23:25:40.184219017 (13h46m1.928275867s) [upload=14177]: Upload finished. err: <nil> Aug 7 23:26:54.625649110 (13h47m16.369705971s) [upload=14198]: Starting upload Aug 7 23:26:54.625738563 (13h47m16.369795424s) [upload=14198]: Done starting upload Aug 7 23:26:54.625842691 (13h47m16.369899539s) [upload=14198]: Uploading segments... Aug 7 23:26:54.626016839 (13h47m16.37007371s) [upload=14198]: Got next segment. Inline: true Aug 7 23:26:54.626052112 (13h47m16.370108973s) [upload=14198]: Waiting for error group managing segments... Aug 7 23:26:54.626129780 (13h47m16.370186645s) [upload=14198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:26:54.626137392 (13h47m16.370194264s) [upload=14198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:26:54.626143182 (13h47m16.370200048s) [upload=14198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:26:54.775804171 (13h47m16.519861021s) [upload=14198]: Done uploading segments... Aug 7 23:26:54.775824249 (13h47m16.519881102s) [upload=14198]: Upload finished. err: <nil> Aug 7 23:27:32.263746490 (13h47m54.007803361s) [upload=14208]: Starting upload Aug 7 23:27:32.263833655 (13h47m54.007890501s) [upload=14208]: Done starting upload Aug 7 23:27:32.263976536 (13h47m54.008033385s) [upload=14208]: Uploading segments... Aug 7 23:27:32.264070926 (13h47m54.008127796s) [upload=14208]: Got next segment. Inline: true Aug 7 23:27:32.264080356 (13h47m54.00813733s) [upload=14208]: Waiting for error group managing segments... Aug 7 23:27:32.264154657 (13h47m54.008211526s) [upload=14208]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:27:32.264163587 (13h47m54.008220454s) [upload=14208]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:27:32.264172017 (13h47m54.008228914s) [upload=14208]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:27:32.490188559 (13h47m54.23424541s) [upload=14208]: Done uploading segments... Aug 7 23:27:32.490207326 (13h47m54.234264172s) [upload=14208]: Upload finished. err: <nil> Aug 7 23:27:57.927940477 (13h48m19.671997323s) [upload=14217]: Starting upload Aug 7 23:27:57.928016423 (13h48m19.672073283s) [upload=14217]: Done starting upload Aug 7 23:27:57.928101290 (13h48m19.672158142s) [upload=14217]: Uploading segments... Aug 7 23:27:57.928155692 (13h48m19.672212566s) [upload=14217]: Got next segment. Inline: true Aug 7 23:27:57.928162009 (13h48m19.672218869s) [upload=14217]: Waiting for error group managing segments... Aug 7 23:27:57.928213294 (13h48m19.672270145s) [upload=14217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:27:57.928219212 (13h48m19.672276074s) [upload=14217]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:27:57.928223521 (13h48m19.672280383s) [upload=14217]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:27:58.087544005 (13h48m19.831600874s) [upload=14217]: Done uploading segments... Aug 7 23:27:58.087570226 (13h48m19.831627094s) [upload=14217]: Upload finished. err: <nil> Aug 7 23:29:12.765200935 (13h49m34.509257795s) [upload=14237]: Starting upload Aug 7 23:29:12.765253106 (13h49m34.509309977s) [upload=14237]: Done starting upload Aug 7 23:29:12.765324641 (13h49m34.509381499s) [upload=14237]: Uploading segments... Aug 7 23:29:12.765374187 (13h49m34.509431056s) [upload=14237]: Got next segment. Inline: true Aug 7 23:29:12.765378947 (13h49m34.509435807s) [upload=14237]: Waiting for error group managing segments... Aug 7 23:29:12.765430859 (13h49m34.50948772s) [upload=14237]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:29:12.765438106 (13h49m34.50949497s) [upload=14237]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:29:12.765443127 (13h49m34.509499988s) [upload=14237]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:29:12.926437674 (13h49m34.670494535s) [upload=14237]: Done uploading segments... Aug 7 23:29:12.926457418 (13h49m34.670514268s) [upload=14237]: Upload finished. err: <nil> Aug 7 23:29:42.225760316 (13h50m3.969817177s) [upload=14244]: Starting upload Aug 7 23:29:42.225855063 (13h50m3.969911929s) [upload=14244]: Done starting upload Aug 7 23:29:42.225962489 (13h50m3.970019355s) [upload=14244]: Uploading segments... Aug 7 23:29:42.226078484 (13h50m3.970135331s) [upload=14244]: Got next segment. Inline: true Aug 7 23:29:42.226083996 (13h50m3.970140842s) [upload=14244]: Waiting for error group managing segments... Aug 7 23:29:42.226143067 (13h50m3.970199914s) [upload=14244]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:29:42.226148511 (13h50m3.970205371s) [upload=14244]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:29:42.226153006 (13h50m3.970209865s) [upload=14244]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:29:42.401822947 (13h50m4.14587981s) [upload=14244]: Done uploading segments... Aug 7 23:29:42.401847556 (13h50m4.145904405s) [upload=14244]: Upload finished. err: <nil> Aug 7 23:30:31.529840301 (13h50m53.273897165s) [upload=14254]: Starting upload Aug 7 23:30:31.530002531 (13h50m53.274059378s) [upload=14254]: Done starting upload Aug 7 23:30:31.530201544 (13h50m53.274258391s) [upload=14254]: Uploading segments... Aug 7 23:30:31.530298120 (13h50m53.274354985s) [upload=14254]: Got next segment. Inline: true Aug 7 23:30:31.530303457 (13h50m53.274360317s) [upload=14254]: Waiting for error group managing segments... Aug 7 23:30:31.530353721 (13h50m53.274410567s) [upload=14254]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:30:31.530358927 (13h50m53.274415788s) [upload=14254]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:30:31.530363758 (13h50m53.274420619s) [upload=14254]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:30:31.696350967 (13h50m53.440407828s) [upload=14254]: Done uploading segments... Aug 7 23:30:31.696369460 (13h50m53.440426307s) [upload=14254]: Upload finished. err: <nil> Duration: 838.320823ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 15 out of 60 is failed (threshold=3) [XX.XXXXXX..XX.X...................................X...XX.X..] duration: 865.642581ms FAIL (2025-08-07 23:56:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ad finished. err: <nil> Aug 7 23:49:47.788743937 (14h10m9.532800783s) [upload=14591]: Starting upload Aug 7 23:49:47.788822734 (14h10m9.5328796s) [upload=14591]: Done starting upload Aug 7 23:49:47.788889290 (14h10m9.532946138s) [upload=14591]: Uploading segments... Aug 7 23:49:47.788950938 (14h10m9.533007798s) [upload=14591]: Got next segment. Inline: true Aug 7 23:49:47.788955912 (14h10m9.533012775s) [upload=14591]: Waiting for error group managing segments... Aug 7 23:49:47.789005080 (14h10m9.533061926s) [upload=14591]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:49:47.789009334 (14h10m9.533066194s) [upload=14591]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:49:47.789014285 (14h10m9.53307113s) [upload=14591]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:49:47.932333015 (14h10m9.676389863s) [upload=14591]: Done uploading segments... Aug 7 23:49:47.932362673 (14h10m9.676419519s) [upload=14591]: Upload finished. err: <nil> Aug 7 23:50:23.531847225 (14h10m45.275904072s) [upload=14601]: Starting upload Aug 7 23:50:23.531926184 (14h10m45.275983047s) [upload=14601]: Done starting upload Aug 7 23:50:23.531993110 (14h10m45.27604996s) [upload=14601]: Uploading segments... Aug 7 23:50:23.532035359 (14h10m45.276092227s) [upload=14601]: Got next segment. Inline: true Aug 7 23:50:23.532041373 (14h10m45.276098223s) [upload=14601]: Waiting for error group managing segments... Aug 7 23:50:23.532083061 (14h10m45.276139912s) [upload=14601]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:23.532088436 (14h10m45.276145285s) [upload=14601]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:23.532091688 (14h10m45.276148539s) [upload=14601]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:23.673405160 (14h10m45.417462011s) [upload=14601]: Done uploading segments... Aug 7 23:50:23.673423409 (14h10m45.417480259s) [upload=14601]: Upload finished. err: <nil> Aug 7 23:51:32.204614635 (14h11m53.948671502s) [upload=14620]: Starting upload Aug 7 23:51:32.204700450 (14h11m53.948757313s) [upload=14620]: Done starting upload Aug 7 23:51:32.204781575 (14h11m53.948838427s) [upload=14620]: Uploading segments... Aug 7 23:51:32.204839741 (14h11m53.94889661s) [upload=14620]: Got next segment. Inline: true Aug 7 23:51:32.204847624 (14h11m53.948904474s) [upload=14620]: Waiting for error group managing segments... Aug 7 23:51:32.204902472 (14h11m53.948959327s) [upload=14620]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:51:32.204907775 (14h11m53.948964625s) [upload=14620]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:51:32.204911177 (14h11m53.948968027s) [upload=14620]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:01.937570352 (14h12m23.681627223s) [upload=14620]: Done uploading segments... Aug 7 23:52:01.937615027 (14h12m23.681671888s) [upload=14620]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:52:03.484443714 (14h12m25.228500575s) [upload=14625]: Starting upload Aug 7 23:52:03.484557174 (14h12m25.228614037s) [upload=14625]: Done starting upload Aug 7 23:52:03.484649660 (14h12m25.22870651s) [upload=14625]: Uploading segments... Aug 7 23:52:03.484705970 (14h12m25.22876283s) [upload=14625]: Got next segment. Inline: true Aug 7 23:52:03.484711540 (14h12m25.228768402s) [upload=14625]: Waiting for error group managing segments... Aug 7 23:52:03.484758149 (14h12m25.228815009s) [upload=14625]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:03.484763263 (14h12m25.228820122s) [upload=14625]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:03.484766746 (14h12m25.228823606s) [upload=14625]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:03.643631412 (14h12m25.387688267s) [upload=14625]: Done uploading segments... Aug 7 23:52:03.643670624 (14h12m25.387727482s) [upload=14625]: Upload finished. err: <nil> Aug 7 23:52:25.641252610 (14h12m47.385309456s) [upload=14634]: Starting upload Aug 7 23:52:25.641347708 (14h12m47.385404571s) [upload=14634]: Done starting upload Aug 7 23:52:25.641412738 (14h12m47.385469584s) [upload=14634]: Uploading segments... Aug 7 23:52:25.641487429 (14h12m47.385544276s) [upload=14634]: Got next segment. Inline: true Aug 7 23:52:25.641494756 (14h12m47.385551602s) [upload=14634]: Waiting for error group managing segments... Aug 7 23:52:25.641535502 (14h12m47.385592365s) [upload=14634]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:25.641556088 (14h12m47.385612953s) [upload=14634]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:25.641559364 (14h12m47.385616224s) [upload=14634]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:25.852337418 (14h12m47.596394279s) [upload=14634]: Done uploading segments... Aug 7 23:52:25.852355736 (14h12m47.596412584s) [upload=14634]: Upload finished. err: <nil> Aug 7 23:52:51.292140355 (14h13m13.036197215s) [upload=14641]: Starting upload Aug 7 23:52:51.292216133 (14h13m13.036272985s) [upload=14641]: Done starting upload Aug 7 23:52:51.292307446 (14h13m13.036364307s) [upload=14641]: Uploading segments... Aug 7 23:52:51.292389087 (14h13m13.036445956s) [upload=14641]: Got next segment. Inline: true Aug 7 23:52:51.292394933 (14h13m13.036451793s) [upload=14641]: Waiting for error group managing segments... Aug 7 23:52:51.292456032 (14h13m13.036512883s) [upload=14641]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:51.292461787 (14h13m13.036518647s) [upload=14641]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:51.292467676 (14h13m13.036524537s) [upload=14641]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:51.436915828 (14h13m13.180972678s) [upload=14641]: Done uploading segments... Aug 7 23:52:51.436934458 (14h13m13.18099131s) [upload=14641]: Upload finished. err: <nil> Aug 7 23:53:10.946261153 (14h13m32.690318028s) [upload=14646]: Starting upload Aug 7 23:53:10.946350198 (14h13m32.690407065s) [upload=14646]: Done starting upload Aug 7 23:53:10.946435423 (14h13m32.690492276s) [upload=14646]: Uploading segments... Aug 7 23:53:10.946489255 (14h13m32.690546124s) [upload=14646]: Got next segment. Inline: true Aug 7 23:53:10.946497660 (14h13m32.690554519s) [upload=14646]: Waiting for error group managing segments... Aug 7 23:53:10.946542678 (14h13m32.690599529s) [upload=14646]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:10.946548267 (14h13m32.690605128s) [upload=14646]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:10.946553792 (14h13m32.690610652s) [upload=14646]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:14.554651436 (14h13m36.298708309s) [upload=14646]: Done uploading segments... Aug 7 23:53:14.554675183 (14h13m36.298732045s) [upload=14646]: Upload finished. err: <nil> Aug 7 23:53:53.584502029 (14h14m15.328558897s) [upload=14658]: Starting upload Aug 7 23:53:53.584599757 (14h14m15.328656616s) [upload=14658]: Done starting upload Aug 7 23:53:53.584702364 (14h14m15.328759218s) [upload=14658]: Uploading segments... Aug 7 23:53:53.584798610 (14h14m15.32885548s) [upload=14658]: Got next segment. Inline: true Aug 7 23:53:53.584806594 (14h14m15.328863461s) [upload=14658]: Waiting for error group managing segments... Aug 7 23:53:53.584875193 (14h14m15.328932062s) [upload=14658]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:53.584882324 (14h14m15.328939193s) [upload=14658]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:53.584901657 (14h14m15.328958528s) [upload=14658]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:23.318821682 (14h14m45.062878542s) [upload=14658]: Done uploading segments... Aug 7 23:54:23.318863033 (14h14m45.062919884s) [upload=14658]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:36.775209359 (14h14m58.519266209s) [upload=14671]: Starting upload Aug 7 23:54:36.775292235 (14h14m58.519349112s) [upload=14671]: Done starting upload Aug 7 23:54:36.775412031 (14h14m58.519468883s) [upload=14671]: Uploading segments... Aug 7 23:54:36.775480767 (14h14m58.519537636s) [upload=14671]: Got next segment. Inline: true Aug 7 23:54:36.775487320 (14h14m58.519544181s) [upload=14671]: Waiting for error group managing segments... Aug 7 23:54:36.775539752 (14h14m58.519596612s) [upload=14671]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:36.775545893 (14h14m58.519602753s) [upload=14671]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:36.775549619 (14h14m58.519606479s) [upload=14671]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:36.926276427 (14h14m58.670333277s) [upload=14671]: Done uploading segments... Aug 7 23:54:36.926295206 (14h14m58.670352057s) [upload=14671]: Upload finished. err: <nil> Aug 7 23:54:57.675347843 (14h15m19.419404694s) [upload=14676]: Starting upload Aug 7 23:54:57.675412352 (14h15m19.419469212s) [upload=14676]: Done starting upload Aug 7 23:54:57.675475670 (14h15m19.41953252s) [upload=14676]: Uploading segments... Aug 7 23:54:57.675516336 (14h15m19.419573205s) [upload=14676]: Got next segment. Inline: true Aug 7 23:54:57.675521071 (14h15m19.419577931s) [upload=14676]: Waiting for error group managing segments... Aug 7 23:54:57.675565512 (14h15m19.419622375s) [upload=14676]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:57.675574255 (14h15m19.419631202s) [upload=14676]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:57.675580694 (14h15m19.419637563s) [upload=14676]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:27.407552485 (14h15m49.151609344s) [upload=14676]: Done uploading segments... Aug 7 23:55:27.407592978 (14h15m49.151649829s) [upload=14676]: Upload finished. err: metaclient: context deadline exceeded Duration: 865.678715ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 12 out of 60 is failed (threshold=3) [...XXXXXX.XX........................................X.XX...X] duration: 30.000946893s FAIL (2025-08-07 23:56:57 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 30.00096961s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 14 out of 60 is failed (threshold=3) [.XXXXX.XX.XX.X.........................................X.XXX] duration: 30.001191475s FAIL (2025-08-07 23:56:55 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.001210853s core uplink us1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 391.450546ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 23:56:53 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 391.477529ms edge gateway vhost us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 11 out of 60 is failed (threshold=3) [..X.XXXXX.X............................................XXX.X] duration: 1m0.001991794s FAIL (2025-08-07 23:56:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.002016314s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 113.459999ms remote: "tcp:185.244.226.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 23:56:40 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 113.48311ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 985.453096ms FAIL (2025-08-07 23:56:38 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== pload=14121]: Done uploading segments... Aug 7 23:22:28.505845197 (13h42m50.249902044s) [upload=14121]: Upload finished. err: <nil> Aug 7 23:23:10.377483605 (13h43m32.121540458s) [upload=14132]: Starting upload Aug 7 23:23:10.377535635 (13h43m32.121592495s) [upload=14132]: Done starting upload Aug 7 23:23:10.377603797 (13h43m32.121660647s) [upload=14132]: Uploading segments... Aug 7 23:23:10.377638935 (13h43m32.121695804s) [upload=14132]: Got next segment. Inline: true Aug 7 23:23:10.377643594 (13h43m32.121700454s) [upload=14132]: Waiting for error group managing segments... Aug 7 23:23:10.377688252 (13h43m32.121745104s) [upload=14132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:23:10.377693427 (13h43m32.121750286s) [upload=14132]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:23:10.377696805 (13h43m32.121753666s) [upload=14132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:23:10.562260724 (13h43m32.306317573s) [upload=14132]: Done uploading segments... Aug 7 23:23:10.562279228 (13h43m32.306336078s) [upload=14132]: Upload finished. err: <nil> Aug 7 23:24:07.896404858 (13h44m29.640461721s) [upload=14147]: Starting upload Aug 7 23:24:07.896490216 (13h44m29.640547079s) [upload=14147]: Done starting upload Aug 7 23:24:07.896580488 (13h44m29.640637339s) [upload=14147]: Uploading segments... Aug 7 23:24:07.896655006 (13h44m29.640711874s) [upload=14147]: Got next segment. Inline: true Aug 7 23:24:07.896660179 (13h44m29.640717039s) [upload=14147]: Waiting for error group managing segments... Aug 7 23:24:07.896714840 (13h44m29.6407717s) [upload=14147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:24:07.896719399 (13h44m29.64077626s) [upload=14147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:24:07.896722726 (13h44m29.640779586s) [upload=14147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:24:08.048898130 (13h44m29.792954991s) [upload=14147]: Done uploading segments... Aug 7 23:24:08.048952526 (13h44m29.793009386s) [upload=14147]: Upload finished. err: <nil> Aug 7 23:24:53.794754379 (13h45m15.53881125s) [upload=14162]: Starting upload Aug 7 23:24:53.794842317 (13h45m15.538899177s) [upload=14162]: Done starting upload Aug 7 23:24:53.794935964 (13h45m15.538992814s) [upload=14162]: Uploading segments... Aug 7 23:24:53.795003752 (13h45m15.539060623s) [upload=14162]: Got next segment. Inline: true Aug 7 23:24:53.795010814 (13h45m15.539067676s) [upload=14162]: Waiting for error group managing segments... Aug 7 23:24:53.795066607 (13h45m15.539123488s) [upload=14162]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:24:53.795158449 (13h45m15.5392153s) [upload=14162]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:24:53.795163166 (13h45m15.539220026s) [upload=14162]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:24:53.946994982 (13h45m15.691051833s) [upload=14162]: Done uploading segments... Aug 7 23:24:53.947013862 (13h45m15.691070711s) [upload=14162]: Upload finished. err: <nil> Aug 7 23:25:40.029856966 (13h46m1.773913836s) [upload=14177]: Starting upload Aug 7 23:25:40.029986938 (13h46m1.774043809s) [upload=14177]: Done starting upload Aug 7 23:25:40.030076796 (13h46m1.774133642s) [upload=14177]: Uploading segments... Aug 7 23:25:40.030132567 (13h46m1.774189438s) [upload=14177]: Got next segment. Inline: true Aug 7 23:25:40.030139885 (13h46m1.774196757s) [upload=14177]: Waiting for error group managing segments... Aug 7 23:25:40.030197588 (13h46m1.774254461s) [upload=14177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:25:40.030204146 (13h46m1.774261016s) [upload=14177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:25:40.030207984 (13h46m1.77426485s) [upload=14177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:25:40.184198742 (13h46m1.928255592s) [upload=14177]: Done uploading segments... Aug 7 23:25:40.184219017 (13h46m1.928275867s) [upload=14177]: Upload finished. err: <nil> Aug 7 23:26:54.625649110 (13h47m16.369705971s) [upload=14198]: Starting upload Aug 7 23:26:54.625738563 (13h47m16.369795424s) [upload=14198]: Done starting upload Aug 7 23:26:54.625842691 (13h47m16.369899539s) [upload=14198]: Uploading segments... Aug 7 23:26:54.626016839 (13h47m16.37007371s) [upload=14198]: Got next segment. Inline: true Aug 7 23:26:54.626052112 (13h47m16.370108973s) [upload=14198]: Waiting for error group managing segments... Aug 7 23:26:54.626129780 (13h47m16.370186645s) [upload=14198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:26:54.626137392 (13h47m16.370194264s) [upload=14198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:26:54.626143182 (13h47m16.370200048s) [upload=14198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:26:54.775804171 (13h47m16.519861021s) [upload=14198]: Done uploading segments... Aug 7 23:26:54.775824249 (13h47m16.519881102s) [upload=14198]: Upload finished. err: <nil> Aug 7 23:27:32.263746490 (13h47m54.007803361s) [upload=14208]: Starting upload Aug 7 23:27:32.263833655 (13h47m54.007890501s) [upload=14208]: Done starting upload Aug 7 23:27:32.263976536 (13h47m54.008033385s) [upload=14208]: Uploading segments... Aug 7 23:27:32.264070926 (13h47m54.008127796s) [upload=14208]: Got next segment. Inline: true Aug 7 23:27:32.264080356 (13h47m54.00813733s) [upload=14208]: Waiting for error group managing segments... Aug 7 23:27:32.264154657 (13h47m54.008211526s) [upload=14208]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:27:32.264163587 (13h47m54.008220454s) [upload=14208]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:27:32.264172017 (13h47m54.008228914s) [upload=14208]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:27:32.490188559 (13h47m54.23424541s) [upload=14208]: Done uploading segments... Aug 7 23:27:32.490207326 (13h47m54.234264172s) [upload=14208]: Upload finished. err: <nil> Aug 7 23:27:57.927940477 (13h48m19.671997323s) [upload=14217]: Starting upload Aug 7 23:27:57.928016423 (13h48m19.672073283s) [upload=14217]: Done starting upload Aug 7 23:27:57.928101290 (13h48m19.672158142s) [upload=14217]: Uploading segments... Aug 7 23:27:57.928155692 (13h48m19.672212566s) [upload=14217]: Got next segment. Inline: true Aug 7 23:27:57.928162009 (13h48m19.672218869s) [upload=14217]: Waiting for error group managing segments... Aug 7 23:27:57.928213294 (13h48m19.672270145s) [upload=14217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:27:57.928219212 (13h48m19.672276074s) [upload=14217]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:27:57.928223521 (13h48m19.672280383s) [upload=14217]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:27:58.087544005 (13h48m19.831600874s) [upload=14217]: Done uploading segments... Aug 7 23:27:58.087570226 (13h48m19.831627094s) [upload=14217]: Upload finished. err: <nil> Aug 7 23:29:12.765200935 (13h49m34.509257795s) [upload=14237]: Starting upload Aug 7 23:29:12.765253106 (13h49m34.509309977s) [upload=14237]: Done starting upload Aug 7 23:29:12.765324641 (13h49m34.509381499s) [upload=14237]: Uploading segments... Aug 7 23:29:12.765374187 (13h49m34.509431056s) [upload=14237]: Got next segment. Inline: true Aug 7 23:29:12.765378947 (13h49m34.509435807s) [upload=14237]: Waiting for error group managing segments... Aug 7 23:29:12.765430859 (13h49m34.50948772s) [upload=14237]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:29:12.765438106 (13h49m34.50949497s) [upload=14237]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:29:12.765443127 (13h49m34.509499988s) [upload=14237]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:29:12.926437674 (13h49m34.670494535s) [upload=14237]: Done uploading segments... Aug 7 23:29:12.926457418 (13h49m34.670514268s) [upload=14237]: Upload finished. err: <nil> Aug 7 23:29:42.225760316 (13h50m3.969817177s) [upload=14244]: Starting upload Aug 7 23:29:42.225855063 (13h50m3.969911929s) [upload=14244]: Done starting upload Aug 7 23:29:42.225962489 (13h50m3.970019355s) [upload=14244]: Uploading segments... Aug 7 23:29:42.226078484 (13h50m3.970135331s) [upload=14244]: Got next segment. Inline: true Aug 7 23:29:42.226083996 (13h50m3.970140842s) [upload=14244]: Waiting for error group managing segments... Aug 7 23:29:42.226143067 (13h50m3.970199914s) [upload=14244]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:29:42.226148511 (13h50m3.970205371s) [upload=14244]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:29:42.226153006 (13h50m3.970209865s) [upload=14244]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:29:42.401822947 (13h50m4.14587981s) [upload=14244]: Done uploading segments... Aug 7 23:29:42.401847556 (13h50m4.145904405s) [upload=14244]: Upload finished. err: <nil> Aug 7 23:30:31.529840301 (13h50m53.273897165s) [upload=14254]: Starting upload Aug 7 23:30:31.530002531 (13h50m53.274059378s) [upload=14254]: Done starting upload Aug 7 23:30:31.530201544 (13h50m53.274258391s) [upload=14254]: Uploading segments... Aug 7 23:30:31.530298120 (13h50m53.274354985s) [upload=14254]: Got next segment. Inline: true Aug 7 23:30:31.530303457 (13h50m53.274360317s) [upload=14254]: Waiting for error group managing segments... Aug 7 23:30:31.530353721 (13h50m53.274410567s) [upload=14254]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:30:31.530358927 (13h50m53.274415788s) [upload=14254]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:30:31.530363758 (13h50m53.274420619s) [upload=14254]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:30:31.696350967 (13h50m53.440407828s) [upload=14254]: Done uploading segments... Aug 7 23:30:31.696369460 (13h50m53.440426307s) [upload=14254]: Upload finished. err: <nil> Duration: 985.479446ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 13 out of 60 is failed (threshold=3) [..XXXXX.XX.XX.X.........................................X.XX] duration: 30.001079331s FAIL (2025-08-07 23:56:25 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.001143302s core uplink us1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 314.929088ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 23:56:20 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 315.026776ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 3.036839428s FAIL (2025-08-07 23:56:17 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== 294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:32:45.172127356 (13h53m6.916184241s) [upload=14294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:32:45.390149151 (13h53m7.134206011s) [seg_pos={0 0}, upload=14294]: Begin upload segment... Aug 7 23:32:45.390215910 (13h53m7.134272761s) [seg_pos={0 0}, upload=14294]: Done begin upload segment. Aug 7 23:32:45.390249193 (13h53m7.134306059s) [noise=true , address=104.36.82.19:4405 , piece_id=MHM3MWZJFTXSJT7PTOFSWHUYO3L6NIBUV6VU66XXZDH3MC3EFR2Q, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390272919 (13h53m7.134329788s) [noise=true , address=77.90.31.12:4593 , piece_id=C3RVQHFS4L7SKPFFIZ52MAQWGD7P4JNG3SNPCNY53SQUD72PZRUA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390658970 (13h53m7.13471583s) [noise=true , address=104.36.82.16:4321 , piece_id=YTF4RWV2SD6Z4UV4XRYPBRGPCTSUTC6TQKL3DHVKU65ALTNA4TEQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390852777 (13h53m7.134909659s) [noise=true , address=104.36.82.34:4566 , piece_id=3YEAUSXEL7WNZ3O7WM24Z2DKY2TDVKMRZSHMEPMNEL2LR7BY7DBQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390957991 (13h53m7.135014851s) [noise=true , address=104.36.82.47:4523 , piece_id=MJIMHSO66V7OWFFK2UALPKSI7CGGCTHNPZINDXMVIHDLENIW5LFA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.391483291 (13h53m7.135540162s) [noise=true , address=104.36.82.5:4861 , piece_id=GRDGO5LNCETQWRU3M6LACGRW5MNA2KFLXUBT5R2UH6SEGWP6RVRQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.391931094 (13h53m7.135987964s) [noise=true , address=104.36.82.10:4090 , piece_id=V6NPPVIYEH6WLIW7OMT3NDFPR7UDWIT46GW6AFCAWUJTBLC3O55Q, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392250266 (13h53m7.136307136s) [noise=true , address=104.36.82.62:4439 , piece_id=ILRD4TEFW4PSLV4EGPORG3FM6U2HB3WSL5KP3PN566FAOGWODLYQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392611689 (13h53m7.136668559s) [noise=true , address=104.36.82.54:4160 , piece_id=GDBORRYX5QBQGH7ZMMUUAV7AAR5UHZOTK5FTQUPC7YKZC2KD7I3A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392917410 (13h53m7.136974282s) [noise=true , address=104.36.82.23:4477 , piece_id=HAF6424ZCIBGSR4US5LN5ZWE5JMNMLFGUMQQ4YP2ARP7PQXBH5GQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393270536 (13h53m7.137327398s) [noise=true , address=104.36.82.44:4881 , piece_id=XXVSXG3ILFXG4IVMELHL45TRRHDFVVJMJ5LGOHU5EDHG6XXCDH6A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393582635 (13h53m7.137639498s) [noise=true , address=104.36.82.37:4665 , piece_id=RUKKGC4Y3FIVLZO2GRWJN3J6P36VWP5VSC2HCLEU3MZDDFXODJAQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393856926 (13h53m7.137913788s) [noise=true , address=104.36.82.11:4128 , piece_id=XRPULHI7BYDF2JOIASSN5QKGEAC6FJS2F5C22WIY7EKYILRHZ73A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394196364 (13h53m7.138253227s) [noise=true , address=104.36.82.38:4706 , piece_id=BIHIPQX6LRWBNFN4SDC3K3V22AMGL3RC5B4RVE7QXJFKB4XDN3BA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394484397 (13h53m7.138541257s) [noise=true , address=104.36.82.61:4381 , piece_id=TTZOSRP6YPL6P4C6PY5S4K7Z6CIZJDPX7NUNBCK73CWZPCA3HQHA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394593709 (13h53m7.138650578s) [noise=true , address=45.91.116.5:4300 , piece_id=MADJCANN2O35MKXT2PIR5PGMJ3PNTAQAJNNGOK6ISJNVAH2L7YIA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.395093013 (13h53m7.139149879s) [noise=true , address=104.36.82.22:4530 , piece_id=HMUUW5GYH6TRHYXP4YXBHD566FWQRILEHI53AVURA5QTQGG6USFA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.395447945 (13h53m7.139504805s) [noise=true , address=104.36.82.58:4293 , piece_id=I2ETFXYTHYWYJYSHTVOBQTO6ZPGNMA4D3NVKYDE2W34EH4SQE23A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.954469844 (13h53m7.698526694s) [upload=14294]: Next returned nil segment Aug 7 23:32:45.954487953 (13h53m7.69854487s) [upload=14294]: Waiting for error group managing segments... Aug 7 23:32:46.126888627 (13h53m7.870945474s) [noise=true , address=104.36.82.10:4090 , piece_id=V6NPPVIYEH6WLIW7OMT3NDFPR7UDWIT46GW6AFCAWUJTBLC3O55Q, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.132983844 (13h53m7.877040704s) [noise=true , address=104.36.82.16:4321 , piece_id=YTF4RWV2SD6Z4UV4XRYPBRGPCTSUTC6TQKL3DHVKU65ALTNA4TEQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.138277216 (13h53m7.882334067s) [noise=true , address=104.36.82.23:4477 , piece_id=HAF6424ZCIBGSR4US5LN5ZWE5JMNMLFGUMQQ4YP2ARP7PQXBH5GQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.142033326 (13h53m7.886090185s) [noise=true , address=104.36.82.34:4566 , piece_id=3YEAUSXEL7WNZ3O7WM24Z2DKY2TDVKMRZSHMEPMNEL2LR7BY7DBQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.151247206 (13h53m7.895304058s) [noise=true , address=104.36.82.22:4530 , piece_id=HMUUW5GYH6TRHYXP4YXBHD566FWQRILEHI53AVURA5QTQGG6USFA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.151981081 (13h53m7.896037932s) [noise=true , address=104.36.82.58:4293 , piece_id=I2ETFXYTHYWYJYSHTVOBQTO6ZPGNMA4D3NVKYDE2W34EH4SQE23A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.165022882 (13h53m7.909079743s) [noise=true , address=104.36.82.44:4881 , piece_id=XXVSXG3ILFXG4IVMELHL45TRRHDFVVJMJ5LGOHU5EDHG6XXCDH6A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.184418821 (13h53m7.928475667s) [noise=true , address=104.36.82.47:4523 , piece_id=MJIMHSO66V7OWFFK2UALPKSI7CGGCTHNPZINDXMVIHDLENIW5LFA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.184532669 (13h53m7.928589516s) [noise=true , address=45.91.116.5:4300 , piece_id=MADJCANN2O35MKXT2PIR5PGMJ3PNTAQAJNNGOK6ISJNVAH2L7YIA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.193375808 (13h53m7.937432656s) [noise=true , address=104.36.82.54:4160 , piece_id=GDBORRYX5QBQGH7ZMMUUAV7AAR5UHZOTK5FTQUPC7YKZC2KD7I3A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.204247501 (13h53m7.948304372s) [noise=true , address=104.36.82.38:4706 , piece_id=BIHIPQX6LRWBNFN4SDC3K3V22AMGL3RC5B4RVE7QXJFKB4XDN3BA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.235225782 (13h53m7.979282629s) [noise=true , address=104.36.82.62:4439 , piece_id=ILRD4TEFW4PSLV4EGPORG3FM6U2HB3WSL5KP3PN566FAOGWODLYQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.269129867 (13h53m8.013186721s) [noise=true , address=104.36.82.5:4861 , piece_id=GRDGO5LNCETQWRU3M6LACGRW5MNA2KFLXUBT5R2UH6SEGWP6RVRQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.278216912 (13h53m8.022273764s) [noise=true , address=104.36.82.19:4405 , piece_id=MHM3MWZJFTXSJT7PTOFSWHUYO3L6NIBUV6VU66XXZDH3MC3EFR2Q, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.278247847 (13h53m8.022304708s) [seg_pos={0 0}, upload=14294]: Segment reached optimal threshold of 14 pieces. Aug 7 23:32:46.278485612 (13h53m8.022542463s) [noise=true , address=104.36.82.11:4128 , piece_id=XRPULHI7BYDF2JOIASSN5QKGEAC6FJS2F5C22WIY7EKYILRHZ73A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wABkAtAjoYkKyNoAiaVCxSvY6WgvHvAYFPCjpNojtpiYiGtDS): protocol: expected piece hash; context canceled; EOF Aug 7 23:32:46.278511698 (13h53m8.022568566s) [noise=true , address=77.90.31.12:4593 , piece_id=C3RVQHFS4L7SKPFFIZ52MAQWGD7P4JNG3SNPCNY53SQUD72PZRUA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kSxoZb9nmG6oGNytpFTsoiGh3ZPmebVVe8TZxGzANaw7EfdeF): protocol: expected piece hash; context canceled; EOF Aug 7 23:32:46.278642807 (13h53m8.022699657s) [noise=true , address=104.36.82.61:4381 , piece_id=TTZOSRP6YPL6P4C6PY5S4K7Z6CIZJDPX7NUNBCK73CWZPCA3HQHA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zU35MWWaaC1npkP5qGcWBXAkfxGsd5UsdkM2nwX9x7wUGCDbY): protocol: write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection; write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection Aug 7 23:32:46.278650926 (13h53m8.022707786s) [noise=true , address=104.36.82.37:4665 , piece_id=RUKKGC4Y3FIVLZO2GRWJN3J6P36VWP5VSC2HCLEU3MZDDFXODJAQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jP9JrCmoQSdqrzXZxkNN1ihdFkaUMjGZss6Fh19zbL3YyWj5N): protocol: write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection; write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection Aug 7 23:32:46.278674050 (13h53m8.022730911s) [seg_pos={0 0}, upload=14294]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:32:46.278751154 (13h53m8.022808014s) [upload=14294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:32:46.278756168 (13h53m8.022813029s) [upload=14294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:32:46.498660020 (13h53m8.242716882s) [upload=14294]: Done uploading segments... Aug 7 23:32:46.498701263 (13h53m8.242758128s) [upload=14294]: Upload finished. err: <nil> Duration: 3.036887052s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 105.965259ms remote: "tcp:185.244.226.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 23:56:16 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 105.987022ms edge gateway vhost bgp us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...X.XX..X..................................................] duration: 2.739418796s FAIL (2025-08-07 23:56:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.739449993s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 6 out of 60 is failed (threshold=3) [.....XXX...XX.X.............................................] duration: 1.165579453s FAIL (2025-08-07 23:56:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.165605625s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [..XXXXXXXXXX...........X............................XX.X.X..] duration: 2.445824197s FAIL (2025-08-07 23:56:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== exceeded Aug 7 23:53:43.167705717 (14h14m4.911762564s) [upload=14648]: Done uploading segments... Aug 7 23:53:43.167737922 (14h14m4.911794768s) [upload=14648]: Upload finished. err: context deadline exceeded Aug 7 23:53:43.606184070 (14h14m5.350240917s) [upload=14655]: Starting upload Aug 7 23:53:43.606251699 (14h14m5.350308565s) [upload=14655]: Done starting upload Aug 7 23:53:43.606378812 (14h14m5.350435674s) [upload=14655]: Uploading segments... Aug 7 23:53:43.606613336 (14h14m5.350670197s) [upload=14655]: Got next segment. Inline: false Aug 7 23:53:43.606652700 (14h14m5.35070955s) [upload=14655]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:43.606683541 (14h14m5.350740402s) [upload=14655]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:53:43.711705294 (14h14m5.455762157s) [seg_pos={0 0}, upload=14655]: Begin upload segment... Aug 7 23:53:43.711752671 (14h14m5.455809532s) [seg_pos={0 0}, upload=14655]: Done begin upload segment. Aug 7 23:53:43.711786432 (14h14m5.455843291s) [noise=true , address=104.36.82.10:4077 , piece_id=XOE3BBCSJ64O7A7LC7HAAYEBF3F6CIU77UVI5HY3RI2FWSXEHJXA, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.711812017 (14h14m5.455868889s) [noise=true , address=77.90.31.16:4735 , piece_id=OE7TL5NB6UH3A4JE2BDYFA4FZKHCRWT6YK6KJZHD6JJ55FOKXIEQ, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.712247313 (14h14m5.456304173s) [noise=true , address=45.91.116.10:4081 , piece_id=MMI5DUXMAD27VJI7VRQHBBEC2MJUDT5KWX2OGWLUPMTZLBIRWSAA, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.712257082 (14h14m5.456313945s) [noise=true , address=77.90.31.13:4626 , piece_id=74QG6PW6CJDJRESXUF534PQ463VRJQF33PMQRX6W6VVQCQMX2SUA, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.712579895 (14h14m5.456636755s) [noise=true , address=104.36.82.15:4258 , piece_id=JWNJIQN5WUWHVOQOQASGTGXOZWJVZZICABZCFOGZQVCDAVZ7WNVQ, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.712592145 (14h14m5.456649005s) [noise=true , address=45.91.116.4:4939 , piece_id=YLQB6XZJ6FL7SBJQ6FE4VIGKE5QYA3XPFU4E37OQKF5O2A7ZV4DQ, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.712863854 (14h14m5.456920715s) [noise=true , address=104.36.82.9:4007 , piece_id=XHRH7RHA6VKCFN7TDCMKIP6EH5EQWW3ZWVP7QE67X6LS7IR4J2MQ, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.712991802 (14h14m5.457048663s) [noise=true , address=77.90.31.13:4628 , piece_id=ZLWT3WANPKHZ6XFV7PC3KZULY73ZEOYD3E3EVY3OMIQI3OYM3LKA, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.713196040 (14h14m5.4572529s) [noise=true , address=104.36.82.14:4222 , piece_id=AAQ6HQOYMMNNHXABB2MOCCJHF43PXWJZH4SDYF6ZNSLQWMKY32AA, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.713340669 (14h14m5.457397532s) [noise=true , address=45.91.116.6:4975 , piece_id=BBULAO6EEFXHYP2AKMUF63YRSQJGVKXLIWGCKWUM5JYUVLZAB25Q, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.713490538 (14h14m5.457547399s) [noise=true , address=104.36.82.11:4114 , piece_id=IKJQ5SDEBNDZGJ6TUDTQBA2A74BQC6WV3SC4UKE7AGWEENDBI57Q, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.713730980 (14h14m5.45778784s) [noise=true , address=45.91.116.8:4045 , piece_id=7PDSFMO4MC4SMKYW3MN6OIU33MIKCLFBSJ6LYFQAGUFMOZQCLVXQ, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.713818259 (14h14m5.45787511s) [noise=true , address=77.90.31.19:4843 , piece_id=HLLIY2T4U45TYQ7L7RKY26MUTAA4KWAZ6LV7KP6HOTRJURY3WUSQ, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.714085345 (14h14m5.458142208s) [noise=true , address=45.91.116.5:4299 , piece_id=F5FM3VSMCDN3BSCGDQO5XGPZPQACVJAQG4Y2HWJEPQUF7BFGWNUA, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.714148553 (14h14m5.458205416s) [noise=true , address=77.90.31.14:4662 , piece_id=FDS6SHOXCIIUA4DKBYNMKY67RG6X5OLUKJLZ5M2M76UQB3ANWPLA, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.714360745 (14h14m5.458417608s) [noise=true , address=45.91.116.7:4010 , piece_id=RXF2GDUICLNB25V6S2CVFIKNTQHZOLWDZZWV3AEYWONFCFPZE2GA, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:44.327859573 (14h14m6.071916423s) [upload=14655]: Next returned nil segment Aug 7 23:53:44.327877823 (14h14m6.071934683s) [upload=14655]: Waiting for error group managing segments... Aug 7 23:53:44.492864996 (14h14m6.236921844s) [noise=true , address=104.36.82.11:4114 , piece_id=IKJQ5SDEBNDZGJ6TUDTQBA2A74BQC6WV3SC4UKE7AGWEENDBI57Q, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.496078923 (14h14m6.240135773s) [noise=true , address=104.36.82.9:4007 , piece_id=XHRH7RHA6VKCFN7TDCMKIP6EH5EQWW3ZWVP7QE67X6LS7IR4J2MQ, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.519818170 (14h14m6.263875019s) [noise=true , address=104.36.82.15:4258 , piece_id=JWNJIQN5WUWHVOQOQASGTGXOZWJVZZICABZCFOGZQVCDAVZ7WNVQ, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.540418125 (14h14m6.284474976s) [noise=true , address=45.91.116.7:4010 , piece_id=RXF2GDUICLNB25V6S2CVFIKNTQHZOLWDZZWV3AEYWONFCFPZE2GA, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.568753323 (14h14m6.312810176s) [noise=true , address=104.36.82.10:4077 , piece_id=XOE3BBCSJ64O7A7LC7HAAYEBF3F6CIU77UVI5HY3RI2FWSXEHJXA, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.577903595 (14h14m6.321960456s) [noise=true , address=77.90.31.14:4662 , piece_id=FDS6SHOXCIIUA4DKBYNMKY67RG6X5OLUKJLZ5M2M76UQB3ANWPLA, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.580925505 (14h14m6.324982356s) [noise=true , address=45.91.116.6:4975 , piece_id=BBULAO6EEFXHYP2AKMUF63YRSQJGVKXLIWGCKWUM5JYUVLZAB25Q, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.585723335 (14h14m6.329780193s) [noise=true , address=77.90.31.13:4628 , piece_id=ZLWT3WANPKHZ6XFV7PC3KZULY73ZEOYD3E3EVY3OMIQI3OYM3LKA, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.588082614 (14h14m6.332139464s) [noise=true , address=77.90.31.16:4735 , piece_id=OE7TL5NB6UH3A4JE2BDYFA4FZKHCRWT6YK6KJZHD6JJ55FOKXIEQ, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.626221649 (14h14m6.370278495s) [noise=true , address=45.91.116.8:4045 , piece_id=7PDSFMO4MC4SMKYW3MN6OIU33MIKCLFBSJ6LYFQAGUFMOZQCLVXQ, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.628668635 (14h14m6.372725499s) [noise=true , address=104.36.82.14:4222 , piece_id=AAQ6HQOYMMNNHXABB2MOCCJHF43PXWJZH4SDYF6ZNSLQWMKY32AA, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.636831349 (14h14m6.38088821s) [noise=true , address=77.90.31.19:4843 , piece_id=HLLIY2T4U45TYQ7L7RKY26MUTAA4KWAZ6LV7KP6HOTRJURY3WUSQ, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.645180863 (14h14m6.389237709s) [noise=true , address=45.91.116.10:4081 , piece_id=MMI5DUXMAD27VJI7VRQHBBEC2MJUDT5KWX2OGWLUPMTZLBIRWSAA, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.645221241 (14h14m6.389278101s) [seg_pos={0 0}, upload=14655]: Segment reached optimal threshold of 13 pieces. Aug 7 23:53:44.645393922 (14h14m6.389450768s) [noise=true , address=77.90.31.13:4626 , piece_id=74QG6PW6CJDJRESXUF534PQ463VRJQF33PMQRX6W6VVQCQMX2SUA, seg_pos={0 0}, upload=14655]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF Aug 7 23:53:44.645466730 (14h14m6.389523595s) [noise=true , address=45.91.116.4:4939 , piece_id=YLQB6XZJ6FL7SBJQ6FE4VIGKE5QYA3XPFU4E37OQKF5O2A7ZV4DQ, seg_pos={0 0}, upload=14655]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Aug 7 23:53:44.645496683 (14h14m6.389553539s) [noise=true , address=45.91.116.5:4299 , piece_id=F5FM3VSMCDN3BSCGDQO5XGPZPQACVJAQG4Y2HWJEPQUF7BFGWNUA, seg_pos={0 0}, upload=14655]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: expected piece hash; context canceled; EOF Aug 7 23:53:44.645513654 (14h14m6.389570515s) [seg_pos={0 0}, upload=14655]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:53:44.645605002 (14h14m6.389661865s) [upload=14655]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:53:44.645610128 (14h14m6.389666989s) [upload=14655]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:44.783671038 (14h14m6.527727892s) [upload=14655]: Done uploading segments... Aug 7 23:53:44.783719939 (14h14m6.5277768s) [upload=14655]: Upload finished. err: <nil> Aug 7 23:54:44.874516624 (14h15m6.618573476s) [upload=14672]: Starting upload Aug 7 23:54:44.874591338 (14h15m6.618648203s) [upload=14672]: Done starting upload Aug 7 23:54:44.874717241 (14h15m6.618774089s) [upload=14672]: Uploading segments... Aug 7 23:54:44.874890680 (14h15m6.61894753s) [upload=14672]: Got next segment. Inline: false Aug 7 23:54:44.874947399 (14h15m6.619004253s) [upload=14672]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:44.874953184 (14h15m6.619010035s) [upload=14672]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:55:14.597054629 (14h15m36.341111495s) [upload=14672]: Next segment err: context deadline exceeded Aug 7 23:55:14.597269046 (14h15m36.341325894s) [upload=14672]: Done uploading segments... Aug 7 23:55:14.597309239 (14h15m36.341366085s) [upload=14672]: Upload finished. err: context deadline exceeded Duration: 2.445923109s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 16 out of 60 is failed (threshold=3) [XXX.XXXXXX..XX.X...................................X...XX.X.] duration: 867.814291ms FAIL (2025-08-07 23:56:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ad finished. err: <nil> Aug 7 23:49:47.788743937 (14h10m9.532800783s) [upload=14591]: Starting upload Aug 7 23:49:47.788822734 (14h10m9.5328796s) [upload=14591]: Done starting upload Aug 7 23:49:47.788889290 (14h10m9.532946138s) [upload=14591]: Uploading segments... Aug 7 23:49:47.788950938 (14h10m9.533007798s) [upload=14591]: Got next segment. Inline: true Aug 7 23:49:47.788955912 (14h10m9.533012775s) [upload=14591]: Waiting for error group managing segments... Aug 7 23:49:47.789005080 (14h10m9.533061926s) [upload=14591]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:49:47.789009334 (14h10m9.533066194s) [upload=14591]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:49:47.789014285 (14h10m9.53307113s) [upload=14591]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:49:47.932333015 (14h10m9.676389863s) [upload=14591]: Done uploading segments... Aug 7 23:49:47.932362673 (14h10m9.676419519s) [upload=14591]: Upload finished. err: <nil> Aug 7 23:50:23.531847225 (14h10m45.275904072s) [upload=14601]: Starting upload Aug 7 23:50:23.531926184 (14h10m45.275983047s) [upload=14601]: Done starting upload Aug 7 23:50:23.531993110 (14h10m45.27604996s) [upload=14601]: Uploading segments... Aug 7 23:50:23.532035359 (14h10m45.276092227s) [upload=14601]: Got next segment. Inline: true Aug 7 23:50:23.532041373 (14h10m45.276098223s) [upload=14601]: Waiting for error group managing segments... Aug 7 23:50:23.532083061 (14h10m45.276139912s) [upload=14601]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:23.532088436 (14h10m45.276145285s) [upload=14601]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:23.532091688 (14h10m45.276148539s) [upload=14601]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:23.673405160 (14h10m45.417462011s) [upload=14601]: Done uploading segments... Aug 7 23:50:23.673423409 (14h10m45.417480259s) [upload=14601]: Upload finished. err: <nil> Aug 7 23:51:32.204614635 (14h11m53.948671502s) [upload=14620]: Starting upload Aug 7 23:51:32.204700450 (14h11m53.948757313s) [upload=14620]: Done starting upload Aug 7 23:51:32.204781575 (14h11m53.948838427s) [upload=14620]: Uploading segments... Aug 7 23:51:32.204839741 (14h11m53.94889661s) [upload=14620]: Got next segment. Inline: true Aug 7 23:51:32.204847624 (14h11m53.948904474s) [upload=14620]: Waiting for error group managing segments... Aug 7 23:51:32.204902472 (14h11m53.948959327s) [upload=14620]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:51:32.204907775 (14h11m53.948964625s) [upload=14620]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:51:32.204911177 (14h11m53.948968027s) [upload=14620]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:01.937570352 (14h12m23.681627223s) [upload=14620]: Done uploading segments... Aug 7 23:52:01.937615027 (14h12m23.681671888s) [upload=14620]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:52:03.484443714 (14h12m25.228500575s) [upload=14625]: Starting upload Aug 7 23:52:03.484557174 (14h12m25.228614037s) [upload=14625]: Done starting upload Aug 7 23:52:03.484649660 (14h12m25.22870651s) [upload=14625]: Uploading segments... Aug 7 23:52:03.484705970 (14h12m25.22876283s) [upload=14625]: Got next segment. Inline: true Aug 7 23:52:03.484711540 (14h12m25.228768402s) [upload=14625]: Waiting for error group managing segments... Aug 7 23:52:03.484758149 (14h12m25.228815009s) [upload=14625]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:03.484763263 (14h12m25.228820122s) [upload=14625]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:03.484766746 (14h12m25.228823606s) [upload=14625]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:03.643631412 (14h12m25.387688267s) [upload=14625]: Done uploading segments... Aug 7 23:52:03.643670624 (14h12m25.387727482s) [upload=14625]: Upload finished. err: <nil> Aug 7 23:52:25.641252610 (14h12m47.385309456s) [upload=14634]: Starting upload Aug 7 23:52:25.641347708 (14h12m47.385404571s) [upload=14634]: Done starting upload Aug 7 23:52:25.641412738 (14h12m47.385469584s) [upload=14634]: Uploading segments... Aug 7 23:52:25.641487429 (14h12m47.385544276s) [upload=14634]: Got next segment. Inline: true Aug 7 23:52:25.641494756 (14h12m47.385551602s) [upload=14634]: Waiting for error group managing segments... Aug 7 23:52:25.641535502 (14h12m47.385592365s) [upload=14634]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:25.641556088 (14h12m47.385612953s) [upload=14634]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:25.641559364 (14h12m47.385616224s) [upload=14634]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:25.852337418 (14h12m47.596394279s) [upload=14634]: Done uploading segments... Aug 7 23:52:25.852355736 (14h12m47.596412584s) [upload=14634]: Upload finished. err: <nil> Aug 7 23:52:51.292140355 (14h13m13.036197215s) [upload=14641]: Starting upload Aug 7 23:52:51.292216133 (14h13m13.036272985s) [upload=14641]: Done starting upload Aug 7 23:52:51.292307446 (14h13m13.036364307s) [upload=14641]: Uploading segments... Aug 7 23:52:51.292389087 (14h13m13.036445956s) [upload=14641]: Got next segment. Inline: true Aug 7 23:52:51.292394933 (14h13m13.036451793s) [upload=14641]: Waiting for error group managing segments... Aug 7 23:52:51.292456032 (14h13m13.036512883s) [upload=14641]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:51.292461787 (14h13m13.036518647s) [upload=14641]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:51.292467676 (14h13m13.036524537s) [upload=14641]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:51.436915828 (14h13m13.180972678s) [upload=14641]: Done uploading segments... Aug 7 23:52:51.436934458 (14h13m13.18099131s) [upload=14641]: Upload finished. err: <nil> Aug 7 23:53:10.946261153 (14h13m32.690318028s) [upload=14646]: Starting upload Aug 7 23:53:10.946350198 (14h13m32.690407065s) [upload=14646]: Done starting upload Aug 7 23:53:10.946435423 (14h13m32.690492276s) [upload=14646]: Uploading segments... Aug 7 23:53:10.946489255 (14h13m32.690546124s) [upload=14646]: Got next segment. Inline: true Aug 7 23:53:10.946497660 (14h13m32.690554519s) [upload=14646]: Waiting for error group managing segments... Aug 7 23:53:10.946542678 (14h13m32.690599529s) [upload=14646]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:10.946548267 (14h13m32.690605128s) [upload=14646]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:10.946553792 (14h13m32.690610652s) [upload=14646]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:14.554651436 (14h13m36.298708309s) [upload=14646]: Done uploading segments... Aug 7 23:53:14.554675183 (14h13m36.298732045s) [upload=14646]: Upload finished. err: <nil> Aug 7 23:53:53.584502029 (14h14m15.328558897s) [upload=14658]: Starting upload Aug 7 23:53:53.584599757 (14h14m15.328656616s) [upload=14658]: Done starting upload Aug 7 23:53:53.584702364 (14h14m15.328759218s) [upload=14658]: Uploading segments... Aug 7 23:53:53.584798610 (14h14m15.32885548s) [upload=14658]: Got next segment. Inline: true Aug 7 23:53:53.584806594 (14h14m15.328863461s) [upload=14658]: Waiting for error group managing segments... Aug 7 23:53:53.584875193 (14h14m15.328932062s) [upload=14658]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:53.584882324 (14h14m15.328939193s) [upload=14658]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:53.584901657 (14h14m15.328958528s) [upload=14658]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:23.318821682 (14h14m45.062878542s) [upload=14658]: Done uploading segments... Aug 7 23:54:23.318863033 (14h14m45.062919884s) [upload=14658]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:36.775209359 (14h14m58.519266209s) [upload=14671]: Starting upload Aug 7 23:54:36.775292235 (14h14m58.519349112s) [upload=14671]: Done starting upload Aug 7 23:54:36.775412031 (14h14m58.519468883s) [upload=14671]: Uploading segments... Aug 7 23:54:36.775480767 (14h14m58.519537636s) [upload=14671]: Got next segment. Inline: true Aug 7 23:54:36.775487320 (14h14m58.519544181s) [upload=14671]: Waiting for error group managing segments... Aug 7 23:54:36.775539752 (14h14m58.519596612s) [upload=14671]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:36.775545893 (14h14m58.519602753s) [upload=14671]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:36.775549619 (14h14m58.519606479s) [upload=14671]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:36.926276427 (14h14m58.670333277s) [upload=14671]: Done uploading segments... Aug 7 23:54:36.926295206 (14h14m58.670352057s) [upload=14671]: Upload finished. err: <nil> Aug 7 23:54:57.675347843 (14h15m19.419404694s) [upload=14676]: Starting upload Aug 7 23:54:57.675412352 (14h15m19.419469212s) [upload=14676]: Done starting upload Aug 7 23:54:57.675475670 (14h15m19.41953252s) [upload=14676]: Uploading segments... Aug 7 23:54:57.675516336 (14h15m19.419573205s) [upload=14676]: Got next segment. Inline: true Aug 7 23:54:57.675521071 (14h15m19.419577931s) [upload=14676]: Waiting for error group managing segments... Aug 7 23:54:57.675565512 (14h15m19.419622375s) [upload=14676]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:57.675574255 (14h15m19.419631202s) [upload=14676]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:57.675580694 (14h15m19.419637563s) [upload=14676]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:27.407552485 (14h15m49.151609344s) [upload=14676]: Done uploading segments... Aug 7 23:55:27.407592978 (14h15m49.151649829s) [upload=14676]: Upload finished. err: metaclient: context deadline exceeded Duration: 867.845944ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 8 out of 60 is failed (threshold=3) [X..XXXXX................................................XX..] duration: 1.0380225s FAIL (2025-08-07 23:56:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== nts... Aug 7 23:47:11.744570899 (14h7m33.488627759s) [upload=14546]: Upload finished. err: <nil> Aug 7 23:47:36.646321654 (14h7m58.390378505s) [upload=14555]: Starting upload Aug 7 23:47:36.646397959 (14h7m58.390454819s) [upload=14555]: Done starting upload Aug 7 23:47:36.646478843 (14h7m58.390535692s) [upload=14555]: Uploading segments... Aug 7 23:47:36.646531314 (14h7m58.390588184s) [upload=14555]: Got next segment. Inline: true Aug 7 23:47:36.646537170 (14h7m58.390594035s) [upload=14555]: Waiting for error group managing segments... Aug 7 23:47:36.646608407 (14h7m58.390665271s) [upload=14555]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:47:36.646614964 (14h7m58.390671828s) [upload=14555]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:47:36.646619338 (14h7m58.3906762s) [upload=14555]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:47:36.788927126 (14h7m58.532983976s) [upload=14555]: Done uploading segments... Aug 7 23:47:36.788947833 (14h7m58.533004684s) [upload=14555]: Upload finished. err: <nil> Aug 7 23:48:14.273939802 (14h8m36.017996675s) [upload=14564]: Starting upload Aug 7 23:48:14.274034437 (14h8m36.018091288s) [upload=14564]: Done starting upload Aug 7 23:48:14.274143056 (14h8m36.018199915s) [upload=14564]: Uploading segments... Aug 7 23:48:14.274205362 (14h8m36.01826223s) [upload=14564]: Got next segment. Inline: true Aug 7 23:48:14.274210960 (14h8m36.018267824s) [upload=14564]: Waiting for error group managing segments... Aug 7 23:48:14.274266836 (14h8m36.018323686s) [upload=14564]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:48:14.274272451 (14h8m36.018329314s) [upload=14564]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:48:14.274276048 (14h8m36.018332908s) [upload=14564]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:48:14.410395079 (14h8m36.154451958s) [upload=14564]: Done uploading segments... Aug 7 23:48:14.410429515 (14h8m36.154486376s) [upload=14564]: Upload finished. err: <nil> Aug 7 23:48:40.673955862 (14h9m2.418012725s) [upload=14570]: Starting upload Aug 7 23:48:40.674027551 (14h9m2.418084403s) [upload=14570]: Done starting upload Aug 7 23:48:40.674101278 (14h9m2.418158129s) [upload=14570]: Uploading segments... Aug 7 23:48:40.674137993 (14h9m2.418194862s) [upload=14570]: Got next segment. Inline: true Aug 7 23:48:40.674142901 (14h9m2.418199761s) [upload=14570]: Waiting for error group managing segments... Aug 7 23:48:40.674183743 (14h9m2.418240593s) [upload=14570]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:48:40.674188816 (14h9m2.418245676s) [upload=14570]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:48:40.674192280 (14h9m2.41824914s) [upload=14570]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:48:40.820583815 (14h9m2.564640664s) [upload=14570]: Done uploading segments... Aug 7 23:48:40.820602074 (14h9m2.564658926s) [upload=14570]: Upload finished. err: <nil> Aug 7 23:49:25.777587368 (14h9m47.521644216s) [upload=14582]: Starting upload Aug 7 23:49:25.777649686 (14h9m47.521706547s) [upload=14582]: Done starting upload Aug 7 23:49:25.777703819 (14h9m47.521760666s) [upload=14582]: Uploading segments... Aug 7 23:49:25.777747152 (14h9m47.521804022s) [upload=14582]: Got next segment. Inline: true Aug 7 23:49:25.777751825 (14h9m47.521808685s) [upload=14582]: Waiting for error group managing segments... Aug 7 23:49:25.777787950 (14h9m47.521844813s) [upload=14582]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:49:25.777792452 (14h9m47.521849312s) [upload=14582]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:49:25.777795064 (14h9m47.521851924s) [upload=14582]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:49:25.918566855 (14h9m47.662623707s) [upload=14582]: Done uploading segments... Aug 7 23:49:25.918602989 (14h9m47.66265984s) [upload=14582]: Upload finished. err: <nil> Aug 7 23:50:24.097841430 (14h10m45.841898277s) [upload=14602]: Starting upload Aug 7 23:50:24.097929459 (14h10m45.841986306s) [upload=14602]: Done starting upload Aug 7 23:50:24.097991265 (14h10m45.842048125s) [upload=14602]: Uploading segments... Aug 7 23:50:24.098029893 (14h10m45.842086762s) [upload=14602]: Got next segment. Inline: true Aug 7 23:50:24.098034701 (14h10m45.84209156s) [upload=14602]: Waiting for error group managing segments... Aug 7 23:50:24.098071586 (14h10m45.842128432s) [upload=14602]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:24.098076409 (14h10m45.842133268s) [upload=14602]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:24.098079284 (14h10m45.842136144s) [upload=14602]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:24.238728801 (14h10m45.982785662s) [upload=14602]: Done uploading segments... Aug 7 23:50:24.238753595 (14h10m45.982810442s) [upload=14602]: Upload finished. err: <nil> Aug 7 23:51:03.108197922 (14h11m24.852254772s) [upload=14612]: Starting upload Aug 7 23:51:03.108300837 (14h11m24.852357707s) [upload=14612]: Done starting upload Aug 7 23:51:03.108453294 (14h11m24.852510146s) [upload=14612]: Uploading segments... Aug 7 23:51:03.108972593 (14h11m24.853029491s) [upload=14612]: Got next segment. Inline: true Aug 7 23:51:03.108988857 (14h11m24.85304573s) [upload=14612]: Waiting for error group managing segments... Aug 7 23:51:03.109073931 (14h11m24.853130786s) [upload=14612]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:51:03.109080082 (14h11m24.853136933s) [upload=14612]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:51:03.109083894 (14h11m24.853140746s) [upload=14612]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:03.446817008 (14h11m25.190873858s) [upload=14612]: Done uploading segments... Aug 7 23:51:03.446842582 (14h11m25.190899436s) [upload=14612]: Upload finished. err: <nil> Aug 7 23:52:09.742356648 (14h12m31.486413495s) [upload=14628]: Starting upload Aug 7 23:52:09.742413553 (14h12m31.486470415s) [upload=14628]: Done starting upload Aug 7 23:52:09.742476702 (14h12m31.486533548s) [upload=14628]: Uploading segments... Aug 7 23:52:09.742511425 (14h12m31.486568294s) [upload=14628]: Got next segment. Inline: true Aug 7 23:52:09.742515305 (14h12m31.486572152s) [upload=14628]: Waiting for error group managing segments... Aug 7 23:52:09.742550700 (14h12m31.486607545s) [upload=14628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:09.742555041 (14h12m31.486611889s) [upload=14628]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:09.742557771 (14h12m31.486614632s) [upload=14628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:09.895389395 (14h12m31.639446255s) [upload=14628]: Done uploading segments... Aug 7 23:52:09.895421188 (14h12m31.639478039s) [upload=14628]: Upload finished. err: <nil> Aug 7 23:52:28.792030311 (14h12m50.536087163s) [upload=14635]: Starting upload Aug 7 23:52:28.792092644 (14h12m50.536149508s) [upload=14635]: Done starting upload Aug 7 23:52:28.792152087 (14h12m50.53620894s) [upload=14635]: Uploading segments... Aug 7 23:52:28.792190462 (14h12m50.536247332s) [upload=14635]: Got next segment. Inline: true Aug 7 23:52:28.792195687 (14h12m50.536252548s) [upload=14635]: Waiting for error group managing segments... Aug 7 23:52:28.792239958 (14h12m50.536296808s) [upload=14635]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:28.792245830 (14h12m50.536302691s) [upload=14635]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:28.792249165 (14h12m50.536306026s) [upload=14635]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:28.947698658 (14h12m50.691755509s) [upload=14635]: Done uploading segments... Aug 7 23:52:28.947717721 (14h12m50.691774571s) [upload=14635]: Upload finished. err: <nil> Aug 7 23:53:17.152926358 (14h13m38.896983229s) [upload=14650]: Starting upload Aug 7 23:53:17.153036870 (14h13m38.89709373s) [upload=14650]: Done starting upload Aug 7 23:53:17.153154678 (14h13m38.897211544s) [upload=14650]: Uploading segments... Aug 7 23:53:17.153233089 (14h13m38.897289957s) [upload=14650]: Got next segment. Inline: true Aug 7 23:53:17.153240256 (14h13m38.897297124s) [upload=14650]: Waiting for error group managing segments... Aug 7 23:53:17.153318892 (14h13m38.897375757s) [upload=14650]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:17.153325816 (14h13m38.89738268s) [upload=14650]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:17.153330453 (14h13m38.897387314s) [upload=14650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:16.875757982 (14h14m38.619814845s) [upload=14650]: Done uploading segments... Aug 7 23:54:16.875781849 (14h14m38.619838701s) [upload=14650]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:24.158448119 (14h14m45.90250498s) [upload=14667]: Starting upload Aug 7 23:54:24.158525946 (14h14m45.902582806s) [upload=14667]: Done starting upload Aug 7 23:54:24.158598972 (14h14m45.902655824s) [upload=14667]: Uploading segments... Aug 7 23:54:24.158654148 (14h14m45.902711017s) [upload=14667]: Got next segment. Inline: true Aug 7 23:54:24.158659995 (14h14m45.902716856s) [upload=14667]: Waiting for error group managing segments... Aug 7 23:54:24.158714896 (14h14m45.902771747s) [upload=14667]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:24.158720357 (14h14m45.902777209s) [upload=14667]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:24.158724105 (14h14m45.902780965s) [upload=14667]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:23.888400190 (14h15m45.63245704s) [upload=14667]: Done uploading segments... Aug 7 23:55:23.888419087 (14h15m45.632475938s) [upload=14667]: Upload finished. err: metaclient: context deadline exceeded Duration: 1.038059456s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 305.994786ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 23:55:56 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 306.056485ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 11 out of 60 is failed (threshold=3) [....XXXXXX.XX........................................X.XX...] duration: 2.480733777s FAIL (2025-08-07 23:55:55 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.480773093s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 826.585948ms FAIL (2025-08-07 23:55:48 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== pload=14121]: Done uploading segments... Aug 7 23:22:28.505845197 (13h42m50.249902044s) [upload=14121]: Upload finished. err: <nil> Aug 7 23:23:10.377483605 (13h43m32.121540458s) [upload=14132]: Starting upload Aug 7 23:23:10.377535635 (13h43m32.121592495s) [upload=14132]: Done starting upload Aug 7 23:23:10.377603797 (13h43m32.121660647s) [upload=14132]: Uploading segments... Aug 7 23:23:10.377638935 (13h43m32.121695804s) [upload=14132]: Got next segment. Inline: true Aug 7 23:23:10.377643594 (13h43m32.121700454s) [upload=14132]: Waiting for error group managing segments... Aug 7 23:23:10.377688252 (13h43m32.121745104s) [upload=14132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:23:10.377693427 (13h43m32.121750286s) [upload=14132]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:23:10.377696805 (13h43m32.121753666s) [upload=14132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:23:10.562260724 (13h43m32.306317573s) [upload=14132]: Done uploading segments... Aug 7 23:23:10.562279228 (13h43m32.306336078s) [upload=14132]: Upload finished. err: <nil> Aug 7 23:24:07.896404858 (13h44m29.640461721s) [upload=14147]: Starting upload Aug 7 23:24:07.896490216 (13h44m29.640547079s) [upload=14147]: Done starting upload Aug 7 23:24:07.896580488 (13h44m29.640637339s) [upload=14147]: Uploading segments... Aug 7 23:24:07.896655006 (13h44m29.640711874s) [upload=14147]: Got next segment. Inline: true Aug 7 23:24:07.896660179 (13h44m29.640717039s) [upload=14147]: Waiting for error group managing segments... Aug 7 23:24:07.896714840 (13h44m29.6407717s) [upload=14147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:24:07.896719399 (13h44m29.64077626s) [upload=14147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:24:07.896722726 (13h44m29.640779586s) [upload=14147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:24:08.048898130 (13h44m29.792954991s) [upload=14147]: Done uploading segments... Aug 7 23:24:08.048952526 (13h44m29.793009386s) [upload=14147]: Upload finished. err: <nil> Aug 7 23:24:53.794754379 (13h45m15.53881125s) [upload=14162]: Starting upload Aug 7 23:24:53.794842317 (13h45m15.538899177s) [upload=14162]: Done starting upload Aug 7 23:24:53.794935964 (13h45m15.538992814s) [upload=14162]: Uploading segments... Aug 7 23:24:53.795003752 (13h45m15.539060623s) [upload=14162]: Got next segment. Inline: true Aug 7 23:24:53.795010814 (13h45m15.539067676s) [upload=14162]: Waiting for error group managing segments... Aug 7 23:24:53.795066607 (13h45m15.539123488s) [upload=14162]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:24:53.795158449 (13h45m15.5392153s) [upload=14162]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:24:53.795163166 (13h45m15.539220026s) [upload=14162]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:24:53.946994982 (13h45m15.691051833s) [upload=14162]: Done uploading segments... Aug 7 23:24:53.947013862 (13h45m15.691070711s) [upload=14162]: Upload finished. err: <nil> Aug 7 23:25:40.029856966 (13h46m1.773913836s) [upload=14177]: Starting upload Aug 7 23:25:40.029986938 (13h46m1.774043809s) [upload=14177]: Done starting upload Aug 7 23:25:40.030076796 (13h46m1.774133642s) [upload=14177]: Uploading segments... Aug 7 23:25:40.030132567 (13h46m1.774189438s) [upload=14177]: Got next segment. Inline: true Aug 7 23:25:40.030139885 (13h46m1.774196757s) [upload=14177]: Waiting for error group managing segments... Aug 7 23:25:40.030197588 (13h46m1.774254461s) [upload=14177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:25:40.030204146 (13h46m1.774261016s) [upload=14177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:25:40.030207984 (13h46m1.77426485s) [upload=14177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:25:40.184198742 (13h46m1.928255592s) [upload=14177]: Done uploading segments... Aug 7 23:25:40.184219017 (13h46m1.928275867s) [upload=14177]: Upload finished. err: <nil> Aug 7 23:26:54.625649110 (13h47m16.369705971s) [upload=14198]: Starting upload Aug 7 23:26:54.625738563 (13h47m16.369795424s) [upload=14198]: Done starting upload Aug 7 23:26:54.625842691 (13h47m16.369899539s) [upload=14198]: Uploading segments... Aug 7 23:26:54.626016839 (13h47m16.37007371s) [upload=14198]: Got next segment. Inline: true Aug 7 23:26:54.626052112 (13h47m16.370108973s) [upload=14198]: Waiting for error group managing segments... Aug 7 23:26:54.626129780 (13h47m16.370186645s) [upload=14198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:26:54.626137392 (13h47m16.370194264s) [upload=14198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:26:54.626143182 (13h47m16.370200048s) [upload=14198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:26:54.775804171 (13h47m16.519861021s) [upload=14198]: Done uploading segments... Aug 7 23:26:54.775824249 (13h47m16.519881102s) [upload=14198]: Upload finished. err: <nil> Aug 7 23:27:32.263746490 (13h47m54.007803361s) [upload=14208]: Starting upload Aug 7 23:27:32.263833655 (13h47m54.007890501s) [upload=14208]: Done starting upload Aug 7 23:27:32.263976536 (13h47m54.008033385s) [upload=14208]: Uploading segments... Aug 7 23:27:32.264070926 (13h47m54.008127796s) [upload=14208]: Got next segment. Inline: true Aug 7 23:27:32.264080356 (13h47m54.00813733s) [upload=14208]: Waiting for error group managing segments... Aug 7 23:27:32.264154657 (13h47m54.008211526s) [upload=14208]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:27:32.264163587 (13h47m54.008220454s) [upload=14208]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:27:32.264172017 (13h47m54.008228914s) [upload=14208]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:27:32.490188559 (13h47m54.23424541s) [upload=14208]: Done uploading segments... Aug 7 23:27:32.490207326 (13h47m54.234264172s) [upload=14208]: Upload finished. err: <nil> Aug 7 23:27:57.927940477 (13h48m19.671997323s) [upload=14217]: Starting upload Aug 7 23:27:57.928016423 (13h48m19.672073283s) [upload=14217]: Done starting upload Aug 7 23:27:57.928101290 (13h48m19.672158142s) [upload=14217]: Uploading segments... Aug 7 23:27:57.928155692 (13h48m19.672212566s) [upload=14217]: Got next segment. Inline: true Aug 7 23:27:57.928162009 (13h48m19.672218869s) [upload=14217]: Waiting for error group managing segments... Aug 7 23:27:57.928213294 (13h48m19.672270145s) [upload=14217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:27:57.928219212 (13h48m19.672276074s) [upload=14217]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:27:57.928223521 (13h48m19.672280383s) [upload=14217]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:27:58.087544005 (13h48m19.831600874s) [upload=14217]: Done uploading segments... Aug 7 23:27:58.087570226 (13h48m19.831627094s) [upload=14217]: Upload finished. err: <nil> Aug 7 23:29:12.765200935 (13h49m34.509257795s) [upload=14237]: Starting upload Aug 7 23:29:12.765253106 (13h49m34.509309977s) [upload=14237]: Done starting upload Aug 7 23:29:12.765324641 (13h49m34.509381499s) [upload=14237]: Uploading segments... Aug 7 23:29:12.765374187 (13h49m34.509431056s) [upload=14237]: Got next segment. Inline: true Aug 7 23:29:12.765378947 (13h49m34.509435807s) [upload=14237]: Waiting for error group managing segments... Aug 7 23:29:12.765430859 (13h49m34.50948772s) [upload=14237]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:29:12.765438106 (13h49m34.50949497s) [upload=14237]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:29:12.765443127 (13h49m34.509499988s) [upload=14237]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:29:12.926437674 (13h49m34.670494535s) [upload=14237]: Done uploading segments... Aug 7 23:29:12.926457418 (13h49m34.670514268s) [upload=14237]: Upload finished. err: <nil> Aug 7 23:29:42.225760316 (13h50m3.969817177s) [upload=14244]: Starting upload Aug 7 23:29:42.225855063 (13h50m3.969911929s) [upload=14244]: Done starting upload Aug 7 23:29:42.225962489 (13h50m3.970019355s) [upload=14244]: Uploading segments... Aug 7 23:29:42.226078484 (13h50m3.970135331s) [upload=14244]: Got next segment. Inline: true Aug 7 23:29:42.226083996 (13h50m3.970140842s) [upload=14244]: Waiting for error group managing segments... Aug 7 23:29:42.226143067 (13h50m3.970199914s) [upload=14244]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:29:42.226148511 (13h50m3.970205371s) [upload=14244]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:29:42.226153006 (13h50m3.970209865s) [upload=14244]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:29:42.401822947 (13h50m4.14587981s) [upload=14244]: Done uploading segments... Aug 7 23:29:42.401847556 (13h50m4.145904405s) [upload=14244]: Upload finished. err: <nil> Aug 7 23:30:31.529840301 (13h50m53.273897165s) [upload=14254]: Starting upload Aug 7 23:30:31.530002531 (13h50m53.274059378s) [upload=14254]: Done starting upload Aug 7 23:30:31.530201544 (13h50m53.274258391s) [upload=14254]: Uploading segments... Aug 7 23:30:31.530298120 (13h50m53.274354985s) [upload=14254]: Got next segment. Inline: true Aug 7 23:30:31.530303457 (13h50m53.274360317s) [upload=14254]: Waiting for error group managing segments... Aug 7 23:30:31.530353721 (13h50m53.274410567s) [upload=14254]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:30:31.530358927 (13h50m53.274415788s) [upload=14254]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:30:31.530363758 (13h50m53.274420619s) [upload=14254]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:30:31.696350967 (13h50m53.440407828s) [upload=14254]: Done uploading segments... Aug 7 23:30:31.696369460 (13h50m53.440426307s) [upload=14254]: Upload finished. err: <nil> Duration: 826.686215ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 6 out of 60 is failed (threshold=3) [......XXX...XX.X............................................] duration: 1.026391865s FAIL (2025-08-07 23:55:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.026418831s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 10 out of 60 is failed (threshold=3) [...X.XXXXX.X............................................XXX.] duration: 13.173047385s FAIL (2025-08-07 23:55:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 13.173430651s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 12 out of 60 is failed (threshold=3) [...XXXXX.XX.XX.X.........................................X.X] duration: 30.001363877s FAIL (2025-08-07 23:55:31 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.001391681s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 16 out of 60 is failed (threshold=3) [.XXX.XXXXXX..XX.X...................................X...XX.X] duration: 30.001999355s FAIL (2025-08-07 23:55:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ad finished. err: <nil> Aug 7 23:49:47.788743937 (14h10m9.532800783s) [upload=14591]: Starting upload Aug 7 23:49:47.788822734 (14h10m9.5328796s) [upload=14591]: Done starting upload Aug 7 23:49:47.788889290 (14h10m9.532946138s) [upload=14591]: Uploading segments... Aug 7 23:49:47.788950938 (14h10m9.533007798s) [upload=14591]: Got next segment. Inline: true Aug 7 23:49:47.788955912 (14h10m9.533012775s) [upload=14591]: Waiting for error group managing segments... Aug 7 23:49:47.789005080 (14h10m9.533061926s) [upload=14591]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:49:47.789009334 (14h10m9.533066194s) [upload=14591]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:49:47.789014285 (14h10m9.53307113s) [upload=14591]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:49:47.932333015 (14h10m9.676389863s) [upload=14591]: Done uploading segments... Aug 7 23:49:47.932362673 (14h10m9.676419519s) [upload=14591]: Upload finished. err: <nil> Aug 7 23:50:23.531847225 (14h10m45.275904072s) [upload=14601]: Starting upload Aug 7 23:50:23.531926184 (14h10m45.275983047s) [upload=14601]: Done starting upload Aug 7 23:50:23.531993110 (14h10m45.27604996s) [upload=14601]: Uploading segments... Aug 7 23:50:23.532035359 (14h10m45.276092227s) [upload=14601]: Got next segment. Inline: true Aug 7 23:50:23.532041373 (14h10m45.276098223s) [upload=14601]: Waiting for error group managing segments... Aug 7 23:50:23.532083061 (14h10m45.276139912s) [upload=14601]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:23.532088436 (14h10m45.276145285s) [upload=14601]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:23.532091688 (14h10m45.276148539s) [upload=14601]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:23.673405160 (14h10m45.417462011s) [upload=14601]: Done uploading segments... Aug 7 23:50:23.673423409 (14h10m45.417480259s) [upload=14601]: Upload finished. err: <nil> Aug 7 23:51:32.204614635 (14h11m53.948671502s) [upload=14620]: Starting upload Aug 7 23:51:32.204700450 (14h11m53.948757313s) [upload=14620]: Done starting upload Aug 7 23:51:32.204781575 (14h11m53.948838427s) [upload=14620]: Uploading segments... Aug 7 23:51:32.204839741 (14h11m53.94889661s) [upload=14620]: Got next segment. Inline: true Aug 7 23:51:32.204847624 (14h11m53.948904474s) [upload=14620]: Waiting for error group managing segments... Aug 7 23:51:32.204902472 (14h11m53.948959327s) [upload=14620]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:51:32.204907775 (14h11m53.948964625s) [upload=14620]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:51:32.204911177 (14h11m53.948968027s) [upload=14620]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:01.937570352 (14h12m23.681627223s) [upload=14620]: Done uploading segments... Aug 7 23:52:01.937615027 (14h12m23.681671888s) [upload=14620]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:52:03.484443714 (14h12m25.228500575s) [upload=14625]: Starting upload Aug 7 23:52:03.484557174 (14h12m25.228614037s) [upload=14625]: Done starting upload Aug 7 23:52:03.484649660 (14h12m25.22870651s) [upload=14625]: Uploading segments... Aug 7 23:52:03.484705970 (14h12m25.22876283s) [upload=14625]: Got next segment. Inline: true Aug 7 23:52:03.484711540 (14h12m25.228768402s) [upload=14625]: Waiting for error group managing segments... Aug 7 23:52:03.484758149 (14h12m25.228815009s) [upload=14625]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:03.484763263 (14h12m25.228820122s) [upload=14625]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:03.484766746 (14h12m25.228823606s) [upload=14625]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:03.643631412 (14h12m25.387688267s) [upload=14625]: Done uploading segments... Aug 7 23:52:03.643670624 (14h12m25.387727482s) [upload=14625]: Upload finished. err: <nil> Aug 7 23:52:25.641252610 (14h12m47.385309456s) [upload=14634]: Starting upload Aug 7 23:52:25.641347708 (14h12m47.385404571s) [upload=14634]: Done starting upload Aug 7 23:52:25.641412738 (14h12m47.385469584s) [upload=14634]: Uploading segments... Aug 7 23:52:25.641487429 (14h12m47.385544276s) [upload=14634]: Got next segment. Inline: true Aug 7 23:52:25.641494756 (14h12m47.385551602s) [upload=14634]: Waiting for error group managing segments... Aug 7 23:52:25.641535502 (14h12m47.385592365s) [upload=14634]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:25.641556088 (14h12m47.385612953s) [upload=14634]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:25.641559364 (14h12m47.385616224s) [upload=14634]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:25.852337418 (14h12m47.596394279s) [upload=14634]: Done uploading segments... Aug 7 23:52:25.852355736 (14h12m47.596412584s) [upload=14634]: Upload finished. err: <nil> Aug 7 23:52:51.292140355 (14h13m13.036197215s) [upload=14641]: Starting upload Aug 7 23:52:51.292216133 (14h13m13.036272985s) [upload=14641]: Done starting upload Aug 7 23:52:51.292307446 (14h13m13.036364307s) [upload=14641]: Uploading segments... Aug 7 23:52:51.292389087 (14h13m13.036445956s) [upload=14641]: Got next segment. Inline: true Aug 7 23:52:51.292394933 (14h13m13.036451793s) [upload=14641]: Waiting for error group managing segments... Aug 7 23:52:51.292456032 (14h13m13.036512883s) [upload=14641]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:51.292461787 (14h13m13.036518647s) [upload=14641]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:51.292467676 (14h13m13.036524537s) [upload=14641]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:51.436915828 (14h13m13.180972678s) [upload=14641]: Done uploading segments... Aug 7 23:52:51.436934458 (14h13m13.18099131s) [upload=14641]: Upload finished. err: <nil> Aug 7 23:53:10.946261153 (14h13m32.690318028s) [upload=14646]: Starting upload Aug 7 23:53:10.946350198 (14h13m32.690407065s) [upload=14646]: Done starting upload Aug 7 23:53:10.946435423 (14h13m32.690492276s) [upload=14646]: Uploading segments... Aug 7 23:53:10.946489255 (14h13m32.690546124s) [upload=14646]: Got next segment. Inline: true Aug 7 23:53:10.946497660 (14h13m32.690554519s) [upload=14646]: Waiting for error group managing segments... Aug 7 23:53:10.946542678 (14h13m32.690599529s) [upload=14646]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:10.946548267 (14h13m32.690605128s) [upload=14646]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:10.946553792 (14h13m32.690610652s) [upload=14646]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:14.554651436 (14h13m36.298708309s) [upload=14646]: Done uploading segments... Aug 7 23:53:14.554675183 (14h13m36.298732045s) [upload=14646]: Upload finished. err: <nil> Aug 7 23:53:53.584502029 (14h14m15.328558897s) [upload=14658]: Starting upload Aug 7 23:53:53.584599757 (14h14m15.328656616s) [upload=14658]: Done starting upload Aug 7 23:53:53.584702364 (14h14m15.328759218s) [upload=14658]: Uploading segments... Aug 7 23:53:53.584798610 (14h14m15.32885548s) [upload=14658]: Got next segment. Inline: true Aug 7 23:53:53.584806594 (14h14m15.328863461s) [upload=14658]: Waiting for error group managing segments... Aug 7 23:53:53.584875193 (14h14m15.328932062s) [upload=14658]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:53.584882324 (14h14m15.328939193s) [upload=14658]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:53.584901657 (14h14m15.328958528s) [upload=14658]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:23.318821682 (14h14m45.062878542s) [upload=14658]: Done uploading segments... Aug 7 23:54:23.318863033 (14h14m45.062919884s) [upload=14658]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:36.775209359 (14h14m58.519266209s) [upload=14671]: Starting upload Aug 7 23:54:36.775292235 (14h14m58.519349112s) [upload=14671]: Done starting upload Aug 7 23:54:36.775412031 (14h14m58.519468883s) [upload=14671]: Uploading segments... Aug 7 23:54:36.775480767 (14h14m58.519537636s) [upload=14671]: Got next segment. Inline: true Aug 7 23:54:36.775487320 (14h14m58.519544181s) [upload=14671]: Waiting for error group managing segments... Aug 7 23:54:36.775539752 (14h14m58.519596612s) [upload=14671]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:36.775545893 (14h14m58.519602753s) [upload=14671]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:36.775549619 (14h14m58.519606479s) [upload=14671]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:36.926276427 (14h14m58.670333277s) [upload=14671]: Done uploading segments... Aug 7 23:54:36.926295206 (14h14m58.670352057s) [upload=14671]: Upload finished. err: <nil> Aug 7 23:54:57.675347843 (14h15m19.419404694s) [upload=14676]: Starting upload Aug 7 23:54:57.675412352 (14h15m19.419469212s) [upload=14676]: Done starting upload Aug 7 23:54:57.675475670 (14h15m19.41953252s) [upload=14676]: Uploading segments... Aug 7 23:54:57.675516336 (14h15m19.419573205s) [upload=14676]: Got next segment. Inline: true Aug 7 23:54:57.675521071 (14h15m19.419577931s) [upload=14676]: Waiting for error group managing segments... Aug 7 23:54:57.675565512 (14h15m19.419622375s) [upload=14676]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:57.675574255 (14h15m19.419631202s) [upload=14676]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:57.675580694 (14h15m19.419637563s) [upload=14676]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:27.407552485 (14h15m49.151609344s) [upload=14676]: Done uploading segments... Aug 7 23:55:27.407592978 (14h15m49.151649829s) [upload=14676]: Upload finished. err: metaclient: context deadline exceeded Duration: 30.002061335s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 10 out of 60 is failed (threshold=3) [....X.XXXXX.X............................................XXX] duration: 1m0.001160863s FAIL (2025-08-07 23:55:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.00118663s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 9 out of 60 is failed (threshold=3) [XX..XXXXX................................................XX.] duration: 855.309625ms FAIL (2025-08-07 23:55:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== nts... Aug 7 23:47:11.744570899 (14h7m33.488627759s) [upload=14546]: Upload finished. err: <nil> Aug 7 23:47:36.646321654 (14h7m58.390378505s) [upload=14555]: Starting upload Aug 7 23:47:36.646397959 (14h7m58.390454819s) [upload=14555]: Done starting upload Aug 7 23:47:36.646478843 (14h7m58.390535692s) [upload=14555]: Uploading segments... Aug 7 23:47:36.646531314 (14h7m58.390588184s) [upload=14555]: Got next segment. Inline: true Aug 7 23:47:36.646537170 (14h7m58.390594035s) [upload=14555]: Waiting for error group managing segments... Aug 7 23:47:36.646608407 (14h7m58.390665271s) [upload=14555]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:47:36.646614964 (14h7m58.390671828s) [upload=14555]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:47:36.646619338 (14h7m58.3906762s) [upload=14555]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:47:36.788927126 (14h7m58.532983976s) [upload=14555]: Done uploading segments... Aug 7 23:47:36.788947833 (14h7m58.533004684s) [upload=14555]: Upload finished. err: <nil> Aug 7 23:48:14.273939802 (14h8m36.017996675s) [upload=14564]: Starting upload Aug 7 23:48:14.274034437 (14h8m36.018091288s) [upload=14564]: Done starting upload Aug 7 23:48:14.274143056 (14h8m36.018199915s) [upload=14564]: Uploading segments... Aug 7 23:48:14.274205362 (14h8m36.01826223s) [upload=14564]: Got next segment. Inline: true Aug 7 23:48:14.274210960 (14h8m36.018267824s) [upload=14564]: Waiting for error group managing segments... Aug 7 23:48:14.274266836 (14h8m36.018323686s) [upload=14564]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:48:14.274272451 (14h8m36.018329314s) [upload=14564]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:48:14.274276048 (14h8m36.018332908s) [upload=14564]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:48:14.410395079 (14h8m36.154451958s) [upload=14564]: Done uploading segments... Aug 7 23:48:14.410429515 (14h8m36.154486376s) [upload=14564]: Upload finished. err: <nil> Aug 7 23:48:40.673955862 (14h9m2.418012725s) [upload=14570]: Starting upload Aug 7 23:48:40.674027551 (14h9m2.418084403s) [upload=14570]: Done starting upload Aug 7 23:48:40.674101278 (14h9m2.418158129s) [upload=14570]: Uploading segments... Aug 7 23:48:40.674137993 (14h9m2.418194862s) [upload=14570]: Got next segment. Inline: true Aug 7 23:48:40.674142901 (14h9m2.418199761s) [upload=14570]: Waiting for error group managing segments... Aug 7 23:48:40.674183743 (14h9m2.418240593s) [upload=14570]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:48:40.674188816 (14h9m2.418245676s) [upload=14570]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:48:40.674192280 (14h9m2.41824914s) [upload=14570]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:48:40.820583815 (14h9m2.564640664s) [upload=14570]: Done uploading segments... Aug 7 23:48:40.820602074 (14h9m2.564658926s) [upload=14570]: Upload finished. err: <nil> Aug 7 23:49:25.777587368 (14h9m47.521644216s) [upload=14582]: Starting upload Aug 7 23:49:25.777649686 (14h9m47.521706547s) [upload=14582]: Done starting upload Aug 7 23:49:25.777703819 (14h9m47.521760666s) [upload=14582]: Uploading segments... Aug 7 23:49:25.777747152 (14h9m47.521804022s) [upload=14582]: Got next segment. Inline: true Aug 7 23:49:25.777751825 (14h9m47.521808685s) [upload=14582]: Waiting for error group managing segments... Aug 7 23:49:25.777787950 (14h9m47.521844813s) [upload=14582]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:49:25.777792452 (14h9m47.521849312s) [upload=14582]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:49:25.777795064 (14h9m47.521851924s) [upload=14582]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:49:25.918566855 (14h9m47.662623707s) [upload=14582]: Done uploading segments... Aug 7 23:49:25.918602989 (14h9m47.66265984s) [upload=14582]: Upload finished. err: <nil> Aug 7 23:50:24.097841430 (14h10m45.841898277s) [upload=14602]: Starting upload Aug 7 23:50:24.097929459 (14h10m45.841986306s) [upload=14602]: Done starting upload Aug 7 23:50:24.097991265 (14h10m45.842048125s) [upload=14602]: Uploading segments... Aug 7 23:50:24.098029893 (14h10m45.842086762s) [upload=14602]: Got next segment. Inline: true Aug 7 23:50:24.098034701 (14h10m45.84209156s) [upload=14602]: Waiting for error group managing segments... Aug 7 23:50:24.098071586 (14h10m45.842128432s) [upload=14602]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:24.098076409 (14h10m45.842133268s) [upload=14602]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:24.098079284 (14h10m45.842136144s) [upload=14602]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:24.238728801 (14h10m45.982785662s) [upload=14602]: Done uploading segments... Aug 7 23:50:24.238753595 (14h10m45.982810442s) [upload=14602]: Upload finished. err: <nil> Aug 7 23:51:03.108197922 (14h11m24.852254772s) [upload=14612]: Starting upload Aug 7 23:51:03.108300837 (14h11m24.852357707s) [upload=14612]: Done starting upload Aug 7 23:51:03.108453294 (14h11m24.852510146s) [upload=14612]: Uploading segments... Aug 7 23:51:03.108972593 (14h11m24.853029491s) [upload=14612]: Got next segment. Inline: true Aug 7 23:51:03.108988857 (14h11m24.85304573s) [upload=14612]: Waiting for error group managing segments... Aug 7 23:51:03.109073931 (14h11m24.853130786s) [upload=14612]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:51:03.109080082 (14h11m24.853136933s) [upload=14612]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:51:03.109083894 (14h11m24.853140746s) [upload=14612]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:03.446817008 (14h11m25.190873858s) [upload=14612]: Done uploading segments... Aug 7 23:51:03.446842582 (14h11m25.190899436s) [upload=14612]: Upload finished. err: <nil> Aug 7 23:52:09.742356648 (14h12m31.486413495s) [upload=14628]: Starting upload Aug 7 23:52:09.742413553 (14h12m31.486470415s) [upload=14628]: Done starting upload Aug 7 23:52:09.742476702 (14h12m31.486533548s) [upload=14628]: Uploading segments... Aug 7 23:52:09.742511425 (14h12m31.486568294s) [upload=14628]: Got next segment. Inline: true Aug 7 23:52:09.742515305 (14h12m31.486572152s) [upload=14628]: Waiting for error group managing segments... Aug 7 23:52:09.742550700 (14h12m31.486607545s) [upload=14628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:09.742555041 (14h12m31.486611889s) [upload=14628]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:09.742557771 (14h12m31.486614632s) [upload=14628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:09.895389395 (14h12m31.639446255s) [upload=14628]: Done uploading segments... Aug 7 23:52:09.895421188 (14h12m31.639478039s) [upload=14628]: Upload finished. err: <nil> Aug 7 23:52:28.792030311 (14h12m50.536087163s) [upload=14635]: Starting upload Aug 7 23:52:28.792092644 (14h12m50.536149508s) [upload=14635]: Done starting upload Aug 7 23:52:28.792152087 (14h12m50.53620894s) [upload=14635]: Uploading segments... Aug 7 23:52:28.792190462 (14h12m50.536247332s) [upload=14635]: Got next segment. Inline: true Aug 7 23:52:28.792195687 (14h12m50.536252548s) [upload=14635]: Waiting for error group managing segments... Aug 7 23:52:28.792239958 (14h12m50.536296808s) [upload=14635]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:28.792245830 (14h12m50.536302691s) [upload=14635]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:28.792249165 (14h12m50.536306026s) [upload=14635]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:28.947698658 (14h12m50.691755509s) [upload=14635]: Done uploading segments... Aug 7 23:52:28.947717721 (14h12m50.691774571s) [upload=14635]: Upload finished. err: <nil> Aug 7 23:53:17.152926358 (14h13m38.896983229s) [upload=14650]: Starting upload Aug 7 23:53:17.153036870 (14h13m38.89709373s) [upload=14650]: Done starting upload Aug 7 23:53:17.153154678 (14h13m38.897211544s) [upload=14650]: Uploading segments... Aug 7 23:53:17.153233089 (14h13m38.897289957s) [upload=14650]: Got next segment. Inline: true Aug 7 23:53:17.153240256 (14h13m38.897297124s) [upload=14650]: Waiting for error group managing segments... Aug 7 23:53:17.153318892 (14h13m38.897375757s) [upload=14650]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:17.153325816 (14h13m38.89738268s) [upload=14650]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:17.153330453 (14h13m38.897387314s) [upload=14650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:16.875757982 (14h14m38.619814845s) [upload=14650]: Done uploading segments... Aug 7 23:54:16.875781849 (14h14m38.619838701s) [upload=14650]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:24.158448119 (14h14m45.90250498s) [upload=14667]: Starting upload Aug 7 23:54:24.158525946 (14h14m45.902582806s) [upload=14667]: Done starting upload Aug 7 23:54:24.158598972 (14h14m45.902655824s) [upload=14667]: Uploading segments... Aug 7 23:54:24.158654148 (14h14m45.902711017s) [upload=14667]: Got next segment. Inline: true Aug 7 23:54:24.158659995 (14h14m45.902716856s) [upload=14667]: Waiting for error group managing segments... Aug 7 23:54:24.158714896 (14h14m45.902771747s) [upload=14667]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:24.158720357 (14h14m45.902777209s) [upload=14667]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:24.158724105 (14h14m45.902780965s) [upload=14667]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:23.888400190 (14h15m45.63245704s) [upload=14667]: Done uploading segments... Aug 7 23:55:23.888419087 (14h15m45.632475938s) [upload=14667]: Upload finished. err: metaclient: context deadline exceeded Duration: 855.335769ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 9 out of 60 is failed (threshold=3) [.XX..XXXXX................................................XX] duration: 1m0.000948778s FAIL (2025-08-07 23:55:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== nts... Aug 7 23:47:11.744570899 (14h7m33.488627759s) [upload=14546]: Upload finished. err: <nil> Aug 7 23:47:36.646321654 (14h7m58.390378505s) [upload=14555]: Starting upload Aug 7 23:47:36.646397959 (14h7m58.390454819s) [upload=14555]: Done starting upload Aug 7 23:47:36.646478843 (14h7m58.390535692s) [upload=14555]: Uploading segments... Aug 7 23:47:36.646531314 (14h7m58.390588184s) [upload=14555]: Got next segment. Inline: true Aug 7 23:47:36.646537170 (14h7m58.390594035s) [upload=14555]: Waiting for error group managing segments... Aug 7 23:47:36.646608407 (14h7m58.390665271s) [upload=14555]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:47:36.646614964 (14h7m58.390671828s) [upload=14555]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:47:36.646619338 (14h7m58.3906762s) [upload=14555]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:47:36.788927126 (14h7m58.532983976s) [upload=14555]: Done uploading segments... Aug 7 23:47:36.788947833 (14h7m58.533004684s) [upload=14555]: Upload finished. err: <nil> Aug 7 23:48:14.273939802 (14h8m36.017996675s) [upload=14564]: Starting upload Aug 7 23:48:14.274034437 (14h8m36.018091288s) [upload=14564]: Done starting upload Aug 7 23:48:14.274143056 (14h8m36.018199915s) [upload=14564]: Uploading segments... Aug 7 23:48:14.274205362 (14h8m36.01826223s) [upload=14564]: Got next segment. Inline: true Aug 7 23:48:14.274210960 (14h8m36.018267824s) [upload=14564]: Waiting for error group managing segments... Aug 7 23:48:14.274266836 (14h8m36.018323686s) [upload=14564]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:48:14.274272451 (14h8m36.018329314s) [upload=14564]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:48:14.274276048 (14h8m36.018332908s) [upload=14564]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:48:14.410395079 (14h8m36.154451958s) [upload=14564]: Done uploading segments... Aug 7 23:48:14.410429515 (14h8m36.154486376s) [upload=14564]: Upload finished. err: <nil> Aug 7 23:48:40.673955862 (14h9m2.418012725s) [upload=14570]: Starting upload Aug 7 23:48:40.674027551 (14h9m2.418084403s) [upload=14570]: Done starting upload Aug 7 23:48:40.674101278 (14h9m2.418158129s) [upload=14570]: Uploading segments... Aug 7 23:48:40.674137993 (14h9m2.418194862s) [upload=14570]: Got next segment. Inline: true Aug 7 23:48:40.674142901 (14h9m2.418199761s) [upload=14570]: Waiting for error group managing segments... Aug 7 23:48:40.674183743 (14h9m2.418240593s) [upload=14570]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:48:40.674188816 (14h9m2.418245676s) [upload=14570]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:48:40.674192280 (14h9m2.41824914s) [upload=14570]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:48:40.820583815 (14h9m2.564640664s) [upload=14570]: Done uploading segments... Aug 7 23:48:40.820602074 (14h9m2.564658926s) [upload=14570]: Upload finished. err: <nil> Aug 7 23:49:25.777587368 (14h9m47.521644216s) [upload=14582]: Starting upload Aug 7 23:49:25.777649686 (14h9m47.521706547s) [upload=14582]: Done starting upload Aug 7 23:49:25.777703819 (14h9m47.521760666s) [upload=14582]: Uploading segments... Aug 7 23:49:25.777747152 (14h9m47.521804022s) [upload=14582]: Got next segment. Inline: true Aug 7 23:49:25.777751825 (14h9m47.521808685s) [upload=14582]: Waiting for error group managing segments... Aug 7 23:49:25.777787950 (14h9m47.521844813s) [upload=14582]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:49:25.777792452 (14h9m47.521849312s) [upload=14582]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:49:25.777795064 (14h9m47.521851924s) [upload=14582]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:49:25.918566855 (14h9m47.662623707s) [upload=14582]: Done uploading segments... Aug 7 23:49:25.918602989 (14h9m47.66265984s) [upload=14582]: Upload finished. err: <nil> Aug 7 23:50:24.097841430 (14h10m45.841898277s) [upload=14602]: Starting upload Aug 7 23:50:24.097929459 (14h10m45.841986306s) [upload=14602]: Done starting upload Aug 7 23:50:24.097991265 (14h10m45.842048125s) [upload=14602]: Uploading segments... Aug 7 23:50:24.098029893 (14h10m45.842086762s) [upload=14602]: Got next segment. Inline: true Aug 7 23:50:24.098034701 (14h10m45.84209156s) [upload=14602]: Waiting for error group managing segments... Aug 7 23:50:24.098071586 (14h10m45.842128432s) [upload=14602]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:24.098076409 (14h10m45.842133268s) [upload=14602]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:24.098079284 (14h10m45.842136144s) [upload=14602]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:24.238728801 (14h10m45.982785662s) [upload=14602]: Done uploading segments... Aug 7 23:50:24.238753595 (14h10m45.982810442s) [upload=14602]: Upload finished. err: <nil> Aug 7 23:51:03.108197922 (14h11m24.852254772s) [upload=14612]: Starting upload Aug 7 23:51:03.108300837 (14h11m24.852357707s) [upload=14612]: Done starting upload Aug 7 23:51:03.108453294 (14h11m24.852510146s) [upload=14612]: Uploading segments... Aug 7 23:51:03.108972593 (14h11m24.853029491s) [upload=14612]: Got next segment. Inline: true Aug 7 23:51:03.108988857 (14h11m24.85304573s) [upload=14612]: Waiting for error group managing segments... Aug 7 23:51:03.109073931 (14h11m24.853130786s) [upload=14612]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:51:03.109080082 (14h11m24.853136933s) [upload=14612]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:51:03.109083894 (14h11m24.853140746s) [upload=14612]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:03.446817008 (14h11m25.190873858s) [upload=14612]: Done uploading segments... Aug 7 23:51:03.446842582 (14h11m25.190899436s) [upload=14612]: Upload finished. err: <nil> Aug 7 23:52:09.742356648 (14h12m31.486413495s) [upload=14628]: Starting upload Aug 7 23:52:09.742413553 (14h12m31.486470415s) [upload=14628]: Done starting upload Aug 7 23:52:09.742476702 (14h12m31.486533548s) [upload=14628]: Uploading segments... Aug 7 23:52:09.742511425 (14h12m31.486568294s) [upload=14628]: Got next segment. Inline: true Aug 7 23:52:09.742515305 (14h12m31.486572152s) [upload=14628]: Waiting for error group managing segments... Aug 7 23:52:09.742550700 (14h12m31.486607545s) [upload=14628]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:09.742555041 (14h12m31.486611889s) [upload=14628]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:09.742557771 (14h12m31.486614632s) [upload=14628]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:09.895389395 (14h12m31.639446255s) [upload=14628]: Done uploading segments... Aug 7 23:52:09.895421188 (14h12m31.639478039s) [upload=14628]: Upload finished. err: <nil> Aug 7 23:52:28.792030311 (14h12m50.536087163s) [upload=14635]: Starting upload Aug 7 23:52:28.792092644 (14h12m50.536149508s) [upload=14635]: Done starting upload Aug 7 23:52:28.792152087 (14h12m50.53620894s) [upload=14635]: Uploading segments... Aug 7 23:52:28.792190462 (14h12m50.536247332s) [upload=14635]: Got next segment. Inline: true Aug 7 23:52:28.792195687 (14h12m50.536252548s) [upload=14635]: Waiting for error group managing segments... Aug 7 23:52:28.792239958 (14h12m50.536296808s) [upload=14635]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:28.792245830 (14h12m50.536302691s) [upload=14635]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:28.792249165 (14h12m50.536306026s) [upload=14635]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:28.947698658 (14h12m50.691755509s) [upload=14635]: Done uploading segments... Aug 7 23:52:28.947717721 (14h12m50.691774571s) [upload=14635]: Upload finished. err: <nil> Aug 7 23:53:17.152926358 (14h13m38.896983229s) [upload=14650]: Starting upload Aug 7 23:53:17.153036870 (14h13m38.89709373s) [upload=14650]: Done starting upload Aug 7 23:53:17.153154678 (14h13m38.897211544s) [upload=14650]: Uploading segments... Aug 7 23:53:17.153233089 (14h13m38.897289957s) [upload=14650]: Got next segment. Inline: true Aug 7 23:53:17.153240256 (14h13m38.897297124s) [upload=14650]: Waiting for error group managing segments... Aug 7 23:53:17.153318892 (14h13m38.897375757s) [upload=14650]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:17.153325816 (14h13m38.89738268s) [upload=14650]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:17.153330453 (14h13m38.897387314s) [upload=14650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:16.875757982 (14h14m38.619814845s) [upload=14650]: Done uploading segments... Aug 7 23:54:16.875781849 (14h14m38.619838701s) [upload=14650]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:24.158448119 (14h14m45.90250498s) [upload=14667]: Starting upload Aug 7 23:54:24.158525946 (14h14m45.902582806s) [upload=14667]: Done starting upload Aug 7 23:54:24.158598972 (14h14m45.902655824s) [upload=14667]: Uploading segments... Aug 7 23:54:24.158654148 (14h14m45.902711017s) [upload=14667]: Got next segment. Inline: true Aug 7 23:54:24.158659995 (14h14m45.902716856s) [upload=14667]: Waiting for error group managing segments... Aug 7 23:54:24.158714896 (14h14m45.902771747s) [upload=14667]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:24.158720357 (14h14m45.902777209s) [upload=14667]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:24.158724105 (14h14m45.902780965s) [upload=14667]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:23.888400190 (14h15m45.63245704s) [upload=14667]: Done uploading segments... Aug 7 23:55:23.888419087 (14h15m45.632475938s) [upload=14667]: Upload finished. err: metaclient: context deadline exceeded Duration: 1m0.000972213s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.....XXXXXX.XX........................................X.XX..] duration: 2.220899721s FAIL (2025-08-07 23:55:21 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.220936001s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 2.428357452s FAIL (2025-08-07 23:55:19 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== 294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:32:45.172127356 (13h53m6.916184241s) [upload=14294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:32:45.390149151 (13h53m7.134206011s) [seg_pos={0 0}, upload=14294]: Begin upload segment... Aug 7 23:32:45.390215910 (13h53m7.134272761s) [seg_pos={0 0}, upload=14294]: Done begin upload segment. Aug 7 23:32:45.390249193 (13h53m7.134306059s) [noise=true , address=104.36.82.19:4405 , piece_id=MHM3MWZJFTXSJT7PTOFSWHUYO3L6NIBUV6VU66XXZDH3MC3EFR2Q, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390272919 (13h53m7.134329788s) [noise=true , address=77.90.31.12:4593 , piece_id=C3RVQHFS4L7SKPFFIZ52MAQWGD7P4JNG3SNPCNY53SQUD72PZRUA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390658970 (13h53m7.13471583s) [noise=true , address=104.36.82.16:4321 , piece_id=YTF4RWV2SD6Z4UV4XRYPBRGPCTSUTC6TQKL3DHVKU65ALTNA4TEQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390852777 (13h53m7.134909659s) [noise=true , address=104.36.82.34:4566 , piece_id=3YEAUSXEL7WNZ3O7WM24Z2DKY2TDVKMRZSHMEPMNEL2LR7BY7DBQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390957991 (13h53m7.135014851s) [noise=true , address=104.36.82.47:4523 , piece_id=MJIMHSO66V7OWFFK2UALPKSI7CGGCTHNPZINDXMVIHDLENIW5LFA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.391483291 (13h53m7.135540162s) [noise=true , address=104.36.82.5:4861 , piece_id=GRDGO5LNCETQWRU3M6LACGRW5MNA2KFLXUBT5R2UH6SEGWP6RVRQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.391931094 (13h53m7.135987964s) [noise=true , address=104.36.82.10:4090 , piece_id=V6NPPVIYEH6WLIW7OMT3NDFPR7UDWIT46GW6AFCAWUJTBLC3O55Q, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392250266 (13h53m7.136307136s) [noise=true , address=104.36.82.62:4439 , piece_id=ILRD4TEFW4PSLV4EGPORG3FM6U2HB3WSL5KP3PN566FAOGWODLYQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392611689 (13h53m7.136668559s) [noise=true , address=104.36.82.54:4160 , piece_id=GDBORRYX5QBQGH7ZMMUUAV7AAR5UHZOTK5FTQUPC7YKZC2KD7I3A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392917410 (13h53m7.136974282s) [noise=true , address=104.36.82.23:4477 , piece_id=HAF6424ZCIBGSR4US5LN5ZWE5JMNMLFGUMQQ4YP2ARP7PQXBH5GQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393270536 (13h53m7.137327398s) [noise=true , address=104.36.82.44:4881 , piece_id=XXVSXG3ILFXG4IVMELHL45TRRHDFVVJMJ5LGOHU5EDHG6XXCDH6A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393582635 (13h53m7.137639498s) [noise=true , address=104.36.82.37:4665 , piece_id=RUKKGC4Y3FIVLZO2GRWJN3J6P36VWP5VSC2HCLEU3MZDDFXODJAQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393856926 (13h53m7.137913788s) [noise=true , address=104.36.82.11:4128 , piece_id=XRPULHI7BYDF2JOIASSN5QKGEAC6FJS2F5C22WIY7EKYILRHZ73A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394196364 (13h53m7.138253227s) [noise=true , address=104.36.82.38:4706 , piece_id=BIHIPQX6LRWBNFN4SDC3K3V22AMGL3RC5B4RVE7QXJFKB4XDN3BA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394484397 (13h53m7.138541257s) [noise=true , address=104.36.82.61:4381 , piece_id=TTZOSRP6YPL6P4C6PY5S4K7Z6CIZJDPX7NUNBCK73CWZPCA3HQHA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394593709 (13h53m7.138650578s) [noise=true , address=45.91.116.5:4300 , piece_id=MADJCANN2O35MKXT2PIR5PGMJ3PNTAQAJNNGOK6ISJNVAH2L7YIA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.395093013 (13h53m7.139149879s) [noise=true , address=104.36.82.22:4530 , piece_id=HMUUW5GYH6TRHYXP4YXBHD566FWQRILEHI53AVURA5QTQGG6USFA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.395447945 (13h53m7.139504805s) [noise=true , address=104.36.82.58:4293 , piece_id=I2ETFXYTHYWYJYSHTVOBQTO6ZPGNMA4D3NVKYDE2W34EH4SQE23A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.954469844 (13h53m7.698526694s) [upload=14294]: Next returned nil segment Aug 7 23:32:45.954487953 (13h53m7.69854487s) [upload=14294]: Waiting for error group managing segments... Aug 7 23:32:46.126888627 (13h53m7.870945474s) [noise=true , address=104.36.82.10:4090 , piece_id=V6NPPVIYEH6WLIW7OMT3NDFPR7UDWIT46GW6AFCAWUJTBLC3O55Q, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.132983844 (13h53m7.877040704s) [noise=true , address=104.36.82.16:4321 , piece_id=YTF4RWV2SD6Z4UV4XRYPBRGPCTSUTC6TQKL3DHVKU65ALTNA4TEQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.138277216 (13h53m7.882334067s) [noise=true , address=104.36.82.23:4477 , piece_id=HAF6424ZCIBGSR4US5LN5ZWE5JMNMLFGUMQQ4YP2ARP7PQXBH5GQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.142033326 (13h53m7.886090185s) [noise=true , address=104.36.82.34:4566 , piece_id=3YEAUSXEL7WNZ3O7WM24Z2DKY2TDVKMRZSHMEPMNEL2LR7BY7DBQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.151247206 (13h53m7.895304058s) [noise=true , address=104.36.82.22:4530 , piece_id=HMUUW5GYH6TRHYXP4YXBHD566FWQRILEHI53AVURA5QTQGG6USFA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.151981081 (13h53m7.896037932s) [noise=true , address=104.36.82.58:4293 , piece_id=I2ETFXYTHYWYJYSHTVOBQTO6ZPGNMA4D3NVKYDE2W34EH4SQE23A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.165022882 (13h53m7.909079743s) [noise=true , address=104.36.82.44:4881 , piece_id=XXVSXG3ILFXG4IVMELHL45TRRHDFVVJMJ5LGOHU5EDHG6XXCDH6A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.184418821 (13h53m7.928475667s) [noise=true , address=104.36.82.47:4523 , piece_id=MJIMHSO66V7OWFFK2UALPKSI7CGGCTHNPZINDXMVIHDLENIW5LFA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.184532669 (13h53m7.928589516s) [noise=true , address=45.91.116.5:4300 , piece_id=MADJCANN2O35MKXT2PIR5PGMJ3PNTAQAJNNGOK6ISJNVAH2L7YIA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.193375808 (13h53m7.937432656s) [noise=true , address=104.36.82.54:4160 , piece_id=GDBORRYX5QBQGH7ZMMUUAV7AAR5UHZOTK5FTQUPC7YKZC2KD7I3A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.204247501 (13h53m7.948304372s) [noise=true , address=104.36.82.38:4706 , piece_id=BIHIPQX6LRWBNFN4SDC3K3V22AMGL3RC5B4RVE7QXJFKB4XDN3BA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.235225782 (13h53m7.979282629s) [noise=true , address=104.36.82.62:4439 , piece_id=ILRD4TEFW4PSLV4EGPORG3FM6U2HB3WSL5KP3PN566FAOGWODLYQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.269129867 (13h53m8.013186721s) [noise=true , address=104.36.82.5:4861 , piece_id=GRDGO5LNCETQWRU3M6LACGRW5MNA2KFLXUBT5R2UH6SEGWP6RVRQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.278216912 (13h53m8.022273764s) [noise=true , address=104.36.82.19:4405 , piece_id=MHM3MWZJFTXSJT7PTOFSWHUYO3L6NIBUV6VU66XXZDH3MC3EFR2Q, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.278247847 (13h53m8.022304708s) [seg_pos={0 0}, upload=14294]: Segment reached optimal threshold of 14 pieces. Aug 7 23:32:46.278485612 (13h53m8.022542463s) [noise=true , address=104.36.82.11:4128 , piece_id=XRPULHI7BYDF2JOIASSN5QKGEAC6FJS2F5C22WIY7EKYILRHZ73A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wABkAtAjoYkKyNoAiaVCxSvY6WgvHvAYFPCjpNojtpiYiGtDS): protocol: expected piece hash; context canceled; EOF Aug 7 23:32:46.278511698 (13h53m8.022568566s) [noise=true , address=77.90.31.12:4593 , piece_id=C3RVQHFS4L7SKPFFIZ52MAQWGD7P4JNG3SNPCNY53SQUD72PZRUA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kSxoZb9nmG6oGNytpFTsoiGh3ZPmebVVe8TZxGzANaw7EfdeF): protocol: expected piece hash; context canceled; EOF Aug 7 23:32:46.278642807 (13h53m8.022699657s) [noise=true , address=104.36.82.61:4381 , piece_id=TTZOSRP6YPL6P4C6PY5S4K7Z6CIZJDPX7NUNBCK73CWZPCA3HQHA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zU35MWWaaC1npkP5qGcWBXAkfxGsd5UsdkM2nwX9x7wUGCDbY): protocol: write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection; write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection Aug 7 23:32:46.278650926 (13h53m8.022707786s) [noise=true , address=104.36.82.37:4665 , piece_id=RUKKGC4Y3FIVLZO2GRWJN3J6P36VWP5VSC2HCLEU3MZDDFXODJAQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jP9JrCmoQSdqrzXZxkNN1ihdFkaUMjGZss6Fh19zbL3YyWj5N): protocol: write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection; write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection Aug 7 23:32:46.278674050 (13h53m8.022730911s) [seg_pos={0 0}, upload=14294]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:32:46.278751154 (13h53m8.022808014s) [upload=14294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:32:46.278756168 (13h53m8.022813029s) [upload=14294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:32:46.498660020 (13h53m8.242716882s) [upload=14294]: Done uploading segments... Aug 7 23:32:46.498701263 (13h53m8.242758128s) [upload=14294]: Upload finished. err: <nil> Duration: 2.428388482s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [...XXXXXXXXXX...........X............................XX.X.X.] duration: 2.48727213s FAIL (2025-08-07 23:55:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== exceeded Aug 7 23:53:43.167705717 (14h14m4.911762564s) [upload=14648]: Done uploading segments... Aug 7 23:53:43.167737922 (14h14m4.911794768s) [upload=14648]: Upload finished. err: context deadline exceeded Aug 7 23:53:43.606184070 (14h14m5.350240917s) [upload=14655]: Starting upload Aug 7 23:53:43.606251699 (14h14m5.350308565s) [upload=14655]: Done starting upload Aug 7 23:53:43.606378812 (14h14m5.350435674s) [upload=14655]: Uploading segments... Aug 7 23:53:43.606613336 (14h14m5.350670197s) [upload=14655]: Got next segment. Inline: false Aug 7 23:53:43.606652700 (14h14m5.35070955s) [upload=14655]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:43.606683541 (14h14m5.350740402s) [upload=14655]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:53:43.711705294 (14h14m5.455762157s) [seg_pos={0 0}, upload=14655]: Begin upload segment... Aug 7 23:53:43.711752671 (14h14m5.455809532s) [seg_pos={0 0}, upload=14655]: Done begin upload segment. Aug 7 23:53:43.711786432 (14h14m5.455843291s) [noise=true , address=104.36.82.10:4077 , piece_id=XOE3BBCSJ64O7A7LC7HAAYEBF3F6CIU77UVI5HY3RI2FWSXEHJXA, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.711812017 (14h14m5.455868889s) [noise=true , address=77.90.31.16:4735 , piece_id=OE7TL5NB6UH3A4JE2BDYFA4FZKHCRWT6YK6KJZHD6JJ55FOKXIEQ, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.712247313 (14h14m5.456304173s) [noise=true , address=45.91.116.10:4081 , piece_id=MMI5DUXMAD27VJI7VRQHBBEC2MJUDT5KWX2OGWLUPMTZLBIRWSAA, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.712257082 (14h14m5.456313945s) [noise=true , address=77.90.31.13:4626 , piece_id=74QG6PW6CJDJRESXUF534PQ463VRJQF33PMQRX6W6VVQCQMX2SUA, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.712579895 (14h14m5.456636755s) [noise=true , address=104.36.82.15:4258 , piece_id=JWNJIQN5WUWHVOQOQASGTGXOZWJVZZICABZCFOGZQVCDAVZ7WNVQ, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.712592145 (14h14m5.456649005s) [noise=true , address=45.91.116.4:4939 , piece_id=YLQB6XZJ6FL7SBJQ6FE4VIGKE5QYA3XPFU4E37OQKF5O2A7ZV4DQ, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.712863854 (14h14m5.456920715s) [noise=true , address=104.36.82.9:4007 , piece_id=XHRH7RHA6VKCFN7TDCMKIP6EH5EQWW3ZWVP7QE67X6LS7IR4J2MQ, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.712991802 (14h14m5.457048663s) [noise=true , address=77.90.31.13:4628 , piece_id=ZLWT3WANPKHZ6XFV7PC3KZULY73ZEOYD3E3EVY3OMIQI3OYM3LKA, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.713196040 (14h14m5.4572529s) [noise=true , address=104.36.82.14:4222 , piece_id=AAQ6HQOYMMNNHXABB2MOCCJHF43PXWJZH4SDYF6ZNSLQWMKY32AA, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.713340669 (14h14m5.457397532s) [noise=true , address=45.91.116.6:4975 , piece_id=BBULAO6EEFXHYP2AKMUF63YRSQJGVKXLIWGCKWUM5JYUVLZAB25Q, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.713490538 (14h14m5.457547399s) [noise=true , address=104.36.82.11:4114 , piece_id=IKJQ5SDEBNDZGJ6TUDTQBA2A74BQC6WV3SC4UKE7AGWEENDBI57Q, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.713730980 (14h14m5.45778784s) [noise=true , address=45.91.116.8:4045 , piece_id=7PDSFMO4MC4SMKYW3MN6OIU33MIKCLFBSJ6LYFQAGUFMOZQCLVXQ, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.713818259 (14h14m5.45787511s) [noise=true , address=77.90.31.19:4843 , piece_id=HLLIY2T4U45TYQ7L7RKY26MUTAA4KWAZ6LV7KP6HOTRJURY3WUSQ, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.714085345 (14h14m5.458142208s) [noise=true , address=45.91.116.5:4299 , piece_id=F5FM3VSMCDN3BSCGDQO5XGPZPQACVJAQG4Y2HWJEPQUF7BFGWNUA, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.714148553 (14h14m5.458205416s) [noise=true , address=77.90.31.14:4662 , piece_id=FDS6SHOXCIIUA4DKBYNMKY67RG6X5OLUKJLZ5M2M76UQB3ANWPLA, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.714360745 (14h14m5.458417608s) [noise=true , address=45.91.116.7:4010 , piece_id=RXF2GDUICLNB25V6S2CVFIKNTQHZOLWDZZWV3AEYWONFCFPZE2GA, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:44.327859573 (14h14m6.071916423s) [upload=14655]: Next returned nil segment Aug 7 23:53:44.327877823 (14h14m6.071934683s) [upload=14655]: Waiting for error group managing segments... Aug 7 23:53:44.492864996 (14h14m6.236921844s) [noise=true , address=104.36.82.11:4114 , piece_id=IKJQ5SDEBNDZGJ6TUDTQBA2A74BQC6WV3SC4UKE7AGWEENDBI57Q, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.496078923 (14h14m6.240135773s) [noise=true , address=104.36.82.9:4007 , piece_id=XHRH7RHA6VKCFN7TDCMKIP6EH5EQWW3ZWVP7QE67X6LS7IR4J2MQ, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.519818170 (14h14m6.263875019s) [noise=true , address=104.36.82.15:4258 , piece_id=JWNJIQN5WUWHVOQOQASGTGXOZWJVZZICABZCFOGZQVCDAVZ7WNVQ, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.540418125 (14h14m6.284474976s) [noise=true , address=45.91.116.7:4010 , piece_id=RXF2GDUICLNB25V6S2CVFIKNTQHZOLWDZZWV3AEYWONFCFPZE2GA, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.568753323 (14h14m6.312810176s) [noise=true , address=104.36.82.10:4077 , piece_id=XOE3BBCSJ64O7A7LC7HAAYEBF3F6CIU77UVI5HY3RI2FWSXEHJXA, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.577903595 (14h14m6.321960456s) [noise=true , address=77.90.31.14:4662 , piece_id=FDS6SHOXCIIUA4DKBYNMKY67RG6X5OLUKJLZ5M2M76UQB3ANWPLA, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.580925505 (14h14m6.324982356s) [noise=true , address=45.91.116.6:4975 , piece_id=BBULAO6EEFXHYP2AKMUF63YRSQJGVKXLIWGCKWUM5JYUVLZAB25Q, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.585723335 (14h14m6.329780193s) [noise=true , address=77.90.31.13:4628 , piece_id=ZLWT3WANPKHZ6XFV7PC3KZULY73ZEOYD3E3EVY3OMIQI3OYM3LKA, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.588082614 (14h14m6.332139464s) [noise=true , address=77.90.31.16:4735 , piece_id=OE7TL5NB6UH3A4JE2BDYFA4FZKHCRWT6YK6KJZHD6JJ55FOKXIEQ, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.626221649 (14h14m6.370278495s) [noise=true , address=45.91.116.8:4045 , piece_id=7PDSFMO4MC4SMKYW3MN6OIU33MIKCLFBSJ6LYFQAGUFMOZQCLVXQ, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.628668635 (14h14m6.372725499s) [noise=true , address=104.36.82.14:4222 , piece_id=AAQ6HQOYMMNNHXABB2MOCCJHF43PXWJZH4SDYF6ZNSLQWMKY32AA, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.636831349 (14h14m6.38088821s) [noise=true , address=77.90.31.19:4843 , piece_id=HLLIY2T4U45TYQ7L7RKY26MUTAA4KWAZ6LV7KP6HOTRJURY3WUSQ, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.645180863 (14h14m6.389237709s) [noise=true , address=45.91.116.10:4081 , piece_id=MMI5DUXMAD27VJI7VRQHBBEC2MJUDT5KWX2OGWLUPMTZLBIRWSAA, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.645221241 (14h14m6.389278101s) [seg_pos={0 0}, upload=14655]: Segment reached optimal threshold of 13 pieces. Aug 7 23:53:44.645393922 (14h14m6.389450768s) [noise=true , address=77.90.31.13:4626 , piece_id=74QG6PW6CJDJRESXUF534PQ463VRJQF33PMQRX6W6VVQCQMX2SUA, seg_pos={0 0}, upload=14655]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF Aug 7 23:53:44.645466730 (14h14m6.389523595s) [noise=true , address=45.91.116.4:4939 , piece_id=YLQB6XZJ6FL7SBJQ6FE4VIGKE5QYA3XPFU4E37OQKF5O2A7ZV4DQ, seg_pos={0 0}, upload=14655]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Aug 7 23:53:44.645496683 (14h14m6.389553539s) [noise=true , address=45.91.116.5:4299 , piece_id=F5FM3VSMCDN3BSCGDQO5XGPZPQACVJAQG4Y2HWJEPQUF7BFGWNUA, seg_pos={0 0}, upload=14655]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: expected piece hash; context canceled; EOF Aug 7 23:53:44.645513654 (14h14m6.389570515s) [seg_pos={0 0}, upload=14655]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:53:44.645605002 (14h14m6.389661865s) [upload=14655]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:53:44.645610128 (14h14m6.389666989s) [upload=14655]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:44.783671038 (14h14m6.527727892s) [upload=14655]: Done uploading segments... Aug 7 23:53:44.783719939 (14h14m6.5277768s) [upload=14655]: Upload finished. err: <nil> Aug 7 23:54:44.874516624 (14h15m6.618573476s) [upload=14672]: Starting upload Aug 7 23:54:44.874591338 (14h15m6.618648203s) [upload=14672]: Done starting upload Aug 7 23:54:44.874717241 (14h15m6.618774089s) [upload=14672]: Uploading segments... Aug 7 23:54:44.874890680 (14h15m6.61894753s) [upload=14672]: Got next segment. Inline: false Aug 7 23:54:44.874947399 (14h15m6.619004253s) [upload=14672]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:44.874953184 (14h15m6.619010035s) [upload=14672]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:55:14.597054629 (14h15m36.341111495s) [upload=14672]: Next segment err: context deadline exceeded Aug 7 23:55:14.597269046 (14h15m36.341325894s) [upload=14672]: Done uploading segments... Aug 7 23:55:14.597309239 (14h15m36.341366085s) [upload=14672]: Upload finished. err: context deadline exceeded Duration: 2.487289684s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [....XXXXXXXXXX...........X............................XX.X.X] duration: 30.001033565s FAIL (2025-08-07 23:55:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== exceeded Aug 7 23:53:43.167705717 (14h14m4.911762564s) [upload=14648]: Done uploading segments... Aug 7 23:53:43.167737922 (14h14m4.911794768s) [upload=14648]: Upload finished. err: context deadline exceeded Aug 7 23:53:43.606184070 (14h14m5.350240917s) [upload=14655]: Starting upload Aug 7 23:53:43.606251699 (14h14m5.350308565s) [upload=14655]: Done starting upload Aug 7 23:53:43.606378812 (14h14m5.350435674s) [upload=14655]: Uploading segments... Aug 7 23:53:43.606613336 (14h14m5.350670197s) [upload=14655]: Got next segment. Inline: false Aug 7 23:53:43.606652700 (14h14m5.35070955s) [upload=14655]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:43.606683541 (14h14m5.350740402s) [upload=14655]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:53:43.711705294 (14h14m5.455762157s) [seg_pos={0 0}, upload=14655]: Begin upload segment... Aug 7 23:53:43.711752671 (14h14m5.455809532s) [seg_pos={0 0}, upload=14655]: Done begin upload segment. Aug 7 23:53:43.711786432 (14h14m5.455843291s) [noise=true , address=104.36.82.10:4077 , piece_id=XOE3BBCSJ64O7A7LC7HAAYEBF3F6CIU77UVI5HY3RI2FWSXEHJXA, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.711812017 (14h14m5.455868889s) [noise=true , address=77.90.31.16:4735 , piece_id=OE7TL5NB6UH3A4JE2BDYFA4FZKHCRWT6YK6KJZHD6JJ55FOKXIEQ, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.712247313 (14h14m5.456304173s) [noise=true , address=45.91.116.10:4081 , piece_id=MMI5DUXMAD27VJI7VRQHBBEC2MJUDT5KWX2OGWLUPMTZLBIRWSAA, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.712257082 (14h14m5.456313945s) [noise=true , address=77.90.31.13:4626 , piece_id=74QG6PW6CJDJRESXUF534PQ463VRJQF33PMQRX6W6VVQCQMX2SUA, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.712579895 (14h14m5.456636755s) [noise=true , address=104.36.82.15:4258 , piece_id=JWNJIQN5WUWHVOQOQASGTGXOZWJVZZICABZCFOGZQVCDAVZ7WNVQ, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.712592145 (14h14m5.456649005s) [noise=true , address=45.91.116.4:4939 , piece_id=YLQB6XZJ6FL7SBJQ6FE4VIGKE5QYA3XPFU4E37OQKF5O2A7ZV4DQ, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.712863854 (14h14m5.456920715s) [noise=true , address=104.36.82.9:4007 , piece_id=XHRH7RHA6VKCFN7TDCMKIP6EH5EQWW3ZWVP7QE67X6LS7IR4J2MQ, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.712991802 (14h14m5.457048663s) [noise=true , address=77.90.31.13:4628 , piece_id=ZLWT3WANPKHZ6XFV7PC3KZULY73ZEOYD3E3EVY3OMIQI3OYM3LKA, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.713196040 (14h14m5.4572529s) [noise=true , address=104.36.82.14:4222 , piece_id=AAQ6HQOYMMNNHXABB2MOCCJHF43PXWJZH4SDYF6ZNSLQWMKY32AA, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.713340669 (14h14m5.457397532s) [noise=true , address=45.91.116.6:4975 , piece_id=BBULAO6EEFXHYP2AKMUF63YRSQJGVKXLIWGCKWUM5JYUVLZAB25Q, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.713490538 (14h14m5.457547399s) [noise=true , address=104.36.82.11:4114 , piece_id=IKJQ5SDEBNDZGJ6TUDTQBA2A74BQC6WV3SC4UKE7AGWEENDBI57Q, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.713730980 (14h14m5.45778784s) [noise=true , address=45.91.116.8:4045 , piece_id=7PDSFMO4MC4SMKYW3MN6OIU33MIKCLFBSJ6LYFQAGUFMOZQCLVXQ, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.713818259 (14h14m5.45787511s) [noise=true , address=77.90.31.19:4843 , piece_id=HLLIY2T4U45TYQ7L7RKY26MUTAA4KWAZ6LV7KP6HOTRJURY3WUSQ, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.714085345 (14h14m5.458142208s) [noise=true , address=45.91.116.5:4299 , piece_id=F5FM3VSMCDN3BSCGDQO5XGPZPQACVJAQG4Y2HWJEPQUF7BFGWNUA, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.714148553 (14h14m5.458205416s) [noise=true , address=77.90.31.14:4662 , piece_id=FDS6SHOXCIIUA4DKBYNMKY67RG6X5OLUKJLZ5M2M76UQB3ANWPLA, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:43.714360745 (14h14m5.458417608s) [noise=true , address=45.91.116.7:4010 , piece_id=RXF2GDUICLNB25V6S2CVFIKNTQHZOLWDZZWV3AEYWONFCFPZE2GA, seg_pos={0 0}, upload=14655]: Uploading piece... Aug 7 23:53:44.327859573 (14h14m6.071916423s) [upload=14655]: Next returned nil segment Aug 7 23:53:44.327877823 (14h14m6.071934683s) [upload=14655]: Waiting for error group managing segments... Aug 7 23:53:44.492864996 (14h14m6.236921844s) [noise=true , address=104.36.82.11:4114 , piece_id=IKJQ5SDEBNDZGJ6TUDTQBA2A74BQC6WV3SC4UKE7AGWEENDBI57Q, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.496078923 (14h14m6.240135773s) [noise=true , address=104.36.82.9:4007 , piece_id=XHRH7RHA6VKCFN7TDCMKIP6EH5EQWW3ZWVP7QE67X6LS7IR4J2MQ, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.519818170 (14h14m6.263875019s) [noise=true , address=104.36.82.15:4258 , piece_id=JWNJIQN5WUWHVOQOQASGTGXOZWJVZZICABZCFOGZQVCDAVZ7WNVQ, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.540418125 (14h14m6.284474976s) [noise=true , address=45.91.116.7:4010 , piece_id=RXF2GDUICLNB25V6S2CVFIKNTQHZOLWDZZWV3AEYWONFCFPZE2GA, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.568753323 (14h14m6.312810176s) [noise=true , address=104.36.82.10:4077 , piece_id=XOE3BBCSJ64O7A7LC7HAAYEBF3F6CIU77UVI5HY3RI2FWSXEHJXA, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.577903595 (14h14m6.321960456s) [noise=true , address=77.90.31.14:4662 , piece_id=FDS6SHOXCIIUA4DKBYNMKY67RG6X5OLUKJLZ5M2M76UQB3ANWPLA, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.580925505 (14h14m6.324982356s) [noise=true , address=45.91.116.6:4975 , piece_id=BBULAO6EEFXHYP2AKMUF63YRSQJGVKXLIWGCKWUM5JYUVLZAB25Q, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.585723335 (14h14m6.329780193s) [noise=true , address=77.90.31.13:4628 , piece_id=ZLWT3WANPKHZ6XFV7PC3KZULY73ZEOYD3E3EVY3OMIQI3OYM3LKA, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.588082614 (14h14m6.332139464s) [noise=true , address=77.90.31.16:4735 , piece_id=OE7TL5NB6UH3A4JE2BDYFA4FZKHCRWT6YK6KJZHD6JJ55FOKXIEQ, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.626221649 (14h14m6.370278495s) [noise=true , address=45.91.116.8:4045 , piece_id=7PDSFMO4MC4SMKYW3MN6OIU33MIKCLFBSJ6LYFQAGUFMOZQCLVXQ, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.628668635 (14h14m6.372725499s) [noise=true , address=104.36.82.14:4222 , piece_id=AAQ6HQOYMMNNHXABB2MOCCJHF43PXWJZH4SDYF6ZNSLQWMKY32AA, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.636831349 (14h14m6.38088821s) [noise=true , address=77.90.31.19:4843 , piece_id=HLLIY2T4U45TYQ7L7RKY26MUTAA4KWAZ6LV7KP6HOTRJURY3WUSQ, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.645180863 (14h14m6.389237709s) [noise=true , address=45.91.116.10:4081 , piece_id=MMI5DUXMAD27VJI7VRQHBBEC2MJUDT5KWX2OGWLUPMTZLBIRWSAA, seg_pos={0 0}, upload=14655]: Done uploading piece. err: <nil> Aug 7 23:53:44.645221241 (14h14m6.389278101s) [seg_pos={0 0}, upload=14655]: Segment reached optimal threshold of 13 pieces. Aug 7 23:53:44.645393922 (14h14m6.389450768s) [noise=true , address=77.90.31.13:4626 , piece_id=74QG6PW6CJDJRESXUF534PQ463VRJQF33PMQRX6W6VVQCQMX2SUA, seg_pos={0 0}, upload=14655]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF Aug 7 23:53:44.645466730 (14h14m6.389523595s) [noise=true , address=45.91.116.4:4939 , piece_id=YLQB6XZJ6FL7SBJQ6FE4VIGKE5QYA3XPFU4E37OQKF5O2A7ZV4DQ, seg_pos={0 0}, upload=14655]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Aug 7 23:53:44.645496683 (14h14m6.389553539s) [noise=true , address=45.91.116.5:4299 , piece_id=F5FM3VSMCDN3BSCGDQO5XGPZPQACVJAQG4Y2HWJEPQUF7BFGWNUA, seg_pos={0 0}, upload=14655]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: expected piece hash; context canceled; EOF Aug 7 23:53:44.645513654 (14h14m6.389570515s) [seg_pos={0 0}, upload=14655]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:53:44.645605002 (14h14m6.389661865s) [upload=14655]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:53:44.645610128 (14h14m6.389666989s) [upload=14655]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:44.783671038 (14h14m6.527727892s) [upload=14655]: Done uploading segments... Aug 7 23:53:44.783719939 (14h14m6.5277768s) [upload=14655]: Upload finished. err: <nil> Aug 7 23:54:44.874516624 (14h15m6.618573476s) [upload=14672]: Starting upload Aug 7 23:54:44.874591338 (14h15m6.618648203s) [upload=14672]: Done starting upload Aug 7 23:54:44.874717241 (14h15m6.618774089s) [upload=14672]: Uploading segments... Aug 7 23:54:44.874890680 (14h15m6.61894753s) [upload=14672]: Got next segment. Inline: false Aug 7 23:54:44.874947399 (14h15m6.619004253s) [upload=14672]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:44.874953184 (14h15m6.619010035s) [upload=14672]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:55:14.597054629 (14h15m36.341111495s) [upload=14672]: Next segment err: context deadline exceeded Aug 7 23:55:14.597269046 (14h15m36.341325894s) [upload=14672]: Done uploading segments... Aug 7 23:55:14.597309239 (14h15m36.341366085s) [upload=14672]: Upload finished. err: context deadline exceeded Duration: 30.001062051s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 846.352227ms FAIL (2025-08-07 23:55:14 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== pload=14121]: Done uploading segments... Aug 7 23:22:28.505845197 (13h42m50.249902044s) [upload=14121]: Upload finished. err: <nil> Aug 7 23:23:10.377483605 (13h43m32.121540458s) [upload=14132]: Starting upload Aug 7 23:23:10.377535635 (13h43m32.121592495s) [upload=14132]: Done starting upload Aug 7 23:23:10.377603797 (13h43m32.121660647s) [upload=14132]: Uploading segments... Aug 7 23:23:10.377638935 (13h43m32.121695804s) [upload=14132]: Got next segment. Inline: true Aug 7 23:23:10.377643594 (13h43m32.121700454s) [upload=14132]: Waiting for error group managing segments... Aug 7 23:23:10.377688252 (13h43m32.121745104s) [upload=14132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:23:10.377693427 (13h43m32.121750286s) [upload=14132]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:23:10.377696805 (13h43m32.121753666s) [upload=14132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:23:10.562260724 (13h43m32.306317573s) [upload=14132]: Done uploading segments... Aug 7 23:23:10.562279228 (13h43m32.306336078s) [upload=14132]: Upload finished. err: <nil> Aug 7 23:24:07.896404858 (13h44m29.640461721s) [upload=14147]: Starting upload Aug 7 23:24:07.896490216 (13h44m29.640547079s) [upload=14147]: Done starting upload Aug 7 23:24:07.896580488 (13h44m29.640637339s) [upload=14147]: Uploading segments... Aug 7 23:24:07.896655006 (13h44m29.640711874s) [upload=14147]: Got next segment. Inline: true Aug 7 23:24:07.896660179 (13h44m29.640717039s) [upload=14147]: Waiting for error group managing segments... Aug 7 23:24:07.896714840 (13h44m29.6407717s) [upload=14147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:24:07.896719399 (13h44m29.64077626s) [upload=14147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:24:07.896722726 (13h44m29.640779586s) [upload=14147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:24:08.048898130 (13h44m29.792954991s) [upload=14147]: Done uploading segments... Aug 7 23:24:08.048952526 (13h44m29.793009386s) [upload=14147]: Upload finished. err: <nil> Aug 7 23:24:53.794754379 (13h45m15.53881125s) [upload=14162]: Starting upload Aug 7 23:24:53.794842317 (13h45m15.538899177s) [upload=14162]: Done starting upload Aug 7 23:24:53.794935964 (13h45m15.538992814s) [upload=14162]: Uploading segments... Aug 7 23:24:53.795003752 (13h45m15.539060623s) [upload=14162]: Got next segment. Inline: true Aug 7 23:24:53.795010814 (13h45m15.539067676s) [upload=14162]: Waiting for error group managing segments... Aug 7 23:24:53.795066607 (13h45m15.539123488s) [upload=14162]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:24:53.795158449 (13h45m15.5392153s) [upload=14162]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:24:53.795163166 (13h45m15.539220026s) [upload=14162]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:24:53.946994982 (13h45m15.691051833s) [upload=14162]: Done uploading segments... Aug 7 23:24:53.947013862 (13h45m15.691070711s) [upload=14162]: Upload finished. err: <nil> Aug 7 23:25:40.029856966 (13h46m1.773913836s) [upload=14177]: Starting upload Aug 7 23:25:40.029986938 (13h46m1.774043809s) [upload=14177]: Done starting upload Aug 7 23:25:40.030076796 (13h46m1.774133642s) [upload=14177]: Uploading segments... Aug 7 23:25:40.030132567 (13h46m1.774189438s) [upload=14177]: Got next segment. Inline: true Aug 7 23:25:40.030139885 (13h46m1.774196757s) [upload=14177]: Waiting for error group managing segments... Aug 7 23:25:40.030197588 (13h46m1.774254461s) [upload=14177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:25:40.030204146 (13h46m1.774261016s) [upload=14177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:25:40.030207984 (13h46m1.77426485s) [upload=14177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:25:40.184198742 (13h46m1.928255592s) [upload=14177]: Done uploading segments... Aug 7 23:25:40.184219017 (13h46m1.928275867s) [upload=14177]: Upload finished. err: <nil> Aug 7 23:26:54.625649110 (13h47m16.369705971s) [upload=14198]: Starting upload Aug 7 23:26:54.625738563 (13h47m16.369795424s) [upload=14198]: Done starting upload Aug 7 23:26:54.625842691 (13h47m16.369899539s) [upload=14198]: Uploading segments... Aug 7 23:26:54.626016839 (13h47m16.37007371s) [upload=14198]: Got next segment. Inline: true Aug 7 23:26:54.626052112 (13h47m16.370108973s) [upload=14198]: Waiting for error group managing segments... Aug 7 23:26:54.626129780 (13h47m16.370186645s) [upload=14198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:26:54.626137392 (13h47m16.370194264s) [upload=14198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:26:54.626143182 (13h47m16.370200048s) [upload=14198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:26:54.775804171 (13h47m16.519861021s) [upload=14198]: Done uploading segments... Aug 7 23:26:54.775824249 (13h47m16.519881102s) [upload=14198]: Upload finished. err: <nil> Aug 7 23:27:32.263746490 (13h47m54.007803361s) [upload=14208]: Starting upload Aug 7 23:27:32.263833655 (13h47m54.007890501s) [upload=14208]: Done starting upload Aug 7 23:27:32.263976536 (13h47m54.008033385s) [upload=14208]: Uploading segments... Aug 7 23:27:32.264070926 (13h47m54.008127796s) [upload=14208]: Got next segment. Inline: true Aug 7 23:27:32.264080356 (13h47m54.00813733s) [upload=14208]: Waiting for error group managing segments... Aug 7 23:27:32.264154657 (13h47m54.008211526s) [upload=14208]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:27:32.264163587 (13h47m54.008220454s) [upload=14208]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:27:32.264172017 (13h47m54.008228914s) [upload=14208]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:27:32.490188559 (13h47m54.23424541s) [upload=14208]: Done uploading segments... Aug 7 23:27:32.490207326 (13h47m54.234264172s) [upload=14208]: Upload finished. err: <nil> Aug 7 23:27:57.927940477 (13h48m19.671997323s) [upload=14217]: Starting upload Aug 7 23:27:57.928016423 (13h48m19.672073283s) [upload=14217]: Done starting upload Aug 7 23:27:57.928101290 (13h48m19.672158142s) [upload=14217]: Uploading segments... Aug 7 23:27:57.928155692 (13h48m19.672212566s) [upload=14217]: Got next segment. Inline: true Aug 7 23:27:57.928162009 (13h48m19.672218869s) [upload=14217]: Waiting for error group managing segments... Aug 7 23:27:57.928213294 (13h48m19.672270145s) [upload=14217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:27:57.928219212 (13h48m19.672276074s) [upload=14217]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:27:57.928223521 (13h48m19.672280383s) [upload=14217]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:27:58.087544005 (13h48m19.831600874s) [upload=14217]: Done uploading segments... Aug 7 23:27:58.087570226 (13h48m19.831627094s) [upload=14217]: Upload finished. err: <nil> Aug 7 23:29:12.765200935 (13h49m34.509257795s) [upload=14237]: Starting upload Aug 7 23:29:12.765253106 (13h49m34.509309977s) [upload=14237]: Done starting upload Aug 7 23:29:12.765324641 (13h49m34.509381499s) [upload=14237]: Uploading segments... Aug 7 23:29:12.765374187 (13h49m34.509431056s) [upload=14237]: Got next segment. Inline: true Aug 7 23:29:12.765378947 (13h49m34.509435807s) [upload=14237]: Waiting for error group managing segments... Aug 7 23:29:12.765430859 (13h49m34.50948772s) [upload=14237]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:29:12.765438106 (13h49m34.50949497s) [upload=14237]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:29:12.765443127 (13h49m34.509499988s) [upload=14237]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:29:12.926437674 (13h49m34.670494535s) [upload=14237]: Done uploading segments... Aug 7 23:29:12.926457418 (13h49m34.670514268s) [upload=14237]: Upload finished. err: <nil> Aug 7 23:29:42.225760316 (13h50m3.969817177s) [upload=14244]: Starting upload Aug 7 23:29:42.225855063 (13h50m3.969911929s) [upload=14244]: Done starting upload Aug 7 23:29:42.225962489 (13h50m3.970019355s) [upload=14244]: Uploading segments... Aug 7 23:29:42.226078484 (13h50m3.970135331s) [upload=14244]: Got next segment. Inline: true Aug 7 23:29:42.226083996 (13h50m3.970140842s) [upload=14244]: Waiting for error group managing segments... Aug 7 23:29:42.226143067 (13h50m3.970199914s) [upload=14244]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:29:42.226148511 (13h50m3.970205371s) [upload=14244]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:29:42.226153006 (13h50m3.970209865s) [upload=14244]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:29:42.401822947 (13h50m4.14587981s) [upload=14244]: Done uploading segments... Aug 7 23:29:42.401847556 (13h50m4.145904405s) [upload=14244]: Upload finished. err: <nil> Aug 7 23:30:31.529840301 (13h50m53.273897165s) [upload=14254]: Starting upload Aug 7 23:30:31.530002531 (13h50m53.274059378s) [upload=14254]: Done starting upload Aug 7 23:30:31.530201544 (13h50m53.274258391s) [upload=14254]: Uploading segments... Aug 7 23:30:31.530298120 (13h50m53.274354985s) [upload=14254]: Got next segment. Inline: true Aug 7 23:30:31.530303457 (13h50m53.274360317s) [upload=14254]: Waiting for error group managing segments... Aug 7 23:30:31.530353721 (13h50m53.274410567s) [upload=14254]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:30:31.530358927 (13h50m53.274415788s) [upload=14254]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:30:31.530363758 (13h50m53.274420619s) [upload=14254]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:30:31.696350967 (13h50m53.440407828s) [upload=14254]: Done uploading segments... Aug 7 23:30:31.696369460 (13h50m53.440426307s) [upload=14254]: Upload finished. err: <nil> Duration: 846.379925ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 7 out of 60 is failed (threshold=3) [........X...X.XX.......................................XXX..] duration: 2.890136567s FAIL (2025-08-07 23:55:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== ={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.400485298 (14h11m59.144542144s) [noise=true , address=104.36.82.61:4387 , piece_id=N3THT2DLNTVJK72OPSHHTBG34WP53NHR7L5VC6IU5EEXW7O7EPIA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.400710393 (14h11m59.144767254s) [noise=true , address=45.91.116.14:4218 , piece_id=4JCROTVISHYY7LBCTQ637ZROGDEDIJUH222CFSKVIYDBJR7BTK3Q, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.400941379 (14h11m59.14499824s) [noise=true , address=104.36.82.23:4453 , piece_id=ZKDR6X6V3GMABOUC66VFXH33QVIEC3PQSMLLVJMMOBQIFNQVOKZA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.401251698 (14h11m59.145308559s) [noise=true , address=104.36.82.42:4808 , piece_id=HBNT2DIZC4L66S7FLKXCK5XL4JDRVXTYZD6NPYYHUBC72ZHRDFKA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.403864642 (14h11m59.147921505s) [noise=true , address=104.36.82.12:4165 , piece_id=XRWHBRGQHCBTIBJUYQMW337V3ZDR2GP67ARSQVLIX5KALUD27AWA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.404219088 (14h11m59.148275951s) [noise=true , address=104.36.82.41:4765 , piece_id=K2M5MCWY2WSDKPTBPCEL5GQ3FDFQYCFUOBW5CNJUVB72ALWM3DRA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.404507731 (14h11m59.148564592s) [noise=true , address=104.36.82.45:4909 , piece_id=CEPK5VMIAWXINNVXZIDAUQMIMO42RGHBSMC5PFECCZ7TM4BZIUUA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.404771250 (14h11m59.148828112s) [noise=true , address=79.127.201.215:4349 , piece_id=NMWPQT2OMODFL4ZPUJMYMAXPQ27KIQOUXSGKJREJGV65ZLZIBY7Q, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.405722883 (14h11m59.149779746s) [noise=true , address=104.36.82.37:4674 , piece_id=CXTRIGLDCWLTVOB3TUPG6S7VAGWBNBGPTNTWZOGJFXJA3CYQQ3EA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.405966619 (14h11m59.150023478s) [noise=true , address=104.36.82.9:4039 , piece_id=TL3QGWAW4ZHOULIJRERYSPBHH5YAA4SRYQ3SMPCZ5T6O7E6SVNUQ, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.406068528 (14h11m59.150125388s) [noise=true , address=104.36.82.62:4434 , piece_id=CQPE5HJRY2LXTHJWWMYBNDHVOR5B5MBM5DGQQ7MRWXJR6M7IKOBA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.406404119 (14h11m59.150460966s) [noise=true , address=104.36.82.60:4352 , piece_id=JZ2HOZNEYEHOFN5JT67EAVLY7EQGP3B5ENHUFRI6NJCH276P72QQ, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.406657446 (14h11m59.150714311s) [noise=true , address=104.36.82.58:4272 , piece_id=5UBUOONWVORSDNUG2D3WW7VYRK4N22D6Y3Y3GNBMHF2ECZUXDAKQ, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.983166607 (14h11m59.727223467s) [upload=14617]: Next returned nil segment Aug 7 23:51:37.983195615 (14h11m59.727252481s) [upload=14617]: Waiting for error group managing segments... Aug 7 23:51:38.130379502 (14h11m59.874436364s) [noise=true , address=104.36.82.53:4136 , piece_id=IOQFF6W5IC5AUDFV76ZVTKWUAIOVNNXK6WSCCINXXTZLFRIJWHWQ, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.151652145 (14h11m59.895708992s) [noise=true , address=104.36.82.41:4765 , piece_id=K2M5MCWY2WSDKPTBPCEL5GQ3FDFQYCFUOBW5CNJUVB72ALWM3DRA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.152381420 (14h11m59.896438274s) [noise=true , address=104.36.82.58:4272 , piece_id=5UBUOONWVORSDNUG2D3WW7VYRK4N22D6Y3Y3GNBMHF2ECZUXDAKQ, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.153368721 (14h11m59.897425572s) [noise=true , address=104.36.82.55:4205 , piece_id=HI7WZRDI3BHRFWQYMDR4Z4IHJUAYOA4B3ECS65MQKQZN2YDEVHZQ, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.166206728 (14h11m59.910263574s) [noise=true , address=104.36.82.60:4352 , piece_id=JZ2HOZNEYEHOFN5JT67EAVLY7EQGP3B5ENHUFRI6NJCH276P72QQ, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.178216859 (14h11m59.922273709s) [noise=true , address=104.36.82.42:4808 , piece_id=HBNT2DIZC4L66S7FLKXCK5XL4JDRVXTYZD6NPYYHUBC72ZHRDFKA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.191290896 (14h11m59.935347742s) [noise=true , address=45.91.116.14:4218 , piece_id=4JCROTVISHYY7LBCTQ637ZROGDEDIJUH222CFSKVIYDBJR7BTK3Q, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.192130895 (14h11m59.936187748s) [noise=true , address=104.36.82.61:4387 , piece_id=N3THT2DLNTVJK72OPSHHTBG34WP53NHR7L5VC6IU5EEXW7O7EPIA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.206383044 (14h11m59.950439898s) [noise=true , address=104.36.82.12:4165 , piece_id=XRWHBRGQHCBTIBJUYQMW337V3ZDR2GP67ARSQVLIX5KALUD27AWA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.214301438 (14h11m59.958358284s) [noise=true , address=104.36.82.9:4039 , piece_id=TL3QGWAW4ZHOULIJRERYSPBHH5YAA4SRYQ3SMPCZ5T6O7E6SVNUQ, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.313332841 (14h12m0.057389691s) [noise=true , address=104.36.82.62:4434 , piece_id=CQPE5HJRY2LXTHJWWMYBNDHVOR5B5MBM5DGQQ7MRWXJR6M7IKOBA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.369163404 (14h12m0.113220261s) [noise=true , address=104.36.82.23:4453 , piece_id=ZKDR6X6V3GMABOUC66VFXH33QVIEC3PQSMLLVJMMOBQIFNQVOKZA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.468307450 (14h12m0.21236431s) [noise=true , address=104.36.82.37:4674 , piece_id=CXTRIGLDCWLTVOB3TUPG6S7VAGWBNBGPTNTWZOGJFXJA3CYQQ3EA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.539243271 (14h12m0.283300147s) [noise=true , address=79.127.201.215:4349 , piece_id=NMWPQT2OMODFL4ZPUJMYMAXPQ27KIQOUXSGKJREJGV65ZLZIBY7Q, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.539285855 (14h12m0.283342716s) [seg_pos={0 0}, upload=14617]: Segment reached optimal threshold of 14 pieces. Aug 7 23:51:38.539484535 (14h12m0.283541382s) [noise=true , address=104.36.82.45:4909 , piece_id=CEPK5VMIAWXINNVXZIDAUQMIMO42RGHBSMC5PFECCZ7TM4BZIUUA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14Pqp5cbhmd44QnikAuD5TgsKFDR3gB2qtLuVSroaWVv4U2cDh): protocol: expected piece hash; context canceled; EOF Aug 7 23:51:38.539517293 (14h12m0.283574153s) [noise=true , address=64.49.13.138:4488 , piece_id=C3DZT3DLUGAFQTRQARWSBWVJ4I77QWHJGIQZTVX4W6LU5HABXAZA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mQqzHsTJvR5wHpEGXDixk3NcRuBKxY2cL82eJMHidcxFiPCZb): protocol: expected piece hash; context canceled; EOF Aug 7 23:51:38.539590397 (14h12m0.283647259s) [noise=true , address=104.36.82.43:4868 , piece_id=VP5ASYQLM6PAWHV4DMBVHIRSDX7WPMXORUCQQHLXQ7UE5WUNGY7Q, seg_pos={0 0}, upload=14617]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124LJtYshPwu1uZJwVyjrjUMk3htyVnp9ntnZWSUb7X3bNkwU6t): protocol: expected piece hash; context canceled; EOF Aug 7 23:51:38.539646799 (14h12m0.28370366s) [noise=true , address=64.49.13.169:4707 , piece_id=Y5MWKGWFH3LP2T42UHWOLNJTHV3SCKK2ZA7HRCBVHVDIG2UQEN7Q, seg_pos={0 0}, upload=14617]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: expected piece hash; context canceled; EOF Aug 7 23:51:38.539673304 (14h12m0.283730165s) [seg_pos={0 0}, upload=14617]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:51:38.539768774 (14h12m0.283825642s) [upload=14617]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:51:38.539775704 (14h12m0.283832576s) [upload=14617]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:17.809583811 (14h12m39.553640671s) [upload=14617]: Done uploading segments... Aug 7 23:52:17.809611159 (14h12m39.55366802s) [upload=14617]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:52:18.222005824 (14h12m39.966062686s) [upload=14632]: Starting upload Aug 7 23:52:18.222080177 (14h12m39.966137038s) [upload=14632]: Done starting upload Aug 7 23:52:18.222201874 (14h12m39.966258723s) [upload=14632]: Uploading segments... Aug 7 23:52:18.222371001 (14h12m39.966427854s) [upload=14632]: Got next segment. Inline: false Aug 7 23:52:18.222437250 (14h12m39.966494102s) [upload=14632]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:18.222442846 (14h12m39.966499706s) [upload=14632]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:53:17.949653879 (14h13m39.693710745s) [upload=14632]: Next segment err: context deadline exceeded Aug 7 23:53:17.949857332 (14h13m39.693914187s) [upload=14632]: Done uploading segments... Aug 7 23:53:17.949878048 (14h13m39.6939349s) [upload=14632]: Upload finished. err: context deadline exceeded Aug 7 23:53:18.377163292 (14h13m40.121220142s) [upload=14651]: Starting upload Aug 7 23:53:18.377258643 (14h13m40.121315509s) [upload=14651]: Done starting upload Aug 7 23:53:18.377377813 (14h13m40.121434664s) [upload=14651]: Uploading segments... Aug 7 23:53:18.377644949 (14h13m40.121701821s) [upload=14651]: Got next segment. Inline: false Aug 7 23:53:18.377703340 (14h13m40.121760202s) [upload=14651]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:18.377710506 (14h13m40.12176737s) [upload=14651]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:54:18.110025843 (14h14m39.854082712s) [upload=14651]: Next segment err: context deadline exceeded Aug 7 23:54:18.110107720 (14h14m39.854164579s) [upload=14651]: Done uploading segments... Aug 7 23:54:18.110139157 (14h14m39.854196008s) [upload=14651]: Upload finished. err: context deadline exceeded Duration: 2.890168398s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 321.820692ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 23:55:08 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 321.85035ms edge gateway vhost us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [....X.XX..X.................................................] duration: 3.041651743s FAIL (2025-08-07 23:55:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.041674661s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 6 out of 60 is failed (threshold=3) [.......XXX...XX.X...........................................] duration: 1.074050934s FAIL (2025-08-07 23:54:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.074075354s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 6 out of 60 is failed (threshold=3) [........XXX...XX.X..........................................] duration: 1.020469926s FAIL (2025-08-07 23:54:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.020494873s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 15 out of 60 is failed (threshold=3) [..XXX.XXXXXX..XX.X...................................X...XX.] duration: 868.983234ms FAIL (2025-08-07 23:54:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7 23:46:47.210142995 (14h7m8.954199841s) [upload=14538]: Upload finished. err: <nil> Aug 7 23:47:29.298549723 (14h7m51.042606575s) [upload=14552]: Starting upload Aug 7 23:47:29.298617898 (14h7m51.042674767s) [upload=14552]: Done starting upload Aug 7 23:47:29.298698678 (14h7m51.042755528s) [upload=14552]: Uploading segments... Aug 7 23:47:29.298753306 (14h7m51.042810175s) [upload=14552]: Got next segment. Inline: true Aug 7 23:47:29.298758297 (14h7m51.042815158s) [upload=14552]: Waiting for error group managing segments... Aug 7 23:47:29.298818439 (14h7m51.042875289s) [upload=14552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:47:29.298823844 (14h7m51.042880694s) [upload=14552]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:47:29.298827075 (14h7m51.042883925s) [upload=14552]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:47:29.457296385 (14h7m51.201353233s) [upload=14552]: Done uploading segments... Aug 7 23:47:29.457315555 (14h7m51.201372401s) [upload=14552]: Upload finished. err: <nil> Aug 7 23:48:42.140972885 (14h9m3.885029736s) [upload=14571]: Starting upload Aug 7 23:48:42.141036695 (14h9m3.885093558s) [upload=14571]: Done starting upload Aug 7 23:48:42.141117369 (14h9m3.88517422s) [upload=14571]: Uploading segments... Aug 7 23:48:42.141203219 (14h9m3.885260088s) [upload=14571]: Got next segment. Inline: true Aug 7 23:48:42.141210076 (14h9m3.885266937s) [upload=14571]: Waiting for error group managing segments... Aug 7 23:48:42.141288577 (14h9m3.885345438s) [upload=14571]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:48:42.141294246 (14h9m3.885351107s) [upload=14571]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:48:42.141297649 (14h9m3.885354509s) [upload=14571]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:48:42.292290080 (14h9m4.036346927s) [upload=14571]: Done uploading segments... Aug 7 23:48:42.292309333 (14h9m4.036366179s) [upload=14571]: Upload finished. err: <nil> Aug 7 23:49:47.788743937 (14h10m9.532800783s) [upload=14591]: Starting upload Aug 7 23:49:47.788822734 (14h10m9.5328796s) [upload=14591]: Done starting upload Aug 7 23:49:47.788889290 (14h10m9.532946138s) [upload=14591]: Uploading segments... Aug 7 23:49:47.788950938 (14h10m9.533007798s) [upload=14591]: Got next segment. Inline: true Aug 7 23:49:47.788955912 (14h10m9.533012775s) [upload=14591]: Waiting for error group managing segments... Aug 7 23:49:47.789005080 (14h10m9.533061926s) [upload=14591]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:49:47.789009334 (14h10m9.533066194s) [upload=14591]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:49:47.789014285 (14h10m9.53307113s) [upload=14591]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:49:47.932333015 (14h10m9.676389863s) [upload=14591]: Done uploading segments... Aug 7 23:49:47.932362673 (14h10m9.676419519s) [upload=14591]: Upload finished. err: <nil> Aug 7 23:50:23.531847225 (14h10m45.275904072s) [upload=14601]: Starting upload Aug 7 23:50:23.531926184 (14h10m45.275983047s) [upload=14601]: Done starting upload Aug 7 23:50:23.531993110 (14h10m45.27604996s) [upload=14601]: Uploading segments... Aug 7 23:50:23.532035359 (14h10m45.276092227s) [upload=14601]: Got next segment. Inline: true Aug 7 23:50:23.532041373 (14h10m45.276098223s) [upload=14601]: Waiting for error group managing segments... Aug 7 23:50:23.532083061 (14h10m45.276139912s) [upload=14601]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:23.532088436 (14h10m45.276145285s) [upload=14601]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:23.532091688 (14h10m45.276148539s) [upload=14601]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:23.673405160 (14h10m45.417462011s) [upload=14601]: Done uploading segments... Aug 7 23:50:23.673423409 (14h10m45.417480259s) [upload=14601]: Upload finished. err: <nil> Aug 7 23:51:32.204614635 (14h11m53.948671502s) [upload=14620]: Starting upload Aug 7 23:51:32.204700450 (14h11m53.948757313s) [upload=14620]: Done starting upload Aug 7 23:51:32.204781575 (14h11m53.948838427s) [upload=14620]: Uploading segments... Aug 7 23:51:32.204839741 (14h11m53.94889661s) [upload=14620]: Got next segment. Inline: true Aug 7 23:51:32.204847624 (14h11m53.948904474s) [upload=14620]: Waiting for error group managing segments... Aug 7 23:51:32.204902472 (14h11m53.948959327s) [upload=14620]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:51:32.204907775 (14h11m53.948964625s) [upload=14620]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:51:32.204911177 (14h11m53.948968027s) [upload=14620]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:01.937570352 (14h12m23.681627223s) [upload=14620]: Done uploading segments... Aug 7 23:52:01.937615027 (14h12m23.681671888s) [upload=14620]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:52:03.484443714 (14h12m25.228500575s) [upload=14625]: Starting upload Aug 7 23:52:03.484557174 (14h12m25.228614037s) [upload=14625]: Done starting upload Aug 7 23:52:03.484649660 (14h12m25.22870651s) [upload=14625]: Uploading segments... Aug 7 23:52:03.484705970 (14h12m25.22876283s) [upload=14625]: Got next segment. Inline: true Aug 7 23:52:03.484711540 (14h12m25.228768402s) [upload=14625]: Waiting for error group managing segments... Aug 7 23:52:03.484758149 (14h12m25.228815009s) [upload=14625]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:03.484763263 (14h12m25.228820122s) [upload=14625]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:03.484766746 (14h12m25.228823606s) [upload=14625]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:03.643631412 (14h12m25.387688267s) [upload=14625]: Done uploading segments... Aug 7 23:52:03.643670624 (14h12m25.387727482s) [upload=14625]: Upload finished. err: <nil> Aug 7 23:52:25.641252610 (14h12m47.385309456s) [upload=14634]: Starting upload Aug 7 23:52:25.641347708 (14h12m47.385404571s) [upload=14634]: Done starting upload Aug 7 23:52:25.641412738 (14h12m47.385469584s) [upload=14634]: Uploading segments... Aug 7 23:52:25.641487429 (14h12m47.385544276s) [upload=14634]: Got next segment. Inline: true Aug 7 23:52:25.641494756 (14h12m47.385551602s) [upload=14634]: Waiting for error group managing segments... Aug 7 23:52:25.641535502 (14h12m47.385592365s) [upload=14634]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:25.641556088 (14h12m47.385612953s) [upload=14634]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:25.641559364 (14h12m47.385616224s) [upload=14634]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:25.852337418 (14h12m47.596394279s) [upload=14634]: Done uploading segments... Aug 7 23:52:25.852355736 (14h12m47.596412584s) [upload=14634]: Upload finished. err: <nil> Aug 7 23:52:51.292140355 (14h13m13.036197215s) [upload=14641]: Starting upload Aug 7 23:52:51.292216133 (14h13m13.036272985s) [upload=14641]: Done starting upload Aug 7 23:52:51.292307446 (14h13m13.036364307s) [upload=14641]: Uploading segments... Aug 7 23:52:51.292389087 (14h13m13.036445956s) [upload=14641]: Got next segment. Inline: true Aug 7 23:52:51.292394933 (14h13m13.036451793s) [upload=14641]: Waiting for error group managing segments... Aug 7 23:52:51.292456032 (14h13m13.036512883s) [upload=14641]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:51.292461787 (14h13m13.036518647s) [upload=14641]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:51.292467676 (14h13m13.036524537s) [upload=14641]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:51.436915828 (14h13m13.180972678s) [upload=14641]: Done uploading segments... Aug 7 23:52:51.436934458 (14h13m13.18099131s) [upload=14641]: Upload finished. err: <nil> Aug 7 23:53:10.946261153 (14h13m32.690318028s) [upload=14646]: Starting upload Aug 7 23:53:10.946350198 (14h13m32.690407065s) [upload=14646]: Done starting upload Aug 7 23:53:10.946435423 (14h13m32.690492276s) [upload=14646]: Uploading segments... Aug 7 23:53:10.946489255 (14h13m32.690546124s) [upload=14646]: Got next segment. Inline: true Aug 7 23:53:10.946497660 (14h13m32.690554519s) [upload=14646]: Waiting for error group managing segments... Aug 7 23:53:10.946542678 (14h13m32.690599529s) [upload=14646]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:10.946548267 (14h13m32.690605128s) [upload=14646]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:10.946553792 (14h13m32.690610652s) [upload=14646]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:14.554651436 (14h13m36.298708309s) [upload=14646]: Done uploading segments... Aug 7 23:53:14.554675183 (14h13m36.298732045s) [upload=14646]: Upload finished. err: <nil> Aug 7 23:53:53.584502029 (14h14m15.328558897s) [upload=14658]: Starting upload Aug 7 23:53:53.584599757 (14h14m15.328656616s) [upload=14658]: Done starting upload Aug 7 23:53:53.584702364 (14h14m15.328759218s) [upload=14658]: Uploading segments... Aug 7 23:53:53.584798610 (14h14m15.32885548s) [upload=14658]: Got next segment. Inline: true Aug 7 23:53:53.584806594 (14h14m15.328863461s) [upload=14658]: Waiting for error group managing segments... Aug 7 23:53:53.584875193 (14h14m15.328932062s) [upload=14658]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:53.584882324 (14h14m15.328939193s) [upload=14658]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:53.584901657 (14h14m15.328958528s) [upload=14658]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:23.318821682 (14h14m45.062878542s) [upload=14658]: Done uploading segments... Aug 7 23:54:23.318863033 (14h14m45.062919884s) [upload=14658]: Upload finished. err: metaclient: context deadline exceeded Duration: 869.014519ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 7 out of 60 is failed (threshold=3) [.........X...X.XX.......................................XXX.] duration: 2.521190484s FAIL (2025-08-07 23:54:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== ={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.400485298 (14h11m59.144542144s) [noise=true , address=104.36.82.61:4387 , piece_id=N3THT2DLNTVJK72OPSHHTBG34WP53NHR7L5VC6IU5EEXW7O7EPIA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.400710393 (14h11m59.144767254s) [noise=true , address=45.91.116.14:4218 , piece_id=4JCROTVISHYY7LBCTQ637ZROGDEDIJUH222CFSKVIYDBJR7BTK3Q, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.400941379 (14h11m59.14499824s) [noise=true , address=104.36.82.23:4453 , piece_id=ZKDR6X6V3GMABOUC66VFXH33QVIEC3PQSMLLVJMMOBQIFNQVOKZA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.401251698 (14h11m59.145308559s) [noise=true , address=104.36.82.42:4808 , piece_id=HBNT2DIZC4L66S7FLKXCK5XL4JDRVXTYZD6NPYYHUBC72ZHRDFKA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.403864642 (14h11m59.147921505s) [noise=true , address=104.36.82.12:4165 , piece_id=XRWHBRGQHCBTIBJUYQMW337V3ZDR2GP67ARSQVLIX5KALUD27AWA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.404219088 (14h11m59.148275951s) [noise=true , address=104.36.82.41:4765 , piece_id=K2M5MCWY2WSDKPTBPCEL5GQ3FDFQYCFUOBW5CNJUVB72ALWM3DRA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.404507731 (14h11m59.148564592s) [noise=true , address=104.36.82.45:4909 , piece_id=CEPK5VMIAWXINNVXZIDAUQMIMO42RGHBSMC5PFECCZ7TM4BZIUUA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.404771250 (14h11m59.148828112s) [noise=true , address=79.127.201.215:4349 , piece_id=NMWPQT2OMODFL4ZPUJMYMAXPQ27KIQOUXSGKJREJGV65ZLZIBY7Q, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.405722883 (14h11m59.149779746s) [noise=true , address=104.36.82.37:4674 , piece_id=CXTRIGLDCWLTVOB3TUPG6S7VAGWBNBGPTNTWZOGJFXJA3CYQQ3EA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.405966619 (14h11m59.150023478s) [noise=true , address=104.36.82.9:4039 , piece_id=TL3QGWAW4ZHOULIJRERYSPBHH5YAA4SRYQ3SMPCZ5T6O7E6SVNUQ, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.406068528 (14h11m59.150125388s) [noise=true , address=104.36.82.62:4434 , piece_id=CQPE5HJRY2LXTHJWWMYBNDHVOR5B5MBM5DGQQ7MRWXJR6M7IKOBA, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.406404119 (14h11m59.150460966s) [noise=true , address=104.36.82.60:4352 , piece_id=JZ2HOZNEYEHOFN5JT67EAVLY7EQGP3B5ENHUFRI6NJCH276P72QQ, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.406657446 (14h11m59.150714311s) [noise=true , address=104.36.82.58:4272 , piece_id=5UBUOONWVORSDNUG2D3WW7VYRK4N22D6Y3Y3GNBMHF2ECZUXDAKQ, seg_pos={0 0}, upload=14617]: Uploading piece... Aug 7 23:51:37.983166607 (14h11m59.727223467s) [upload=14617]: Next returned nil segment Aug 7 23:51:37.983195615 (14h11m59.727252481s) [upload=14617]: Waiting for error group managing segments... Aug 7 23:51:38.130379502 (14h11m59.874436364s) [noise=true , address=104.36.82.53:4136 , piece_id=IOQFF6W5IC5AUDFV76ZVTKWUAIOVNNXK6WSCCINXXTZLFRIJWHWQ, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.151652145 (14h11m59.895708992s) [noise=true , address=104.36.82.41:4765 , piece_id=K2M5MCWY2WSDKPTBPCEL5GQ3FDFQYCFUOBW5CNJUVB72ALWM3DRA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.152381420 (14h11m59.896438274s) [noise=true , address=104.36.82.58:4272 , piece_id=5UBUOONWVORSDNUG2D3WW7VYRK4N22D6Y3Y3GNBMHF2ECZUXDAKQ, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.153368721 (14h11m59.897425572s) [noise=true , address=104.36.82.55:4205 , piece_id=HI7WZRDI3BHRFWQYMDR4Z4IHJUAYOA4B3ECS65MQKQZN2YDEVHZQ, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.166206728 (14h11m59.910263574s) [noise=true , address=104.36.82.60:4352 , piece_id=JZ2HOZNEYEHOFN5JT67EAVLY7EQGP3B5ENHUFRI6NJCH276P72QQ, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.178216859 (14h11m59.922273709s) [noise=true , address=104.36.82.42:4808 , piece_id=HBNT2DIZC4L66S7FLKXCK5XL4JDRVXTYZD6NPYYHUBC72ZHRDFKA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.191290896 (14h11m59.935347742s) [noise=true , address=45.91.116.14:4218 , piece_id=4JCROTVISHYY7LBCTQ637ZROGDEDIJUH222CFSKVIYDBJR7BTK3Q, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.192130895 (14h11m59.936187748s) [noise=true , address=104.36.82.61:4387 , piece_id=N3THT2DLNTVJK72OPSHHTBG34WP53NHR7L5VC6IU5EEXW7O7EPIA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.206383044 (14h11m59.950439898s) [noise=true , address=104.36.82.12:4165 , piece_id=XRWHBRGQHCBTIBJUYQMW337V3ZDR2GP67ARSQVLIX5KALUD27AWA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.214301438 (14h11m59.958358284s) [noise=true , address=104.36.82.9:4039 , piece_id=TL3QGWAW4ZHOULIJRERYSPBHH5YAA4SRYQ3SMPCZ5T6O7E6SVNUQ, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.313332841 (14h12m0.057389691s) [noise=true , address=104.36.82.62:4434 , piece_id=CQPE5HJRY2LXTHJWWMYBNDHVOR5B5MBM5DGQQ7MRWXJR6M7IKOBA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.369163404 (14h12m0.113220261s) [noise=true , address=104.36.82.23:4453 , piece_id=ZKDR6X6V3GMABOUC66VFXH33QVIEC3PQSMLLVJMMOBQIFNQVOKZA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.468307450 (14h12m0.21236431s) [noise=true , address=104.36.82.37:4674 , piece_id=CXTRIGLDCWLTVOB3TUPG6S7VAGWBNBGPTNTWZOGJFXJA3CYQQ3EA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.539243271 (14h12m0.283300147s) [noise=true , address=79.127.201.215:4349 , piece_id=NMWPQT2OMODFL4ZPUJMYMAXPQ27KIQOUXSGKJREJGV65ZLZIBY7Q, seg_pos={0 0}, upload=14617]: Done uploading piece. err: <nil> Aug 7 23:51:38.539285855 (14h12m0.283342716s) [seg_pos={0 0}, upload=14617]: Segment reached optimal threshold of 14 pieces. Aug 7 23:51:38.539484535 (14h12m0.283541382s) [noise=true , address=104.36.82.45:4909 , piece_id=CEPK5VMIAWXINNVXZIDAUQMIMO42RGHBSMC5PFECCZ7TM4BZIUUA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14Pqp5cbhmd44QnikAuD5TgsKFDR3gB2qtLuVSroaWVv4U2cDh): protocol: expected piece hash; context canceled; EOF Aug 7 23:51:38.539517293 (14h12m0.283574153s) [noise=true , address=64.49.13.138:4488 , piece_id=C3DZT3DLUGAFQTRQARWSBWVJ4I77QWHJGIQZTVX4W6LU5HABXAZA, seg_pos={0 0}, upload=14617]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mQqzHsTJvR5wHpEGXDixk3NcRuBKxY2cL82eJMHidcxFiPCZb): protocol: expected piece hash; context canceled; EOF Aug 7 23:51:38.539590397 (14h12m0.283647259s) [noise=true , address=104.36.82.43:4868 , piece_id=VP5ASYQLM6PAWHV4DMBVHIRSDX7WPMXORUCQQHLXQ7UE5WUNGY7Q, seg_pos={0 0}, upload=14617]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124LJtYshPwu1uZJwVyjrjUMk3htyVnp9ntnZWSUb7X3bNkwU6t): protocol: expected piece hash; context canceled; EOF Aug 7 23:51:38.539646799 (14h12m0.28370366s) [noise=true , address=64.49.13.169:4707 , piece_id=Y5MWKGWFH3LP2T42UHWOLNJTHV3SCKK2ZA7HRCBVHVDIG2UQEN7Q, seg_pos={0 0}, upload=14617]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: expected piece hash; context canceled; EOF Aug 7 23:51:38.539673304 (14h12m0.283730165s) [seg_pos={0 0}, upload=14617]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:51:38.539768774 (14h12m0.283825642s) [upload=14617]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:51:38.539775704 (14h12m0.283832576s) [upload=14617]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:17.809583811 (14h12m39.553640671s) [upload=14617]: Done uploading segments... Aug 7 23:52:17.809611159 (14h12m39.55366802s) [upload=14617]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:52:18.222005824 (14h12m39.966062686s) [upload=14632]: Starting upload Aug 7 23:52:18.222080177 (14h12m39.966137038s) [upload=14632]: Done starting upload Aug 7 23:52:18.222201874 (14h12m39.966258723s) [upload=14632]: Uploading segments... Aug 7 23:52:18.222371001 (14h12m39.966427854s) [upload=14632]: Got next segment. Inline: false Aug 7 23:52:18.222437250 (14h12m39.966494102s) [upload=14632]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:18.222442846 (14h12m39.966499706s) [upload=14632]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:53:17.949653879 (14h13m39.693710745s) [upload=14632]: Next segment err: context deadline exceeded Aug 7 23:53:17.949857332 (14h13m39.693914187s) [upload=14632]: Done uploading segments... Aug 7 23:53:17.949878048 (14h13m39.6939349s) [upload=14632]: Upload finished. err: context deadline exceeded Aug 7 23:53:18.377163292 (14h13m40.121220142s) [upload=14651]: Starting upload Aug 7 23:53:18.377258643 (14h13m40.121315509s) [upload=14651]: Done starting upload Aug 7 23:53:18.377377813 (14h13m40.121434664s) [upload=14651]: Uploading segments... Aug 7 23:53:18.377644949 (14h13m40.121701821s) [upload=14651]: Got next segment. Inline: false Aug 7 23:53:18.377703340 (14h13m40.121760202s) [upload=14651]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:18.377710506 (14h13m40.12176737s) [upload=14651]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:54:18.110025843 (14h14m39.854082712s) [upload=14651]: Next segment err: context deadline exceeded Aug 7 23:54:18.110107720 (14h14m39.854164579s) [upload=14651]: Done uploading segments... Aug 7 23:54:18.110139157 (14h14m39.854196008s) [upload=14651]: Upload finished. err: context deadline exceeded Duration: 2.521217358s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)