Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [........X.XX..X.............................................] duration: 3.007655047s FAIL (2025-08-07 23:50:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.007689413s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 6 out of 60 is failed (threshold=3) [.............XXX...XX.X.....................................] duration: 1.099670471s FAIL (2025-08-07 23:50:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.099697367s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..X..................................X......................] duration: 2.792425041s FAIL (2025-08-07 23:50:45 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== 294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:32:45.172127356 (13h53m6.916184241s) [upload=14294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:32:45.390149151 (13h53m7.134206011s) [seg_pos={0 0}, upload=14294]: Begin upload segment... Aug 7 23:32:45.390215910 (13h53m7.134272761s) [seg_pos={0 0}, upload=14294]: Done begin upload segment. Aug 7 23:32:45.390249193 (13h53m7.134306059s) [noise=true , address=104.36.82.19:4405 , piece_id=MHM3MWZJFTXSJT7PTOFSWHUYO3L6NIBUV6VU66XXZDH3MC3EFR2Q, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390272919 (13h53m7.134329788s) [noise=true , address=77.90.31.12:4593 , piece_id=C3RVQHFS4L7SKPFFIZ52MAQWGD7P4JNG3SNPCNY53SQUD72PZRUA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390658970 (13h53m7.13471583s) [noise=true , address=104.36.82.16:4321 , piece_id=YTF4RWV2SD6Z4UV4XRYPBRGPCTSUTC6TQKL3DHVKU65ALTNA4TEQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390852777 (13h53m7.134909659s) [noise=true , address=104.36.82.34:4566 , piece_id=3YEAUSXEL7WNZ3O7WM24Z2DKY2TDVKMRZSHMEPMNEL2LR7BY7DBQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390957991 (13h53m7.135014851s) [noise=true , address=104.36.82.47:4523 , piece_id=MJIMHSO66V7OWFFK2UALPKSI7CGGCTHNPZINDXMVIHDLENIW5LFA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.391483291 (13h53m7.135540162s) [noise=true , address=104.36.82.5:4861 , piece_id=GRDGO5LNCETQWRU3M6LACGRW5MNA2KFLXUBT5R2UH6SEGWP6RVRQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.391931094 (13h53m7.135987964s) [noise=true , address=104.36.82.10:4090 , piece_id=V6NPPVIYEH6WLIW7OMT3NDFPR7UDWIT46GW6AFCAWUJTBLC3O55Q, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392250266 (13h53m7.136307136s) [noise=true , address=104.36.82.62:4439 , piece_id=ILRD4TEFW4PSLV4EGPORG3FM6U2HB3WSL5KP3PN566FAOGWODLYQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392611689 (13h53m7.136668559s) [noise=true , address=104.36.82.54:4160 , piece_id=GDBORRYX5QBQGH7ZMMUUAV7AAR5UHZOTK5FTQUPC7YKZC2KD7I3A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392917410 (13h53m7.136974282s) [noise=true , address=104.36.82.23:4477 , piece_id=HAF6424ZCIBGSR4US5LN5ZWE5JMNMLFGUMQQ4YP2ARP7PQXBH5GQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393270536 (13h53m7.137327398s) [noise=true , address=104.36.82.44:4881 , piece_id=XXVSXG3ILFXG4IVMELHL45TRRHDFVVJMJ5LGOHU5EDHG6XXCDH6A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393582635 (13h53m7.137639498s) [noise=true , address=104.36.82.37:4665 , piece_id=RUKKGC4Y3FIVLZO2GRWJN3J6P36VWP5VSC2HCLEU3MZDDFXODJAQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393856926 (13h53m7.137913788s) [noise=true , address=104.36.82.11:4128 , piece_id=XRPULHI7BYDF2JOIASSN5QKGEAC6FJS2F5C22WIY7EKYILRHZ73A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394196364 (13h53m7.138253227s) [noise=true , address=104.36.82.38:4706 , piece_id=BIHIPQX6LRWBNFN4SDC3K3V22AMGL3RC5B4RVE7QXJFKB4XDN3BA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394484397 (13h53m7.138541257s) [noise=true , address=104.36.82.61:4381 , piece_id=TTZOSRP6YPL6P4C6PY5S4K7Z6CIZJDPX7NUNBCK73CWZPCA3HQHA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394593709 (13h53m7.138650578s) [noise=true , address=45.91.116.5:4300 , piece_id=MADJCANN2O35MKXT2PIR5PGMJ3PNTAQAJNNGOK6ISJNVAH2L7YIA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.395093013 (13h53m7.139149879s) [noise=true , address=104.36.82.22:4530 , piece_id=HMUUW5GYH6TRHYXP4YXBHD566FWQRILEHI53AVURA5QTQGG6USFA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.395447945 (13h53m7.139504805s) [noise=true , address=104.36.82.58:4293 , piece_id=I2ETFXYTHYWYJYSHTVOBQTO6ZPGNMA4D3NVKYDE2W34EH4SQE23A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.954469844 (13h53m7.698526694s) [upload=14294]: Next returned nil segment Aug 7 23:32:45.954487953 (13h53m7.69854487s) [upload=14294]: Waiting for error group managing segments... Aug 7 23:32:46.126888627 (13h53m7.870945474s) [noise=true , address=104.36.82.10:4090 , piece_id=V6NPPVIYEH6WLIW7OMT3NDFPR7UDWIT46GW6AFCAWUJTBLC3O55Q, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.132983844 (13h53m7.877040704s) [noise=true , address=104.36.82.16:4321 , piece_id=YTF4RWV2SD6Z4UV4XRYPBRGPCTSUTC6TQKL3DHVKU65ALTNA4TEQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.138277216 (13h53m7.882334067s) [noise=true , address=104.36.82.23:4477 , piece_id=HAF6424ZCIBGSR4US5LN5ZWE5JMNMLFGUMQQ4YP2ARP7PQXBH5GQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.142033326 (13h53m7.886090185s) [noise=true , address=104.36.82.34:4566 , piece_id=3YEAUSXEL7WNZ3O7WM24Z2DKY2TDVKMRZSHMEPMNEL2LR7BY7DBQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.151247206 (13h53m7.895304058s) [noise=true , address=104.36.82.22:4530 , piece_id=HMUUW5GYH6TRHYXP4YXBHD566FWQRILEHI53AVURA5QTQGG6USFA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.151981081 (13h53m7.896037932s) [noise=true , address=104.36.82.58:4293 , piece_id=I2ETFXYTHYWYJYSHTVOBQTO6ZPGNMA4D3NVKYDE2W34EH4SQE23A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.165022882 (13h53m7.909079743s) [noise=true , address=104.36.82.44:4881 , piece_id=XXVSXG3ILFXG4IVMELHL45TRRHDFVVJMJ5LGOHU5EDHG6XXCDH6A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.184418821 (13h53m7.928475667s) [noise=true , address=104.36.82.47:4523 , piece_id=MJIMHSO66V7OWFFK2UALPKSI7CGGCTHNPZINDXMVIHDLENIW5LFA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.184532669 (13h53m7.928589516s) [noise=true , address=45.91.116.5:4300 , piece_id=MADJCANN2O35MKXT2PIR5PGMJ3PNTAQAJNNGOK6ISJNVAH2L7YIA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.193375808 (13h53m7.937432656s) [noise=true , address=104.36.82.54:4160 , piece_id=GDBORRYX5QBQGH7ZMMUUAV7AAR5UHZOTK5FTQUPC7YKZC2KD7I3A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.204247501 (13h53m7.948304372s) [noise=true , address=104.36.82.38:4706 , piece_id=BIHIPQX6LRWBNFN4SDC3K3V22AMGL3RC5B4RVE7QXJFKB4XDN3BA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.235225782 (13h53m7.979282629s) [noise=true , address=104.36.82.62:4439 , piece_id=ILRD4TEFW4PSLV4EGPORG3FM6U2HB3WSL5KP3PN566FAOGWODLYQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.269129867 (13h53m8.013186721s) [noise=true , address=104.36.82.5:4861 , piece_id=GRDGO5LNCETQWRU3M6LACGRW5MNA2KFLXUBT5R2UH6SEGWP6RVRQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.278216912 (13h53m8.022273764s) [noise=true , address=104.36.82.19:4405 , piece_id=MHM3MWZJFTXSJT7PTOFSWHUYO3L6NIBUV6VU66XXZDH3MC3EFR2Q, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.278247847 (13h53m8.022304708s) [seg_pos={0 0}, upload=14294]: Segment reached optimal threshold of 14 pieces. Aug 7 23:32:46.278485612 (13h53m8.022542463s) [noise=true , address=104.36.82.11:4128 , piece_id=XRPULHI7BYDF2JOIASSN5QKGEAC6FJS2F5C22WIY7EKYILRHZ73A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wABkAtAjoYkKyNoAiaVCxSvY6WgvHvAYFPCjpNojtpiYiGtDS): protocol: expected piece hash; context canceled; EOF Aug 7 23:32:46.278511698 (13h53m8.022568566s) [noise=true , address=77.90.31.12:4593 , piece_id=C3RVQHFS4L7SKPFFIZ52MAQWGD7P4JNG3SNPCNY53SQUD72PZRUA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kSxoZb9nmG6oGNytpFTsoiGh3ZPmebVVe8TZxGzANaw7EfdeF): protocol: expected piece hash; context canceled; EOF Aug 7 23:32:46.278642807 (13h53m8.022699657s) [noise=true , address=104.36.82.61:4381 , piece_id=TTZOSRP6YPL6P4C6PY5S4K7Z6CIZJDPX7NUNBCK73CWZPCA3HQHA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zU35MWWaaC1npkP5qGcWBXAkfxGsd5UsdkM2nwX9x7wUGCDbY): protocol: write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection; write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection Aug 7 23:32:46.278650926 (13h53m8.022707786s) [noise=true , address=104.36.82.37:4665 , piece_id=RUKKGC4Y3FIVLZO2GRWJN3J6P36VWP5VSC2HCLEU3MZDDFXODJAQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jP9JrCmoQSdqrzXZxkNN1ihdFkaUMjGZss6Fh19zbL3YyWj5N): protocol: write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection; write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection Aug 7 23:32:46.278674050 (13h53m8.022730911s) [seg_pos={0 0}, upload=14294]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:32:46.278751154 (13h53m8.022808014s) [upload=14294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:32:46.278756168 (13h53m8.022813029s) [upload=14294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:32:46.498660020 (13h53m8.242716882s) [upload=14294]: Done uploading segments... Aug 7 23:32:46.498701263 (13h53m8.242758128s) [upload=14294]: Upload finished. err: <nil> Duration: 2.792444778s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 8 out of 60 is failed (threshold=3) [...........XXXXXX.XX........................................] duration: 2.201944161s FAIL (2025-08-07 23:50:43 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.201976541s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.............X...X.XX.......................................] duration: 3.472724599s FAIL (2025-08-07 23:50:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== inished. err: context deadline exceeded Aug 7 23:20:07.202369543 (13h40m28.946426394s) [upload=14082]: Starting upload Aug 7 23:20:07.202426657 (13h40m28.946483509s) [upload=14082]: Done starting upload Aug 7 23:20:07.202696950 (13h40m28.946753814s) [upload=14082]: Uploading segments... Aug 7 23:20:07.202761256 (13h40m28.94681811s) [upload=14082]: Got next segment. Inline: false Aug 7 23:20:07.202807046 (13h40m28.946863908s) [upload=14082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:20:07.202814233 (13h40m28.946871102s) [upload=14082]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:20:45.722093118 (13h41m7.466149983s) [seg_pos={0 0}, upload=14082]: Begin upload segment... Aug 7 23:20:45.722177165 (13h41m7.466234018s) [seg_pos={0 0}, upload=14082]: Done begin upload segment. Aug 7 23:20:45.722218538 (13h41m7.4662754s) [noise=true , address=79.127.205.246:4276 , piece_id=6753PO7TQBULISFOSTSIT32I7DSKQPM6M4TEVF2E7TILSMUU5ARQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.722246298 (13h41m7.46630316s) [noise=true , address=104.36.82.58:4292 , piece_id=KQ2RNSZDMCU7OED4IJC4ERBTMTYVGHJCRSVOFZUXG24LW7H3KHUQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.722330454 (13h41m7.466387316s) [noise=true , address=64.49.13.169:4705 , piece_id=YJCLDFSO5ENFREIROYPEZQQTQD4GM4A3DFKNJEHSPFDOKDCFEJ7Q, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.722658583 (13h41m7.466715506s) [noise=true , address=79.127.205.234:4597 , piece_id=WETMAYMNBWD6F6JYCXPNAG5KOJXNCXE3KM7ZJ4F5Y3GEBPHLMDEA, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.722717191 (13h41m7.46677405s) [noise=true , address=104.36.82.5:4867 , piece_id=IEDY33GIP4MZRSN2MKKBLDFAXPMKJUPZVPMOXVCEFJ6BUPZJHYHQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.722989701 (13h41m7.467046568s) [noise=true , address=64.49.13.139:4474 , piece_id=LGXAPMGRQOOEXXG7EBNLO7GAPDKFBCQIBWGYID7KOVQ3UNMDZICQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.723061064 (13h41m7.467117927s) [noise=true , address=45.91.116.14:4201 , piece_id=LCOEAQDFX432OULFXCKDH3FQMZMPR5XSDVKK5MAPEQYEABUAYMDA, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.723458393 (13h41m7.467515259s) [noise=true , address=104.36.82.61:4392 , piece_id=EL44CRMZDDTXOW45GURIDIFQF4K65UUJFKR6L3F3WQBMVXSD2XHQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.729867582 (13h41m7.473924444s) [noise=true , address=104.36.82.60:4345 , piece_id=2AKG3BCGW7TFNVG3FQ576YJOTEZV3V2NP5ITSSZ7BO3NO5CRVDVQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.731063643 (13h41m7.475120512s) [noise=true , address=104.36.82.9:4016 , piece_id=VTTRIIBZ4KCGJW2H5DMG7J63AGFIV7QT34MJJTHRUGFIH6JD6EQQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.732350031 (13h41m7.476406894s) [noise=true , address=104.36.82.52:4106 , piece_id=KTACAAM5SP3PCN3VE6CKF3KHMLGYCHONLC7FDUI4YHLFKIEJ65HQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.733989697 (13h41m7.478046556s) [noise=true , address=64.49.13.155:4916 , piece_id=PJ3VBT33DUHLG3WHE6AK7IIYX76IZZFJCAWX7WX3UUBA6BLGB5VA, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.734727605 (13h41m7.478784474s) [noise=true , address=104.36.82.37:4680 , piece_id=2MDE2TCPX2ROXJXK7Q234YXMK4BFHPDQB5EB7GECHLMUJY5W4ITQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.734786637 (13h41m7.478843504s) [noise=true , address=104.36.82.4:4813 , piece_id=IZO235F25YVNCQG2GSR3UPI5OPMONHVRQQAWDEBOZWR2EILWGCCA, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.735460668 (13h41m7.479517532s) [noise=true , address=104.36.82.6:4901 , piece_id=7VX47BXB37LW2CKYFTBTFAJSDF3QLHEHQHRD5ZM5EHZPAZFPXBPQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.735486537 (13h41m7.479543401s) [noise=true , address=104.36.82.16:4323 , piece_id=RQYKNED6IWTV47W7HGZCVVOUKFRMHGO5WSPCW36ROCXMEDEFYM4Q, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.736565943 (13h41m7.480622804s) [noise=true , address=104.36.82.36:4651 , piece_id=5HQECPYGS5S43JV2CEJY3MRS33NUE6CJ5WPXKTCWYLYVXUQDXLXQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.737386328 (13h41m7.48144319s) [noise=true , address=104.36.82.44:4876 , piece_id=3WUHXX7GVGZWOGDA7FETGVT772RLHO4HRV6OCZKNTZIMG3VDPDOA, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:46.299198843 (13h41m8.043255689s) [upload=14082]: Next returned nil segment Aug 7 23:20:46.299217870 (13h41m8.043274733s) [upload=14082]: Waiting for error group managing segments... Aug 7 23:20:46.464881860 (13h41m8.208938721s) [noise=true , address=104.36.82.60:4345 , piece_id=2AKG3BCGW7TFNVG3FQ576YJOTEZV3V2NP5ITSSZ7BO3NO5CRVDVQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.473775294 (13h41m8.217832164s) [noise=true , address=104.36.82.52:4106 , piece_id=KTACAAM5SP3PCN3VE6CKF3KHMLGYCHONLC7FDUI4YHLFKIEJ65HQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.474664669 (13h41m8.218721541s) [noise=true , address=104.36.82.9:4016 , piece_id=VTTRIIBZ4KCGJW2H5DMG7J63AGFIV7QT34MJJTHRUGFIH6JD6EQQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.479688336 (13h41m8.223745197s) [noise=true , address=104.36.82.36:4651 , piece_id=5HQECPYGS5S43JV2CEJY3MRS33NUE6CJ5WPXKTCWYLYVXUQDXLXQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.485171804 (13h41m8.229228658s) [noise=true , address=104.36.82.6:4901 , piece_id=7VX47BXB37LW2CKYFTBTFAJSDF3QLHEHQHRD5ZM5EHZPAZFPXBPQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.486836180 (13h41m8.230893032s) [noise=true , address=104.36.82.61:4392 , piece_id=EL44CRMZDDTXOW45GURIDIFQF4K65UUJFKR6L3F3WQBMVXSD2XHQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.496476947 (13h41m8.240533796s) [noise=true , address=104.36.82.44:4876 , piece_id=3WUHXX7GVGZWOGDA7FETGVT772RLHO4HRV6OCZKNTZIMG3VDPDOA, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.556494206 (13h41m8.300551081s) [noise=true , address=104.36.82.16:4323 , piece_id=RQYKNED6IWTV47W7HGZCVVOUKFRMHGO5WSPCW36ROCXMEDEFYM4Q, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.628743021 (13h41m8.372799882s) [noise=true , address=45.91.116.14:4201 , piece_id=LCOEAQDFX432OULFXCKDH3FQMZMPR5XSDVKK5MAPEQYEABUAYMDA, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.645987246 (13h41m8.390044106s) [noise=true , address=104.36.82.4:4813 , piece_id=IZO235F25YVNCQG2GSR3UPI5OPMONHVRQQAWDEBOZWR2EILWGCCA, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.666511132 (13h41m8.410567992s) [noise=true , address=104.36.82.5:4867 , piece_id=IEDY33GIP4MZRSN2MKKBLDFAXPMKJUPZVPMOXVCEFJ6BUPZJHYHQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.671756371 (13h41m8.41581322s) [noise=true , address=104.36.82.37:4680 , piece_id=2MDE2TCPX2ROXJXK7Q234YXMK4BFHPDQB5EB7GECHLMUJY5W4ITQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.934025878 (13h41m8.678082729s) [noise=true , address=79.127.205.234:4597 , piece_id=WETMAYMNBWD6F6JYCXPNAG5KOJXNCXE3KM7ZJ4F5Y3GEBPHLMDEA, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.970907150 (13h41m8.714964016s) [noise=true , address=64.49.13.139:4474 , piece_id=LGXAPMGRQOOEXXG7EBNLO7GAPDKFBCQIBWGYID7KOVQ3UNMDZICQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.970940415 (13h41m8.714997268s) [seg_pos={0 0}, upload=14082]: Segment reached optimal threshold of 14 pieces. Aug 7 23:20:46.971273207 (13h41m8.715330074s) [noise=true , address=64.49.13.155:4916 , piece_id=PJ3VBT33DUHLG3WHE6AK7IIYX76IZZFJCAWX7WX3UUBA6BLGB5VA, seg_pos={0 0}, upload=14082]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12MD2UDNXEmeGra9fvhEb8eGL2R6vGLHtSKLpSyX3H3tZCe1r5B): protocol: expected piece hash; context canceled; EOF Aug 7 23:20:46.971460895 (13h41m8.715517755s) [noise=true , address=64.49.13.169:4705 , piece_id=YJCLDFSO5ENFREIROYPEZQQTQD4GM4A3DFKNJEHSPFDOKDCFEJ7Q, seg_pos={0 0}, upload=14082]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124uqzVwsF5TM4CwUhkzfJvgxd71jZXgXuFNwDLRW1vrSXbwLsr): protocol: expected piece hash; context canceled; EOF Aug 7 23:20:46.971541847 (13h41m8.715598706s) [noise=true , address=79.127.205.246:4276 , piece_id=6753PO7TQBULISFOSTSIT32I7DSKQPM6M4TEVF2E7TILSMUU5ARQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJTNVtPFN48S9ZrayehLD2HcVckyJWLx2rtEvyH56W22R8Suq): protocol: expected piece hash; context canceled; EOF Aug 7 23:20:46.971599101 (13h41m8.715655962s) [noise=true , address=104.36.82.58:4292 , piece_id=KQ2RNSZDMCU7OED4IJC4ERBTMTYVGHJCRSVOFZUXG24LW7H3KHUQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JeDjqgszkkq2asUBCL5oTvH6X21TnWkkmqGEEzBbz1YctdDXX): protocol: expected piece hash; context canceled; EOF Aug 7 23:20:46.971618718 (13h41m8.715675568s) [seg_pos={0 0}, upload=14082]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:20:46.971698846 (13h41m8.715755698s) [upload=14082]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:20:46.971704848 (13h41m8.71576171s) [upload=14082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:21:07.219673652 (13h41m28.963730505s) [upload=14082]: Done uploading segments... Aug 7 23:21:07.219701493 (13h41m28.963758344s) [upload=14082]: Upload finished. err: metaclient: context deadline exceeded Duration: 3.472768047s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 314.752268ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 23:50:39 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 314.770353ms edge gateway vhost us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.........X.XX..X............................................] duration: 3.009889182s FAIL (2025-08-07 23:50:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.009917499s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 12 out of 60 is failed (threshold=3) [.........XXX.XXXXXX..XX.X...................................] duration: 876.523538ms FAIL (2025-08-07 23:50:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 7 23:15:07.832719564 (13h35m29.576776416s) [upload=13995]: Done starting upload Aug 7 23:15:07.832826894 (13h35m29.576883745s) [upload=13995]: Uploading segments... Aug 7 23:15:07.832887307 (13h35m29.576944167s) [upload=13995]: Got next segment. Inline: true Aug 7 23:15:07.832893154 (13h35m29.576950015s) [upload=13995]: Waiting for error group managing segments... Aug 7 23:15:07.832938815 (13h35m29.576995666s) [upload=13995]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:15:07.832943939 (13h35m29.5770008s) [upload=13995]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:15:07.832947902 (13h35m29.577004762s) [upload=13995]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:37.560683179 (13h35m59.304740027s) [upload=13995]: Done uploading segments... Aug 7 23:15:37.560704530 (13h35m59.304761377s) [upload=13995]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:16:13.030124119 (13h36m34.774180979s) [upload=14013]: Starting upload Aug 7 23:16:13.030198850 (13h36m34.774255715s) [upload=14013]: Done starting upload Aug 7 23:16:13.030287975 (13h36m34.774344821s) [upload=14013]: Uploading segments... Aug 7 23:16:13.030344552 (13h36m34.774401422s) [upload=14013]: Got next segment. Inline: true Aug 7 23:16:13.030349665 (13h36m34.774406525s) [upload=14013]: Waiting for error group managing segments... Aug 7 23:16:13.030401514 (13h36m34.774458375s) [upload=14013]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:13.030406613 (13h36m34.774463474s) [upload=14013]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:13.030409548 (13h36m34.774466408s) [upload=14013]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:42.761381352 (13h37m4.505438199s) [upload=14013]: Done uploading segments... Aug 7 23:16:42.761401933 (13h37m4.505458793s) [upload=14013]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:17:23.340430421 (13h37m45.084487291s) [upload=14034]: Starting upload Aug 7 23:17:23.340585398 (13h37m45.084642265s) [upload=14034]: Done starting upload Aug 7 23:17:23.340743782 (13h37m45.084800644s) [upload=14034]: Uploading segments... Aug 7 23:17:23.340847668 (13h37m45.084904539s) [upload=14034]: Got next segment. Inline: true Aug 7 23:17:23.340857086 (13h37m45.084913958s) [upload=14034]: Waiting for error group managing segments... Aug 7 23:17:23.340963706 (13h37m45.085020572s) [upload=14034]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:17:23.340973939 (13h37m45.085031006s) [upload=14034]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:17:23.340979371 (13h37m45.08503624s) [upload=14034]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:53.063547301 (13h38m14.807604164s) [upload=14034]: Done uploading segments... Aug 7 23:17:53.063575766 (13h38m14.807632627s) [upload=14034]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:18:01.235662578 (13h38m22.979719439s) [upload=14043]: Starting upload Aug 7 23:18:01.235769067 (13h38m22.97982594s) [upload=14043]: Done starting upload Aug 7 23:18:01.235873103 (13h38m22.979929971s) [upload=14043]: Uploading segments... Aug 7 23:18:01.235955349 (13h38m22.98001222s) [upload=14043]: Got next segment. Inline: true Aug 7 23:18:01.235976122 (13h38m22.980032984s) [upload=14043]: Waiting for error group managing segments... Aug 7 23:18:01.236032940 (13h38m22.980089801s) [upload=14043]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:18:01.236038848 (13h38m22.980095712s) [upload=14043]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:18:01.236056240 (13h38m22.980113107s) [upload=14043]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:18:30.962606072 (13h38m52.706662932s) [upload=14043]: Done uploading segments... Aug 7 23:18:30.962671417 (13h38m52.706728279s) [upload=14043]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:18:52.160899256 (13h39m13.904956117s) [upload=14060]: Starting upload Aug 7 23:18:52.160970297 (13h39m13.905027157s) [upload=14060]: Done starting upload Aug 7 23:18:52.161065587 (13h39m13.905122439s) [upload=14060]: Uploading segments... Aug 7 23:18:52.161108065 (13h39m13.905164934s) [upload=14060]: Got next segment. Inline: true Aug 7 23:18:52.161113994 (13h39m13.905170844s) [upload=14060]: Waiting for error group managing segments... Aug 7 23:18:52.161171289 (13h39m13.90522814s) [upload=14060]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:18:52.161176753 (13h39m13.905233614s) [upload=14060]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:18:52.161180292 (13h39m13.905237155s) [upload=14060]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:18:52.299645097 (13h39m14.043701961s) [upload=14060]: Done uploading segments... Aug 7 23:18:52.299672156 (13h39m14.043729007s) [upload=14060]: Upload finished. err: <nil> Aug 7 23:19:29.899969190 (13h39m51.64402605s) [upload=14071]: Starting upload Aug 7 23:19:29.900023963 (13h39m51.644080812s) [upload=14071]: Done starting upload Aug 7 23:19:29.900097059 (13h39m51.644153908s) [upload=14071]: Uploading segments... Aug 7 23:19:29.900152738 (13h39m51.644209608s) [upload=14071]: Got next segment. Inline: true Aug 7 23:19:29.900161132 (13h39m51.644218002s) [upload=14071]: Waiting for error group managing segments... Aug 7 23:19:29.900212688 (13h39m51.64426955s) [upload=14071]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:19:29.900219311 (13h39m51.644276178s) [upload=14071]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:19:29.900224731 (13h39m51.64428173s) [upload=14071]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:19:30.208671256 (13h39m51.952728106s) [upload=14071]: Done uploading segments... Aug 7 23:19:30.208715246 (13h39m51.952772096s) [upload=14071]: Upload finished. err: <nil> Aug 7 23:20:17.640185376 (13h40m39.384242236s) [upload=14084]: Starting upload Aug 7 23:20:17.640240796 (13h40m39.384297642s) [upload=14084]: Done starting upload Aug 7 23:20:17.640310076 (13h40m39.384366923s) [upload=14084]: Uploading segments... Aug 7 23:20:17.640362556 (13h40m39.384419425s) [upload=14084]: Got next segment. Inline: true Aug 7 23:20:17.640367233 (13h40m39.384424093s) [upload=14084]: Waiting for error group managing segments... Aug 7 23:20:17.640417842 (13h40m39.384474688s) [upload=14084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:20:17.640422768 (13h40m39.384479628s) [upload=14084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:20:17.640425589 (13h40m39.384482436s) [upload=14084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:20:21.993094963 (13h40m43.737151826s) [upload=14084]: Done uploading segments... Aug 7 23:20:21.993114705 (13h40m43.737171555s) [upload=14084]: Upload finished. err: <nil> Aug 7 23:21:07.375939272 (13h41m29.119996122s) [upload=14096]: Starting upload Aug 7 23:21:07.376022371 (13h41m29.120079233s) [upload=14096]: Done starting upload Aug 7 23:21:07.376131644 (13h41m29.120188494s) [upload=14096]: Uploading segments... Aug 7 23:21:07.376228707 (13h41m29.120285578s) [upload=14096]: Got next segment. Inline: true Aug 7 23:21:07.376236415 (13h41m29.120293285s) [upload=14096]: Waiting for error group managing segments... Aug 7 23:21:07.376314696 (13h41m29.120371567s) [upload=14096]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:21:07.376322308 (13h41m29.120379199s) [upload=14096]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:21:07.376327371 (13h41m29.120384242s) [upload=14096]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:21:37.104686329 (13h41m58.84874319s) [upload=14096]: Done uploading segments... Aug 7 23:21:37.104719076 (13h41m58.848775939s) [upload=14096]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:22:04.330992472 (13h42m26.075049325s) [upload=14111]: Starting upload Aug 7 23:22:04.331290477 (13h42m26.075347342s) [upload=14111]: Done starting upload Aug 7 23:22:04.331454134 (13h42m26.075510996s) [upload=14111]: Uploading segments... Aug 7 23:22:04.331559270 (13h42m26.075616144s) [upload=14111]: Got next segment. Inline: true Aug 7 23:22:04.331568404 (13h42m26.075625271s) [upload=14111]: Waiting for error group managing segments... Aug 7 23:22:04.331647006 (13h42m26.075703873s) [upload=14111]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:22:04.331655756 (13h42m26.075712624s) [upload=14111]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:22:04.331661330 (13h42m26.075718197s) [upload=14111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:22:04.632273425 (13h42m26.376330276s) [upload=14111]: Done uploading segments... Aug 7 23:22:04.632293274 (13h42m26.376350128s) [upload=14111]: Upload finished. err: <nil> Aug 7 23:22:37.291949339 (13h42m59.03600619s) [upload=14122]: Starting upload Aug 7 23:22:37.292013250 (13h42m59.036070113s) [upload=14122]: Done starting upload Aug 7 23:22:37.292085244 (13h42m59.036142097s) [upload=14122]: Uploading segments... Aug 7 23:22:37.292207393 (13h42m59.036264263s) [upload=14122]: Got next segment. Inline: true Aug 7 23:22:37.292216268 (13h42m59.036273132s) [upload=14122]: Waiting for error group managing segments... Aug 7 23:22:37.292274925 (13h42m59.036331788s) [upload=14122]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:22:37.292281983 (13h42m59.036338843s) [upload=14122]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:22:37.292288349 (13h42m59.036345217s) [upload=14122]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:23:07.017248949 (13h43m28.761305819s) [upload=14122]: Done uploading segments... Aug 7 23:23:07.017274459 (13h43m28.761331331s) [upload=14122]: Upload finished. err: metaclient: context deadline exceeded Duration: 876.615489ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 7 out of 60 is failed (threshold=3) [......XX..XXXXX.............................................] duration: 873.000443ms FAIL (2025-08-07 23:50:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== :10:03.085332650 (13h30m24.829389513s) [upload=13904]: Done starting upload Aug 7 23:10:03.085414170 (13h30m24.829471025s) [upload=13904]: Uploading segments... Aug 7 23:10:03.085475435 (13h30m24.829532308s) [upload=13904]: Got next segment. Inline: true Aug 7 23:10:03.085480843 (13h30m24.829537706s) [upload=13904]: Waiting for error group managing segments... Aug 7 23:10:03.085544347 (13h30m24.829601211s) [upload=13904]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:03.085552551 (13h30m24.829609424s) [upload=13904]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:03.085555974 (13h30m24.829612844s) [upload=13904]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:10:03.247308964 (13h30m24.991365825s) [upload=13904]: Done uploading segments... Aug 7 23:10:03.247396260 (13h30m24.991453126s) [upload=13904]: Upload finished. err: <nil> Aug 7 23:10:40.522399484 (13h31m2.266456348s) [upload=13915]: Starting upload Aug 7 23:10:40.522487646 (13h31m2.266544496s) [upload=13915]: Done starting upload Aug 7 23:10:40.522571041 (13h31m2.266627902s) [upload=13915]: Uploading segments... Aug 7 23:10:40.522619425 (13h31m2.266676294s) [upload=13915]: Got next segment. Inline: true Aug 7 23:10:40.522626307 (13h31m2.266683183s) [upload=13915]: Waiting for error group managing segments... Aug 7 23:10:40.522684106 (13h31m2.266740971s) [upload=13915]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:40.522691484 (13h31m2.26674835s) [upload=13915]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:40.522696090 (13h31m2.266752961s) [upload=13915]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:40.254283342 (13h32m1.998340192s) [upload=13915]: Done uploading segments... Aug 7 23:11:40.254304548 (13h32m1.998361398s) [upload=13915]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:11:40.667902567 (13h32m2.411959413s) [upload=13933]: Starting upload Aug 7 23:11:40.667961940 (13h32m2.412018788s) [upload=13933]: Done starting upload Aug 7 23:11:40.668020524 (13h32m2.41207737s) [upload=13933]: Uploading segments... Aug 7 23:11:40.668075247 (13h32m2.412132113s) [upload=13933]: Got next segment. Inline: true Aug 7 23:11:40.668080664 (13h32m2.412137525s) [upload=13933]: Waiting for error group managing segments... Aug 7 23:11:40.668121190 (13h32m2.412178036s) [upload=13933]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:40.668125756 (13h32m2.412182616s) [upload=13933]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:40.668128431 (13h32m2.412185291s) [upload=13933]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:40.394243254 (13h33m2.138300118s) [upload=13933]: Done uploading segments... Aug 7 23:12:40.394270270 (13h33m2.138327132s) [upload=13933]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:12:40.798630406 (13h33m2.542687267s) [upload=13953]: Starting upload Aug 7 23:12:40.798725809 (13h33m2.54278266s) [upload=13953]: Done starting upload Aug 7 23:12:40.798811946 (13h33m2.542868796s) [upload=13953]: Uploading segments... Aug 7 23:12:40.798867069 (13h33m2.542923932s) [upload=13953]: Got next segment. Inline: true Aug 7 23:12:40.798873108 (13h33m2.542929959s) [upload=13953]: Waiting for error group managing segments... Aug 7 23:12:40.798918573 (13h33m2.542975422s) [upload=13953]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:40.798924603 (13h33m2.542981463s) [upload=13953]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:40.798939855 (13h33m2.542996716s) [upload=13953]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:40.942390421 (13h33m2.686447272s) [upload=13953]: Done uploading segments... Aug 7 23:12:40.942410212 (13h33m2.686467062s) [upload=13953]: Upload finished. err: <nil> Aug 7 23:13:18.137108974 (13h33m39.881165825s) [upload=13967]: Starting upload Aug 7 23:13:18.137180408 (13h33m39.881237292s) [upload=13967]: Done starting upload Aug 7 23:13:18.137258925 (13h33m39.881315786s) [upload=13967]: Uploading segments... Aug 7 23:13:18.137309635 (13h33m39.881366505s) [upload=13967]: Got next segment. Inline: true Aug 7 23:13:18.137314511 (13h33m39.881371371s) [upload=13967]: Waiting for error group managing segments... Aug 7 23:13:18.137374772 (13h33m39.881431633s) [upload=13967]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:18.137380412 (13h33m39.881437273s) [upload=13967]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:18.137383733 (13h33m39.881440593s) [upload=13967]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:18.301480519 (13h33m40.045537379s) [upload=13967]: Done uploading segments... Aug 7 23:13:18.301498918 (13h33m40.045555767s) [upload=13967]: Upload finished. err: <nil> Aug 7 23:14:13.169029066 (13h34m34.913085913s) [upload=13980]: Starting upload Aug 7 23:14:13.169095254 (13h34m34.913152101s) [upload=13980]: Done starting upload Aug 7 23:14:13.169153338 (13h34m34.913210184s) [upload=13980]: Uploading segments... Aug 7 23:14:13.169225662 (13h34m34.913282508s) [upload=13980]: Got next segment. Inline: true Aug 7 23:14:13.169230885 (13h34m34.913287744s) [upload=13980]: Waiting for error group managing segments... Aug 7 23:14:13.169272521 (13h34m34.913329381s) [upload=13980]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:13.169288952 (13h34m34.913345813s) [upload=13980]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:13.169291623 (13h34m34.913348483s) [upload=13980]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:12.895960606 (13h35m34.640017467s) [upload=13980]: Done uploading segments... Aug 7 23:15:12.895983608 (13h35m34.640040456s) [upload=13980]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:15:13.308146067 (13h35m35.052202918s) [upload=13997]: Starting upload Aug 7 23:15:13.308219271 (13h35m35.052276134s) [upload=13997]: Done starting upload Aug 7 23:15:13.308297319 (13h35m35.052354171s) [upload=13997]: Uploading segments... Aug 7 23:15:13.308360203 (13h35m35.052417072s) [upload=13997]: Got next segment. Inline: true Aug 7 23:15:13.308365548 (13h35m35.052422408s) [upload=13997]: Waiting for error group managing segments... Aug 7 23:15:13.308409794 (13h35m35.052466657s) [upload=13997]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:15:13.308426102 (13h35m35.052482963s) [upload=13997]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:15:13.308429725 (13h35m35.052486585s) [upload=13997]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:13.040589970 (13h36m34.784646843s) [upload=13997]: Done uploading segments... Aug 7 23:16:13.040614428 (13h36m34.784671274s) [upload=13997]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:16:20.767866726 (13h36m42.511923577s) [upload=14018]: Starting upload Aug 7 23:16:20.767927544 (13h36m42.511984398s) [upload=14018]: Done starting upload Aug 7 23:16:20.767993090 (13h36m42.512049942s) [upload=14018]: Uploading segments... Aug 7 23:16:20.768055054 (13h36m42.512111926s) [upload=14018]: Got next segment. Inline: true Aug 7 23:16:20.768061500 (13h36m42.512118351s) [upload=14018]: Waiting for error group managing segments... Aug 7 23:16:20.768130604 (13h36m42.512187577s) [upload=14018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:20.768139327 (13h36m42.512196192s) [upload=14018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:20.768144987 (13h36m42.512201849s) [upload=14018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:20.496845379 (13h37m42.240902231s) [upload=14018]: Done uploading segments... Aug 7 23:17:20.496873778 (13h37m42.240930632s) [upload=14018]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:17:20.916777280 (13h37m42.660834146s) [upload=14032]: Starting upload Aug 7 23:17:20.916877286 (13h37m42.660934149s) [upload=14032]: Done starting upload Aug 7 23:17:20.917002557 (13h37m42.661059408s) [upload=14032]: Uploading segments... Aug 7 23:17:20.917058206 (13h37m42.661115077s) [upload=14032]: Got next segment. Inline: true Aug 7 23:17:20.917064317 (13h37m42.661121178s) [upload=14032]: Waiting for error group managing segments... Aug 7 23:17:20.917116347 (13h37m42.661173214s) [upload=14032]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:17:20.917123150 (13h37m42.661180014s) [upload=14032]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:17:20.917127779 (13h37m42.661184648s) [upload=14032]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:18:20.640884349 (13h38m42.384941214s) [upload=14032]: Done uploading segments... Aug 7 23:18:20.640914749 (13h38m42.384971617s) [upload=14032]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:18:21.050365938 (13h38m42.79442281s) [upload=14051]: Starting upload Aug 7 23:18:21.050484720 (13h38m42.794541587s) [upload=14051]: Done starting upload Aug 7 23:18:21.050589902 (13h38m42.794646752s) [upload=14051]: Uploading segments... Aug 7 23:18:21.050663797 (13h38m42.794720667s) [upload=14051]: Got next segment. Inline: true Aug 7 23:18:21.050670451 (13h38m42.794727311s) [upload=14051]: Waiting for error group managing segments... Aug 7 23:18:21.050719413 (13h38m42.794776275s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:18:21.050725163 (13h38m42.794782023s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:18:21.050728952 (13h38m42.794785811s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:19:20.780076504 (13h39m42.524133366s) [upload=14051]: Done uploading segments... Aug 7 23:19:20.780099469 (13h39m42.52415633s) [upload=14051]: Upload finished. err: metaclient: context deadline exceeded Duration: 873.028514ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 10 out of 60 is failed (threshold=3) [........XXXXX.XX.XX.X.......................................] duration: 857.967292ms FAIL (2025-08-07 23:50:19 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 858.051129ms core uplink us1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 398.259678ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 23:50:18 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 398.292919ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 8 out of 60 is failed (threshold=3) [............XXXXXX.XX.......................................] duration: 2.529009206s FAIL (2025-08-07 23:50:08 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.529043463s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 835.870181ms FAIL (2025-08-07 23:50:07 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== pload=14121]: Done uploading segments... Aug 7 23:22:28.505845197 (13h42m50.249902044s) [upload=14121]: Upload finished. err: <nil> Aug 7 23:23:10.377483605 (13h43m32.121540458s) [upload=14132]: Starting upload Aug 7 23:23:10.377535635 (13h43m32.121592495s) [upload=14132]: Done starting upload Aug 7 23:23:10.377603797 (13h43m32.121660647s) [upload=14132]: Uploading segments... Aug 7 23:23:10.377638935 (13h43m32.121695804s) [upload=14132]: Got next segment. Inline: true Aug 7 23:23:10.377643594 (13h43m32.121700454s) [upload=14132]: Waiting for error group managing segments... Aug 7 23:23:10.377688252 (13h43m32.121745104s) [upload=14132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:23:10.377693427 (13h43m32.121750286s) [upload=14132]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:23:10.377696805 (13h43m32.121753666s) [upload=14132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:23:10.562260724 (13h43m32.306317573s) [upload=14132]: Done uploading segments... Aug 7 23:23:10.562279228 (13h43m32.306336078s) [upload=14132]: Upload finished. err: <nil> Aug 7 23:24:07.896404858 (13h44m29.640461721s) [upload=14147]: Starting upload Aug 7 23:24:07.896490216 (13h44m29.640547079s) [upload=14147]: Done starting upload Aug 7 23:24:07.896580488 (13h44m29.640637339s) [upload=14147]: Uploading segments... Aug 7 23:24:07.896655006 (13h44m29.640711874s) [upload=14147]: Got next segment. Inline: true Aug 7 23:24:07.896660179 (13h44m29.640717039s) [upload=14147]: Waiting for error group managing segments... Aug 7 23:24:07.896714840 (13h44m29.6407717s) [upload=14147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:24:07.896719399 (13h44m29.64077626s) [upload=14147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:24:07.896722726 (13h44m29.640779586s) [upload=14147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:24:08.048898130 (13h44m29.792954991s) [upload=14147]: Done uploading segments... Aug 7 23:24:08.048952526 (13h44m29.793009386s) [upload=14147]: Upload finished. err: <nil> Aug 7 23:24:53.794754379 (13h45m15.53881125s) [upload=14162]: Starting upload Aug 7 23:24:53.794842317 (13h45m15.538899177s) [upload=14162]: Done starting upload Aug 7 23:24:53.794935964 (13h45m15.538992814s) [upload=14162]: Uploading segments... Aug 7 23:24:53.795003752 (13h45m15.539060623s) [upload=14162]: Got next segment. Inline: true Aug 7 23:24:53.795010814 (13h45m15.539067676s) [upload=14162]: Waiting for error group managing segments... Aug 7 23:24:53.795066607 (13h45m15.539123488s) [upload=14162]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:24:53.795158449 (13h45m15.5392153s) [upload=14162]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:24:53.795163166 (13h45m15.539220026s) [upload=14162]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:24:53.946994982 (13h45m15.691051833s) [upload=14162]: Done uploading segments... Aug 7 23:24:53.947013862 (13h45m15.691070711s) [upload=14162]: Upload finished. err: <nil> Aug 7 23:25:40.029856966 (13h46m1.773913836s) [upload=14177]: Starting upload Aug 7 23:25:40.029986938 (13h46m1.774043809s) [upload=14177]: Done starting upload Aug 7 23:25:40.030076796 (13h46m1.774133642s) [upload=14177]: Uploading segments... Aug 7 23:25:40.030132567 (13h46m1.774189438s) [upload=14177]: Got next segment. Inline: true Aug 7 23:25:40.030139885 (13h46m1.774196757s) [upload=14177]: Waiting for error group managing segments... Aug 7 23:25:40.030197588 (13h46m1.774254461s) [upload=14177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:25:40.030204146 (13h46m1.774261016s) [upload=14177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:25:40.030207984 (13h46m1.77426485s) [upload=14177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:25:40.184198742 (13h46m1.928255592s) [upload=14177]: Done uploading segments... Aug 7 23:25:40.184219017 (13h46m1.928275867s) [upload=14177]: Upload finished. err: <nil> Aug 7 23:26:54.625649110 (13h47m16.369705971s) [upload=14198]: Starting upload Aug 7 23:26:54.625738563 (13h47m16.369795424s) [upload=14198]: Done starting upload Aug 7 23:26:54.625842691 (13h47m16.369899539s) [upload=14198]: Uploading segments... Aug 7 23:26:54.626016839 (13h47m16.37007371s) [upload=14198]: Got next segment. Inline: true Aug 7 23:26:54.626052112 (13h47m16.370108973s) [upload=14198]: Waiting for error group managing segments... Aug 7 23:26:54.626129780 (13h47m16.370186645s) [upload=14198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:26:54.626137392 (13h47m16.370194264s) [upload=14198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:26:54.626143182 (13h47m16.370200048s) [upload=14198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:26:54.775804171 (13h47m16.519861021s) [upload=14198]: Done uploading segments... Aug 7 23:26:54.775824249 (13h47m16.519881102s) [upload=14198]: Upload finished. err: <nil> Aug 7 23:27:32.263746490 (13h47m54.007803361s) [upload=14208]: Starting upload Aug 7 23:27:32.263833655 (13h47m54.007890501s) [upload=14208]: Done starting upload Aug 7 23:27:32.263976536 (13h47m54.008033385s) [upload=14208]: Uploading segments... Aug 7 23:27:32.264070926 (13h47m54.008127796s) [upload=14208]: Got next segment. Inline: true Aug 7 23:27:32.264080356 (13h47m54.00813733s) [upload=14208]: Waiting for error group managing segments... Aug 7 23:27:32.264154657 (13h47m54.008211526s) [upload=14208]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:27:32.264163587 (13h47m54.008220454s) [upload=14208]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:27:32.264172017 (13h47m54.008228914s) [upload=14208]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:27:32.490188559 (13h47m54.23424541s) [upload=14208]: Done uploading segments... Aug 7 23:27:32.490207326 (13h47m54.234264172s) [upload=14208]: Upload finished. err: <nil> Aug 7 23:27:57.927940477 (13h48m19.671997323s) [upload=14217]: Starting upload Aug 7 23:27:57.928016423 (13h48m19.672073283s) [upload=14217]: Done starting upload Aug 7 23:27:57.928101290 (13h48m19.672158142s) [upload=14217]: Uploading segments... Aug 7 23:27:57.928155692 (13h48m19.672212566s) [upload=14217]: Got next segment. Inline: true Aug 7 23:27:57.928162009 (13h48m19.672218869s) [upload=14217]: Waiting for error group managing segments... Aug 7 23:27:57.928213294 (13h48m19.672270145s) [upload=14217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:27:57.928219212 (13h48m19.672276074s) [upload=14217]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:27:57.928223521 (13h48m19.672280383s) [upload=14217]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:27:58.087544005 (13h48m19.831600874s) [upload=14217]: Done uploading segments... Aug 7 23:27:58.087570226 (13h48m19.831627094s) [upload=14217]: Upload finished. err: <nil> Aug 7 23:29:12.765200935 (13h49m34.509257795s) [upload=14237]: Starting upload Aug 7 23:29:12.765253106 (13h49m34.509309977s) [upload=14237]: Done starting upload Aug 7 23:29:12.765324641 (13h49m34.509381499s) [upload=14237]: Uploading segments... Aug 7 23:29:12.765374187 (13h49m34.509431056s) [upload=14237]: Got next segment. Inline: true Aug 7 23:29:12.765378947 (13h49m34.509435807s) [upload=14237]: Waiting for error group managing segments... Aug 7 23:29:12.765430859 (13h49m34.50948772s) [upload=14237]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:29:12.765438106 (13h49m34.50949497s) [upload=14237]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:29:12.765443127 (13h49m34.509499988s) [upload=14237]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:29:12.926437674 (13h49m34.670494535s) [upload=14237]: Done uploading segments... Aug 7 23:29:12.926457418 (13h49m34.670514268s) [upload=14237]: Upload finished. err: <nil> Aug 7 23:29:42.225760316 (13h50m3.969817177s) [upload=14244]: Starting upload Aug 7 23:29:42.225855063 (13h50m3.969911929s) [upload=14244]: Done starting upload Aug 7 23:29:42.225962489 (13h50m3.970019355s) [upload=14244]: Uploading segments... Aug 7 23:29:42.226078484 (13h50m3.970135331s) [upload=14244]: Got next segment. Inline: true Aug 7 23:29:42.226083996 (13h50m3.970140842s) [upload=14244]: Waiting for error group managing segments... Aug 7 23:29:42.226143067 (13h50m3.970199914s) [upload=14244]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:29:42.226148511 (13h50m3.970205371s) [upload=14244]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:29:42.226153006 (13h50m3.970209865s) [upload=14244]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:29:42.401822947 (13h50m4.14587981s) [upload=14244]: Done uploading segments... Aug 7 23:29:42.401847556 (13h50m4.145904405s) [upload=14244]: Upload finished. err: <nil> Aug 7 23:30:31.529840301 (13h50m53.273897165s) [upload=14254]: Starting upload Aug 7 23:30:31.530002531 (13h50m53.274059378s) [upload=14254]: Done starting upload Aug 7 23:30:31.530201544 (13h50m53.274258391s) [upload=14254]: Uploading segments... Aug 7 23:30:31.530298120 (13h50m53.274354985s) [upload=14254]: Got next segment. Inline: true Aug 7 23:30:31.530303457 (13h50m53.274360317s) [upload=14254]: Waiting for error group managing segments... Aug 7 23:30:31.530353721 (13h50m53.274410567s) [upload=14254]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:30:31.530358927 (13h50m53.274415788s) [upload=14254]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:30:31.530363758 (13h50m53.274420619s) [upload=14254]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:30:31.696350967 (13h50m53.440407828s) [upload=14254]: Done uploading segments... Aug 7 23:30:31.696369460 (13h50m53.440426307s) [upload=14254]: Upload finished. err: <nil> Duration: 835.899706ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 10 out of 60 is failed (threshold=3) [.........XXXXX.XX.XX.X......................................] duration: 870.033001ms FAIL (2025-08-07 23:50:01 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 870.069338ms core uplink us1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 287.649088ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 23:49:53 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 287.680465ms edge gateway vhost us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 7 out of 60 is failed (threshold=3) [.........X.XXXXX.X..........................................] duration: 2.780931183s FAIL (2025-08-07 23:49:52 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.780957026s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 854.810304ms FAIL (2025-08-07 23:49:51 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== pload=14121]: Done uploading segments... Aug 7 23:22:28.505845197 (13h42m50.249902044s) [upload=14121]: Upload finished. err: <nil> Aug 7 23:23:10.377483605 (13h43m32.121540458s) [upload=14132]: Starting upload Aug 7 23:23:10.377535635 (13h43m32.121592495s) [upload=14132]: Done starting upload Aug 7 23:23:10.377603797 (13h43m32.121660647s) [upload=14132]: Uploading segments... Aug 7 23:23:10.377638935 (13h43m32.121695804s) [upload=14132]: Got next segment. Inline: true Aug 7 23:23:10.377643594 (13h43m32.121700454s) [upload=14132]: Waiting for error group managing segments... Aug 7 23:23:10.377688252 (13h43m32.121745104s) [upload=14132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:23:10.377693427 (13h43m32.121750286s) [upload=14132]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:23:10.377696805 (13h43m32.121753666s) [upload=14132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:23:10.562260724 (13h43m32.306317573s) [upload=14132]: Done uploading segments... Aug 7 23:23:10.562279228 (13h43m32.306336078s) [upload=14132]: Upload finished. err: <nil> Aug 7 23:24:07.896404858 (13h44m29.640461721s) [upload=14147]: Starting upload Aug 7 23:24:07.896490216 (13h44m29.640547079s) [upload=14147]: Done starting upload Aug 7 23:24:07.896580488 (13h44m29.640637339s) [upload=14147]: Uploading segments... Aug 7 23:24:07.896655006 (13h44m29.640711874s) [upload=14147]: Got next segment. Inline: true Aug 7 23:24:07.896660179 (13h44m29.640717039s) [upload=14147]: Waiting for error group managing segments... Aug 7 23:24:07.896714840 (13h44m29.6407717s) [upload=14147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:24:07.896719399 (13h44m29.64077626s) [upload=14147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:24:07.896722726 (13h44m29.640779586s) [upload=14147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:24:08.048898130 (13h44m29.792954991s) [upload=14147]: Done uploading segments... Aug 7 23:24:08.048952526 (13h44m29.793009386s) [upload=14147]: Upload finished. err: <nil> Aug 7 23:24:53.794754379 (13h45m15.53881125s) [upload=14162]: Starting upload Aug 7 23:24:53.794842317 (13h45m15.538899177s) [upload=14162]: Done starting upload Aug 7 23:24:53.794935964 (13h45m15.538992814s) [upload=14162]: Uploading segments... Aug 7 23:24:53.795003752 (13h45m15.539060623s) [upload=14162]: Got next segment. Inline: true Aug 7 23:24:53.795010814 (13h45m15.539067676s) [upload=14162]: Waiting for error group managing segments... Aug 7 23:24:53.795066607 (13h45m15.539123488s) [upload=14162]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:24:53.795158449 (13h45m15.5392153s) [upload=14162]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:24:53.795163166 (13h45m15.539220026s) [upload=14162]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:24:53.946994982 (13h45m15.691051833s) [upload=14162]: Done uploading segments... Aug 7 23:24:53.947013862 (13h45m15.691070711s) [upload=14162]: Upload finished. err: <nil> Aug 7 23:25:40.029856966 (13h46m1.773913836s) [upload=14177]: Starting upload Aug 7 23:25:40.029986938 (13h46m1.774043809s) [upload=14177]: Done starting upload Aug 7 23:25:40.030076796 (13h46m1.774133642s) [upload=14177]: Uploading segments... Aug 7 23:25:40.030132567 (13h46m1.774189438s) [upload=14177]: Got next segment. Inline: true Aug 7 23:25:40.030139885 (13h46m1.774196757s) [upload=14177]: Waiting for error group managing segments... Aug 7 23:25:40.030197588 (13h46m1.774254461s) [upload=14177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:25:40.030204146 (13h46m1.774261016s) [upload=14177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:25:40.030207984 (13h46m1.77426485s) [upload=14177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:25:40.184198742 (13h46m1.928255592s) [upload=14177]: Done uploading segments... Aug 7 23:25:40.184219017 (13h46m1.928275867s) [upload=14177]: Upload finished. err: <nil> Aug 7 23:26:54.625649110 (13h47m16.369705971s) [upload=14198]: Starting upload Aug 7 23:26:54.625738563 (13h47m16.369795424s) [upload=14198]: Done starting upload Aug 7 23:26:54.625842691 (13h47m16.369899539s) [upload=14198]: Uploading segments... Aug 7 23:26:54.626016839 (13h47m16.37007371s) [upload=14198]: Got next segment. Inline: true Aug 7 23:26:54.626052112 (13h47m16.370108973s) [upload=14198]: Waiting for error group managing segments... Aug 7 23:26:54.626129780 (13h47m16.370186645s) [upload=14198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:26:54.626137392 (13h47m16.370194264s) [upload=14198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:26:54.626143182 (13h47m16.370200048s) [upload=14198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:26:54.775804171 (13h47m16.519861021s) [upload=14198]: Done uploading segments... Aug 7 23:26:54.775824249 (13h47m16.519881102s) [upload=14198]: Upload finished. err: <nil> Aug 7 23:27:32.263746490 (13h47m54.007803361s) [upload=14208]: Starting upload Aug 7 23:27:32.263833655 (13h47m54.007890501s) [upload=14208]: Done starting upload Aug 7 23:27:32.263976536 (13h47m54.008033385s) [upload=14208]: Uploading segments... Aug 7 23:27:32.264070926 (13h47m54.008127796s) [upload=14208]: Got next segment. Inline: true Aug 7 23:27:32.264080356 (13h47m54.00813733s) [upload=14208]: Waiting for error group managing segments... Aug 7 23:27:32.264154657 (13h47m54.008211526s) [upload=14208]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:27:32.264163587 (13h47m54.008220454s) [upload=14208]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:27:32.264172017 (13h47m54.008228914s) [upload=14208]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:27:32.490188559 (13h47m54.23424541s) [upload=14208]: Done uploading segments... Aug 7 23:27:32.490207326 (13h47m54.234264172s) [upload=14208]: Upload finished. err: <nil> Aug 7 23:27:57.927940477 (13h48m19.671997323s) [upload=14217]: Starting upload Aug 7 23:27:57.928016423 (13h48m19.672073283s) [upload=14217]: Done starting upload Aug 7 23:27:57.928101290 (13h48m19.672158142s) [upload=14217]: Uploading segments... Aug 7 23:27:57.928155692 (13h48m19.672212566s) [upload=14217]: Got next segment. Inline: true Aug 7 23:27:57.928162009 (13h48m19.672218869s) [upload=14217]: Waiting for error group managing segments... Aug 7 23:27:57.928213294 (13h48m19.672270145s) [upload=14217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:27:57.928219212 (13h48m19.672276074s) [upload=14217]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:27:57.928223521 (13h48m19.672280383s) [upload=14217]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:27:58.087544005 (13h48m19.831600874s) [upload=14217]: Done uploading segments... Aug 7 23:27:58.087570226 (13h48m19.831627094s) [upload=14217]: Upload finished. err: <nil> Aug 7 23:29:12.765200935 (13h49m34.509257795s) [upload=14237]: Starting upload Aug 7 23:29:12.765253106 (13h49m34.509309977s) [upload=14237]: Done starting upload Aug 7 23:29:12.765324641 (13h49m34.509381499s) [upload=14237]: Uploading segments... Aug 7 23:29:12.765374187 (13h49m34.509431056s) [upload=14237]: Got next segment. Inline: true Aug 7 23:29:12.765378947 (13h49m34.509435807s) [upload=14237]: Waiting for error group managing segments... Aug 7 23:29:12.765430859 (13h49m34.50948772s) [upload=14237]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:29:12.765438106 (13h49m34.50949497s) [upload=14237]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:29:12.765443127 (13h49m34.509499988s) [upload=14237]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:29:12.926437674 (13h49m34.670494535s) [upload=14237]: Done uploading segments... Aug 7 23:29:12.926457418 (13h49m34.670514268s) [upload=14237]: Upload finished. err: <nil> Aug 7 23:29:42.225760316 (13h50m3.969817177s) [upload=14244]: Starting upload Aug 7 23:29:42.225855063 (13h50m3.969911929s) [upload=14244]: Done starting upload Aug 7 23:29:42.225962489 (13h50m3.970019355s) [upload=14244]: Uploading segments... Aug 7 23:29:42.226078484 (13h50m3.970135331s) [upload=14244]: Got next segment. Inline: true Aug 7 23:29:42.226083996 (13h50m3.970140842s) [upload=14244]: Waiting for error group managing segments... Aug 7 23:29:42.226143067 (13h50m3.970199914s) [upload=14244]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:29:42.226148511 (13h50m3.970205371s) [upload=14244]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:29:42.226153006 (13h50m3.970209865s) [upload=14244]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:29:42.401822947 (13h50m4.14587981s) [upload=14244]: Done uploading segments... Aug 7 23:29:42.401847556 (13h50m4.145904405s) [upload=14244]: Upload finished. err: <nil> Aug 7 23:30:31.529840301 (13h50m53.273897165s) [upload=14254]: Starting upload Aug 7 23:30:31.530002531 (13h50m53.274059378s) [upload=14254]: Done starting upload Aug 7 23:30:31.530201544 (13h50m53.274258391s) [upload=14254]: Uploading segments... Aug 7 23:30:31.530298120 (13h50m53.274354985s) [upload=14254]: Got next segment. Inline: true Aug 7 23:30:31.530303457 (13h50m53.274360317s) [upload=14254]: Waiting for error group managing segments... Aug 7 23:30:31.530353721 (13h50m53.274410567s) [upload=14254]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:30:31.530358927 (13h50m53.274415788s) [upload=14254]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:30:31.530363758 (13h50m53.274420619s) [upload=14254]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:30:31.696350967 (13h50m53.440407828s) [upload=14254]: Done uploading segments... Aug 7 23:30:31.696369460 (13h50m53.440426307s) [upload=14254]: Upload finished. err: <nil> Duration: 854.835807ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 12 out of 60 is failed (threshold=3) [..........XXX.XXXXXX..XX.X..................................] duration: 851.035793ms FAIL (2025-08-07 23:49:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 7 23:15:07.832719564 (13h35m29.576776416s) [upload=13995]: Done starting upload Aug 7 23:15:07.832826894 (13h35m29.576883745s) [upload=13995]: Uploading segments... Aug 7 23:15:07.832887307 (13h35m29.576944167s) [upload=13995]: Got next segment. Inline: true Aug 7 23:15:07.832893154 (13h35m29.576950015s) [upload=13995]: Waiting for error group managing segments... Aug 7 23:15:07.832938815 (13h35m29.576995666s) [upload=13995]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:15:07.832943939 (13h35m29.5770008s) [upload=13995]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:15:07.832947902 (13h35m29.577004762s) [upload=13995]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:37.560683179 (13h35m59.304740027s) [upload=13995]: Done uploading segments... Aug 7 23:15:37.560704530 (13h35m59.304761377s) [upload=13995]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:16:13.030124119 (13h36m34.774180979s) [upload=14013]: Starting upload Aug 7 23:16:13.030198850 (13h36m34.774255715s) [upload=14013]: Done starting upload Aug 7 23:16:13.030287975 (13h36m34.774344821s) [upload=14013]: Uploading segments... Aug 7 23:16:13.030344552 (13h36m34.774401422s) [upload=14013]: Got next segment. Inline: true Aug 7 23:16:13.030349665 (13h36m34.774406525s) [upload=14013]: Waiting for error group managing segments... Aug 7 23:16:13.030401514 (13h36m34.774458375s) [upload=14013]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:13.030406613 (13h36m34.774463474s) [upload=14013]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:13.030409548 (13h36m34.774466408s) [upload=14013]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:42.761381352 (13h37m4.505438199s) [upload=14013]: Done uploading segments... Aug 7 23:16:42.761401933 (13h37m4.505458793s) [upload=14013]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:17:23.340430421 (13h37m45.084487291s) [upload=14034]: Starting upload Aug 7 23:17:23.340585398 (13h37m45.084642265s) [upload=14034]: Done starting upload Aug 7 23:17:23.340743782 (13h37m45.084800644s) [upload=14034]: Uploading segments... Aug 7 23:17:23.340847668 (13h37m45.084904539s) [upload=14034]: Got next segment. Inline: true Aug 7 23:17:23.340857086 (13h37m45.084913958s) [upload=14034]: Waiting for error group managing segments... Aug 7 23:17:23.340963706 (13h37m45.085020572s) [upload=14034]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:17:23.340973939 (13h37m45.085031006s) [upload=14034]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:17:23.340979371 (13h37m45.08503624s) [upload=14034]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:53.063547301 (13h38m14.807604164s) [upload=14034]: Done uploading segments... Aug 7 23:17:53.063575766 (13h38m14.807632627s) [upload=14034]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:18:01.235662578 (13h38m22.979719439s) [upload=14043]: Starting upload Aug 7 23:18:01.235769067 (13h38m22.97982594s) [upload=14043]: Done starting upload Aug 7 23:18:01.235873103 (13h38m22.979929971s) [upload=14043]: Uploading segments... Aug 7 23:18:01.235955349 (13h38m22.98001222s) [upload=14043]: Got next segment. Inline: true Aug 7 23:18:01.235976122 (13h38m22.980032984s) [upload=14043]: Waiting for error group managing segments... Aug 7 23:18:01.236032940 (13h38m22.980089801s) [upload=14043]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:18:01.236038848 (13h38m22.980095712s) [upload=14043]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:18:01.236056240 (13h38m22.980113107s) [upload=14043]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:18:30.962606072 (13h38m52.706662932s) [upload=14043]: Done uploading segments... Aug 7 23:18:30.962671417 (13h38m52.706728279s) [upload=14043]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:18:52.160899256 (13h39m13.904956117s) [upload=14060]: Starting upload Aug 7 23:18:52.160970297 (13h39m13.905027157s) [upload=14060]: Done starting upload Aug 7 23:18:52.161065587 (13h39m13.905122439s) [upload=14060]: Uploading segments... Aug 7 23:18:52.161108065 (13h39m13.905164934s) [upload=14060]: Got next segment. Inline: true Aug 7 23:18:52.161113994 (13h39m13.905170844s) [upload=14060]: Waiting for error group managing segments... Aug 7 23:18:52.161171289 (13h39m13.90522814s) [upload=14060]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:18:52.161176753 (13h39m13.905233614s) [upload=14060]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:18:52.161180292 (13h39m13.905237155s) [upload=14060]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:18:52.299645097 (13h39m14.043701961s) [upload=14060]: Done uploading segments... Aug 7 23:18:52.299672156 (13h39m14.043729007s) [upload=14060]: Upload finished. err: <nil> Aug 7 23:19:29.899969190 (13h39m51.64402605s) [upload=14071]: Starting upload Aug 7 23:19:29.900023963 (13h39m51.644080812s) [upload=14071]: Done starting upload Aug 7 23:19:29.900097059 (13h39m51.644153908s) [upload=14071]: Uploading segments... Aug 7 23:19:29.900152738 (13h39m51.644209608s) [upload=14071]: Got next segment. Inline: true Aug 7 23:19:29.900161132 (13h39m51.644218002s) [upload=14071]: Waiting for error group managing segments... Aug 7 23:19:29.900212688 (13h39m51.64426955s) [upload=14071]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:19:29.900219311 (13h39m51.644276178s) [upload=14071]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:19:29.900224731 (13h39m51.64428173s) [upload=14071]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:19:30.208671256 (13h39m51.952728106s) [upload=14071]: Done uploading segments... Aug 7 23:19:30.208715246 (13h39m51.952772096s) [upload=14071]: Upload finished. err: <nil> Aug 7 23:20:17.640185376 (13h40m39.384242236s) [upload=14084]: Starting upload Aug 7 23:20:17.640240796 (13h40m39.384297642s) [upload=14084]: Done starting upload Aug 7 23:20:17.640310076 (13h40m39.384366923s) [upload=14084]: Uploading segments... Aug 7 23:20:17.640362556 (13h40m39.384419425s) [upload=14084]: Got next segment. Inline: true Aug 7 23:20:17.640367233 (13h40m39.384424093s) [upload=14084]: Waiting for error group managing segments... Aug 7 23:20:17.640417842 (13h40m39.384474688s) [upload=14084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:20:17.640422768 (13h40m39.384479628s) [upload=14084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:20:17.640425589 (13h40m39.384482436s) [upload=14084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:20:21.993094963 (13h40m43.737151826s) [upload=14084]: Done uploading segments... Aug 7 23:20:21.993114705 (13h40m43.737171555s) [upload=14084]: Upload finished. err: <nil> Aug 7 23:21:07.375939272 (13h41m29.119996122s) [upload=14096]: Starting upload Aug 7 23:21:07.376022371 (13h41m29.120079233s) [upload=14096]: Done starting upload Aug 7 23:21:07.376131644 (13h41m29.120188494s) [upload=14096]: Uploading segments... Aug 7 23:21:07.376228707 (13h41m29.120285578s) [upload=14096]: Got next segment. Inline: true Aug 7 23:21:07.376236415 (13h41m29.120293285s) [upload=14096]: Waiting for error group managing segments... Aug 7 23:21:07.376314696 (13h41m29.120371567s) [upload=14096]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:21:07.376322308 (13h41m29.120379199s) [upload=14096]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:21:07.376327371 (13h41m29.120384242s) [upload=14096]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:21:37.104686329 (13h41m58.84874319s) [upload=14096]: Done uploading segments... Aug 7 23:21:37.104719076 (13h41m58.848775939s) [upload=14096]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:22:04.330992472 (13h42m26.075049325s) [upload=14111]: Starting upload Aug 7 23:22:04.331290477 (13h42m26.075347342s) [upload=14111]: Done starting upload Aug 7 23:22:04.331454134 (13h42m26.075510996s) [upload=14111]: Uploading segments... Aug 7 23:22:04.331559270 (13h42m26.075616144s) [upload=14111]: Got next segment. Inline: true Aug 7 23:22:04.331568404 (13h42m26.075625271s) [upload=14111]: Waiting for error group managing segments... Aug 7 23:22:04.331647006 (13h42m26.075703873s) [upload=14111]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:22:04.331655756 (13h42m26.075712624s) [upload=14111]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:22:04.331661330 (13h42m26.075718197s) [upload=14111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:22:04.632273425 (13h42m26.376330276s) [upload=14111]: Done uploading segments... Aug 7 23:22:04.632293274 (13h42m26.376350128s) [upload=14111]: Upload finished. err: <nil> Aug 7 23:22:37.291949339 (13h42m59.03600619s) [upload=14122]: Starting upload Aug 7 23:22:37.292013250 (13h42m59.036070113s) [upload=14122]: Done starting upload Aug 7 23:22:37.292085244 (13h42m59.036142097s) [upload=14122]: Uploading segments... Aug 7 23:22:37.292207393 (13h42m59.036264263s) [upload=14122]: Got next segment. Inline: true Aug 7 23:22:37.292216268 (13h42m59.036273132s) [upload=14122]: Waiting for error group managing segments... Aug 7 23:22:37.292274925 (13h42m59.036331788s) [upload=14122]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:22:37.292281983 (13h42m59.036338843s) [upload=14122]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:22:37.292288349 (13h42m59.036345217s) [upload=14122]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:23:07.017248949 (13h43m28.761305819s) [upload=14122]: Done uploading segments... Aug 7 23:23:07.017274459 (13h43m28.761331331s) [upload=14122]: Upload finished. err: metaclient: context deadline exceeded Duration: 851.066713ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..........X.XX..X...........................................] duration: 2.784338572s FAIL (2025-08-07 23:49:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.784356012s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 8 out of 60 is failed (threshold=3) [.............XXXXXX.XX......................................] duration: 2.264406355s FAIL (2025-08-07 23:49:41 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.264433362s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 6 out of 60 is failed (threshold=3) [..............XXX...XX.X....................................] duration: 1.008478067s FAIL (2025-08-07 23:49:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.008500034s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [..........XXXXXXXXXX...........X............................] duration: 2.606074715s FAIL (2025-08-07 23:49:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 0:4517 , piece_id=CIP2H4JR7JNEVTTPEDFI7FVDVPI66H6TNQTP7WDTWO3EFDNJWKXA, seg_pos={0 0}, upload=14216]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Aug 7 23:27:57.305395174 (13h48m19.049452024s) [seg_pos={0 0}, upload=14216]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:27:57.305491984 (13h48m19.049548844s) [upload=14216]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:27:57.305496984 (13h48m19.049553844s) [upload=14216]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:27:57.453740728 (13h48m19.197797589s) [upload=14216]: Done uploading segments... Aug 7 23:27:57.453760898 (13h48m19.197817752s) [upload=14216]: Upload finished. err: <nil> Aug 7 23:28:48.175431324 (13h49m9.919488199s) [upload=14229]: Starting upload Aug 7 23:28:48.175526742 (13h49m9.919583612s) [upload=14229]: Done starting upload Aug 7 23:28:48.175776798 (13h49m9.919833649s) [upload=14229]: Uploading segments... Aug 7 23:28:48.175982529 (13h49m9.920039392s) [upload=14229]: Got next segment. Inline: false Aug 7 23:28:48.176023543 (13h49m9.920080406s) [upload=14229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:28:48.176029551 (13h49m9.920086414s) [upload=14229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:28:48.284809382 (13h49m10.02886623s) [seg_pos={0 0}, upload=14229]: Begin upload segment... Aug 7 23:28:48.284855416 (13h49m10.028912275s) [seg_pos={0 0}, upload=14229]: Done begin upload segment. Aug 7 23:28:48.284878655 (13h49m10.028935518s) [noise=true , address=77.90.31.13:4626 , piece_id=QH4O3MHPFYQ6LH4Y6KGGELCSO5KRAHBPXGX4BWZLPTH2EBUBEBZA, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.285087631 (13h49m10.029144514s) [noise=true , address=77.90.31.17:4772 , piece_id=J6QWEIV73JC5VHU5BYRA7WBN5N7QOVTDIEOI4GQIL5MDDY3F32PQ, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.285200468 (13h49m10.029257328s) [noise=true , address=45.91.116.6:4975 , piece_id=VBM5UD6OVZCMPJ6QF664V2ZACFG6UQH6SOPHJFUE4WMYGZZYGCKQ, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.285437499 (13h49m10.029494358s) [noise=true , address=77.90.31.11:4553 , piece_id=WHCCT2GMH2DRWLK7TCINB4JR5C5SETU3XS2ARSIIWQN52E2EFVKQ, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.285628611 (13h49m10.029685478s) [noise=true , address=45.91.116.14:4190 , piece_id=XVD5ISVMY4722XYPEDTYPOHOGYL7XDNPBDNBMUQZNZJLLLYWIXFQ, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.285699438 (13h49m10.029756306s) [noise=true , address=104.36.82.14:4221 , piece_id=6AFA2RGZEI6YLGAGYE6ZSPPBJHMRGQIMRJH44AVZPG5JKCKK2F7A, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.286010251 (13h49m10.030067111s) [noise=true , address=77.90.31.10:4520 , piece_id=S3FSHHVOQS5UVL27Q7GAXEDB3H2QHQZSL6Z36QP7UABBABT3AKQQ, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.286078142 (13h49m10.030135004s) [noise=true , address=104.36.82.13:4185 , piece_id=EVQM6WWKOM5HCW64MOQHEGEZNCGBPB6KHD7H4G47NHEI6FQUAFZQ, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.286284224 (13h49m10.030341088s) [noise=true , address=77.90.31.12:4590 , piece_id=ZKPV2LZS65UFVUK5JWZAWAAVEQCELDU5FZQTU37PAISE6TWMEVKA, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.286553378 (13h49m10.03061024s) [noise=true , address=77.90.31.16:4733 , piece_id=2W4RBFY3CABLH7JFOCISDF4MP6H4EN6CN5F3GF7HANAJPE26TSHA, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.286568001 (13h49m10.030624861s) [noise=true , address=104.36.82.10:4077 , piece_id=4E23BJAD735PGV45SNNRBJQUFEWVUI4JLJDLZHNUXJXWGIQVQPEQ, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.286799362 (13h49m10.030856223s) [noise=true , address=77.90.31.12:4592 , piece_id=B3J4KUKWAUPNMX775I6UU4PD5XXGWECMLNTHPAVWCMFA4GZZQV2Q, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.287294442 (13h49m10.031351312s) [noise=true , address=45.91.116.7:4010 , piece_id=KFN54772PPRIEMWW35UTAFHODB2YVX3NFFBDH6HVIHXHVLBFUJLA, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.287736301 (13h49m10.031793162s) [noise=true , address=77.90.31.19:4841 , piece_id=ZPYXEINGBZIFNFXDANSECW7S7NLRQKVWOTRFUHXRG7YPIM6VUZYA, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.288128520 (13h49m10.032185379s) [noise=true , address=45.91.116.13:4155 , piece_id=L5FRG63YLCXXLHOO2NGEKCMXW3HQXNGUTPONX76ZEHQDIJG76KEQ, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.288446325 (13h49m10.032503213s) [noise=true , address=77.90.31.19:4843 , piece_id=725EIRIK675YXHGOPGBBGXGK4BMMUDOHQOBL36MCDSYUE5XKTI6Q, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.381456643 (13h49m10.125513496s) [noise=true , address=77.90.31.13:4626 , piece_id=QH4O3MHPFYQ6LH4Y6KGGELCSO5KRAHBPXGX4BWZLPTH2EBUBEBZA, seg_pos={0 0}, upload=14229]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 139.59.144.22:50332->77.90.31.13:4626: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:28:48.394653940 (13h49m10.138710812s) [noise=true , address=77.90.31.16:4733 , piece_id=2W4RBFY3CABLH7JFOCISDF4MP6H4EN6CN5F3GF7HANAJPE26TSHA, seg_pos={0 0}, upload=14229]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 139.59.144.22:43998->77.90.31.16:4733: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:28:48.897346092 (13h49m10.641402943s) [upload=14229]: Next returned nil segment Aug 7 23:28:48.897427052 (13h49m10.641483913s) [upload=14229]: Waiting for error group managing segments... Aug 7 23:28:49.066298229 (13h49m10.810355077s) [noise=true , address=77.90.31.12:4592 , piece_id=B3J4KUKWAUPNMX775I6UU4PD5XXGWECMLNTHPAVWCMFA4GZZQV2Q, seg_pos={0 0}, upload=14229]: Done uploading piece. err: <nil> Aug 7 23:28:49.082629995 (13h49m10.826686848s) [noise=true , address=104.36.82.13:4185 , piece_id=EVQM6WWKOM5HCW64MOQHEGEZNCGBPB6KHD7H4G47NHEI6FQUAFZQ, seg_pos={0 0}, upload=14229]: Done uploading piece. err: <nil> Aug 7 23:28:49.085963999 (13h49m10.830020845s) [noise=true , address=104.36.82.10:4077 , piece_id=4E23BJAD735PGV45SNNRBJQUFEWVUI4JLJDLZHNUXJXWGIQVQPEQ, seg_pos={0 0}, upload=14229]: Done uploading piece. err: <nil> Aug 7 23:28:49.091538387 (13h49m10.83559524s) [noise=true , address=77.90.31.12:4590 , piece_id=ZKPV2LZS65UFVUK5JWZAWAAVEQCELDU5FZQTU37PAISE6TWMEVKA, seg_pos={0 0}, upload=14229]: Done uploading piece. err: <nil> Aug 7 23:28:49.120549301 (13h49m10.864606147s) [noise=true , address=77.90.31.11:4553 , piece_id=WHCCT2GMH2DRWLK7TCINB4JR5C5SETU3XS2ARSIIWQN52E2EFVKQ, seg_pos={0 0}, upload=14229]: Done uploading piece. err: <nil> Aug 7 23:28:49.126986818 (13h49m10.871043668s) [noise=true , address=77.90.31.17:4772 , piece_id=J6QWEIV73JC5VHU5BYRA7WBN5N7QOVTDIEOI4GQIL5MDDY3F32PQ, seg_pos={0 0}, upload=14229]: Done uploading piece. err: <nil> Aug 7 23:28:49.133663068 (13h49m10.877719922s) [noise=true , address=45.91.116.7:4010 , piece_id=KFN54772PPRIEMWW35UTAFHODB2YVX3NFFBDH6HVIHXHVLBFUJLA, seg_pos={0 0}, upload=14229]: Done uploading piece. err: <nil> Aug 7 23:28:49.151613435 (13h49m10.895670288s) [noise=true , address=45.91.116.13:4155 , piece_id=L5FRG63YLCXXLHOO2NGEKCMXW3HQXNGUTPONX76ZEHQDIJG76KEQ, seg_pos={0 0}, upload=14229]: Done uploading piece. err: <nil> Aug 7 23:28:49.179832443 (13h49m10.923889293s) [noise=true , address=45.91.116.14:4190 , piece_id=XVD5ISVMY4722XYPEDTYPOHOGYL7XDNPBDNBMUQZNZJLLLYWIXFQ, seg_pos={0 0}, upload=14229]: Done uploading piece. err: <nil> Aug 7 23:28:49.235424834 (13h49m10.979481694s) [noise=true , address=77.90.31.10:4520 , piece_id=S3FSHHVOQS5UVL27Q7GAXEDB3H2QHQZSL6Z36QP7UABBABT3AKQQ, seg_pos={0 0}, upload=14229]: Done uploading piece. err: <nil> Aug 7 23:28:49.253356810 (13h49m10.997413672s) [noise=true , address=104.36.82.14:4221 , piece_id=6AFA2RGZEI6YLGAGYE6ZSPPBJHMRGQIMRJH44AVZPG5JKCKK2F7A, seg_pos={0 0}, upload=14229]: Done uploading piece. err: <nil> Aug 7 23:28:49.314647975 (13h49m11.058704826s) [noise=true , address=45.91.116.6:4975 , piece_id=VBM5UD6OVZCMPJ6QF664V2ZACFG6UQH6SOPHJFUE4WMYGZZYGCKQ, seg_pos={0 0}, upload=14229]: Done uploading piece. err: <nil> Aug 7 23:29:17.906721377 (13h49m39.650778264s) [noise=true , address=77.90.31.19:4841 , piece_id=ZPYXEINGBZIFNFXDANSECW7S7NLRQKVWOTRFUHXRG7YPIM6VUZYA, seg_pos={0 0}, upload=14229]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:29:17.906810237 (13h49m39.650867112s) [noise=true , address=77.90.31.19:4843 , piece_id=725EIRIK675YXHGOPGBBGXGK4BMMUDOHQOBL36MCDSYUE5XKTI6Q, seg_pos={0 0}, upload=14229]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 139.59.144.22:58688->77.90.31.19:4843: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:29:17.906834622 (13h49m39.650891505s) [seg_pos={0 0}, upload=14229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Aug 7 23:29:18.017467630 (13h49m39.761524483s) [upload=14229]: Done uploading segments... Aug 7 23:29:18.017499122 (13h49m39.761555972s) [upload=14229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.60610135s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...X..................................X.....................] duration: 3.215197163s FAIL (2025-08-07 23:49:33 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== 294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:32:45.172127356 (13h53m6.916184241s) [upload=14294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:32:45.390149151 (13h53m7.134206011s) [seg_pos={0 0}, upload=14294]: Begin upload segment... Aug 7 23:32:45.390215910 (13h53m7.134272761s) [seg_pos={0 0}, upload=14294]: Done begin upload segment. Aug 7 23:32:45.390249193 (13h53m7.134306059s) [noise=true , address=104.36.82.19:4405 , piece_id=MHM3MWZJFTXSJT7PTOFSWHUYO3L6NIBUV6VU66XXZDH3MC3EFR2Q, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390272919 (13h53m7.134329788s) [noise=true , address=77.90.31.12:4593 , piece_id=C3RVQHFS4L7SKPFFIZ52MAQWGD7P4JNG3SNPCNY53SQUD72PZRUA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390658970 (13h53m7.13471583s) [noise=true , address=104.36.82.16:4321 , piece_id=YTF4RWV2SD6Z4UV4XRYPBRGPCTSUTC6TQKL3DHVKU65ALTNA4TEQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390852777 (13h53m7.134909659s) [noise=true , address=104.36.82.34:4566 , piece_id=3YEAUSXEL7WNZ3O7WM24Z2DKY2TDVKMRZSHMEPMNEL2LR7BY7DBQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390957991 (13h53m7.135014851s) [noise=true , address=104.36.82.47:4523 , piece_id=MJIMHSO66V7OWFFK2UALPKSI7CGGCTHNPZINDXMVIHDLENIW5LFA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.391483291 (13h53m7.135540162s) [noise=true , address=104.36.82.5:4861 , piece_id=GRDGO5LNCETQWRU3M6LACGRW5MNA2KFLXUBT5R2UH6SEGWP6RVRQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.391931094 (13h53m7.135987964s) [noise=true , address=104.36.82.10:4090 , piece_id=V6NPPVIYEH6WLIW7OMT3NDFPR7UDWIT46GW6AFCAWUJTBLC3O55Q, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392250266 (13h53m7.136307136s) [noise=true , address=104.36.82.62:4439 , piece_id=ILRD4TEFW4PSLV4EGPORG3FM6U2HB3WSL5KP3PN566FAOGWODLYQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392611689 (13h53m7.136668559s) [noise=true , address=104.36.82.54:4160 , piece_id=GDBORRYX5QBQGH7ZMMUUAV7AAR5UHZOTK5FTQUPC7YKZC2KD7I3A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392917410 (13h53m7.136974282s) [noise=true , address=104.36.82.23:4477 , piece_id=HAF6424ZCIBGSR4US5LN5ZWE5JMNMLFGUMQQ4YP2ARP7PQXBH5GQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393270536 (13h53m7.137327398s) [noise=true , address=104.36.82.44:4881 , piece_id=XXVSXG3ILFXG4IVMELHL45TRRHDFVVJMJ5LGOHU5EDHG6XXCDH6A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393582635 (13h53m7.137639498s) [noise=true , address=104.36.82.37:4665 , piece_id=RUKKGC4Y3FIVLZO2GRWJN3J6P36VWP5VSC2HCLEU3MZDDFXODJAQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393856926 (13h53m7.137913788s) [noise=true , address=104.36.82.11:4128 , piece_id=XRPULHI7BYDF2JOIASSN5QKGEAC6FJS2F5C22WIY7EKYILRHZ73A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394196364 (13h53m7.138253227s) [noise=true , address=104.36.82.38:4706 , piece_id=BIHIPQX6LRWBNFN4SDC3K3V22AMGL3RC5B4RVE7QXJFKB4XDN3BA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394484397 (13h53m7.138541257s) [noise=true , address=104.36.82.61:4381 , piece_id=TTZOSRP6YPL6P4C6PY5S4K7Z6CIZJDPX7NUNBCK73CWZPCA3HQHA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394593709 (13h53m7.138650578s) [noise=true , address=45.91.116.5:4300 , piece_id=MADJCANN2O35MKXT2PIR5PGMJ3PNTAQAJNNGOK6ISJNVAH2L7YIA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.395093013 (13h53m7.139149879s) [noise=true , address=104.36.82.22:4530 , piece_id=HMUUW5GYH6TRHYXP4YXBHD566FWQRILEHI53AVURA5QTQGG6USFA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.395447945 (13h53m7.139504805s) [noise=true , address=104.36.82.58:4293 , piece_id=I2ETFXYTHYWYJYSHTVOBQTO6ZPGNMA4D3NVKYDE2W34EH4SQE23A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.954469844 (13h53m7.698526694s) [upload=14294]: Next returned nil segment Aug 7 23:32:45.954487953 (13h53m7.69854487s) [upload=14294]: Waiting for error group managing segments... Aug 7 23:32:46.126888627 (13h53m7.870945474s) [noise=true , address=104.36.82.10:4090 , piece_id=V6NPPVIYEH6WLIW7OMT3NDFPR7UDWIT46GW6AFCAWUJTBLC3O55Q, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.132983844 (13h53m7.877040704s) [noise=true , address=104.36.82.16:4321 , piece_id=YTF4RWV2SD6Z4UV4XRYPBRGPCTSUTC6TQKL3DHVKU65ALTNA4TEQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.138277216 (13h53m7.882334067s) [noise=true , address=104.36.82.23:4477 , piece_id=HAF6424ZCIBGSR4US5LN5ZWE5JMNMLFGUMQQ4YP2ARP7PQXBH5GQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.142033326 (13h53m7.886090185s) [noise=true , address=104.36.82.34:4566 , piece_id=3YEAUSXEL7WNZ3O7WM24Z2DKY2TDVKMRZSHMEPMNEL2LR7BY7DBQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.151247206 (13h53m7.895304058s) [noise=true , address=104.36.82.22:4530 , piece_id=HMUUW5GYH6TRHYXP4YXBHD566FWQRILEHI53AVURA5QTQGG6USFA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.151981081 (13h53m7.896037932s) [noise=true , address=104.36.82.58:4293 , piece_id=I2ETFXYTHYWYJYSHTVOBQTO6ZPGNMA4D3NVKYDE2W34EH4SQE23A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.165022882 (13h53m7.909079743s) [noise=true , address=104.36.82.44:4881 , piece_id=XXVSXG3ILFXG4IVMELHL45TRRHDFVVJMJ5LGOHU5EDHG6XXCDH6A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.184418821 (13h53m7.928475667s) [noise=true , address=104.36.82.47:4523 , piece_id=MJIMHSO66V7OWFFK2UALPKSI7CGGCTHNPZINDXMVIHDLENIW5LFA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.184532669 (13h53m7.928589516s) [noise=true , address=45.91.116.5:4300 , piece_id=MADJCANN2O35MKXT2PIR5PGMJ3PNTAQAJNNGOK6ISJNVAH2L7YIA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.193375808 (13h53m7.937432656s) [noise=true , address=104.36.82.54:4160 , piece_id=GDBORRYX5QBQGH7ZMMUUAV7AAR5UHZOTK5FTQUPC7YKZC2KD7I3A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.204247501 (13h53m7.948304372s) [noise=true , address=104.36.82.38:4706 , piece_id=BIHIPQX6LRWBNFN4SDC3K3V22AMGL3RC5B4RVE7QXJFKB4XDN3BA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.235225782 (13h53m7.979282629s) [noise=true , address=104.36.82.62:4439 , piece_id=ILRD4TEFW4PSLV4EGPORG3FM6U2HB3WSL5KP3PN566FAOGWODLYQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.269129867 (13h53m8.013186721s) [noise=true , address=104.36.82.5:4861 , piece_id=GRDGO5LNCETQWRU3M6LACGRW5MNA2KFLXUBT5R2UH6SEGWP6RVRQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.278216912 (13h53m8.022273764s) [noise=true , address=104.36.82.19:4405 , piece_id=MHM3MWZJFTXSJT7PTOFSWHUYO3L6NIBUV6VU66XXZDH3MC3EFR2Q, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.278247847 (13h53m8.022304708s) [seg_pos={0 0}, upload=14294]: Segment reached optimal threshold of 14 pieces. Aug 7 23:32:46.278485612 (13h53m8.022542463s) [noise=true , address=104.36.82.11:4128 , piece_id=XRPULHI7BYDF2JOIASSN5QKGEAC6FJS2F5C22WIY7EKYILRHZ73A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wABkAtAjoYkKyNoAiaVCxSvY6WgvHvAYFPCjpNojtpiYiGtDS): protocol: expected piece hash; context canceled; EOF Aug 7 23:32:46.278511698 (13h53m8.022568566s) [noise=true , address=77.90.31.12:4593 , piece_id=C3RVQHFS4L7SKPFFIZ52MAQWGD7P4JNG3SNPCNY53SQUD72PZRUA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kSxoZb9nmG6oGNytpFTsoiGh3ZPmebVVe8TZxGzANaw7EfdeF): protocol: expected piece hash; context canceled; EOF Aug 7 23:32:46.278642807 (13h53m8.022699657s) [noise=true , address=104.36.82.61:4381 , piece_id=TTZOSRP6YPL6P4C6PY5S4K7Z6CIZJDPX7NUNBCK73CWZPCA3HQHA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zU35MWWaaC1npkP5qGcWBXAkfxGsd5UsdkM2nwX9x7wUGCDbY): protocol: write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection; write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection Aug 7 23:32:46.278650926 (13h53m8.022707786s) [noise=true , address=104.36.82.37:4665 , piece_id=RUKKGC4Y3FIVLZO2GRWJN3J6P36VWP5VSC2HCLEU3MZDDFXODJAQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jP9JrCmoQSdqrzXZxkNN1ihdFkaUMjGZss6Fh19zbL3YyWj5N): protocol: write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection; write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection Aug 7 23:32:46.278674050 (13h53m8.022730911s) [seg_pos={0 0}, upload=14294]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:32:46.278751154 (13h53m8.022808014s) [upload=14294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:32:46.278756168 (13h53m8.022813029s) [upload=14294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:32:46.498660020 (13h53m8.242716882s) [upload=14294]: Done uploading segments... Aug 7 23:32:46.498701263 (13h53m8.242758128s) [upload=14294]: Upload finished. err: <nil> Duration: 3.215229552s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [..............X...X.XX......................................] duration: 3.198559046s FAIL (2025-08-07 23:49:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== inished. err: context deadline exceeded Aug 7 23:20:07.202369543 (13h40m28.946426394s) [upload=14082]: Starting upload Aug 7 23:20:07.202426657 (13h40m28.946483509s) [upload=14082]: Done starting upload Aug 7 23:20:07.202696950 (13h40m28.946753814s) [upload=14082]: Uploading segments... Aug 7 23:20:07.202761256 (13h40m28.94681811s) [upload=14082]: Got next segment. Inline: false Aug 7 23:20:07.202807046 (13h40m28.946863908s) [upload=14082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:20:07.202814233 (13h40m28.946871102s) [upload=14082]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:20:45.722093118 (13h41m7.466149983s) [seg_pos={0 0}, upload=14082]: Begin upload segment... Aug 7 23:20:45.722177165 (13h41m7.466234018s) [seg_pos={0 0}, upload=14082]: Done begin upload segment. Aug 7 23:20:45.722218538 (13h41m7.4662754s) [noise=true , address=79.127.205.246:4276 , piece_id=6753PO7TQBULISFOSTSIT32I7DSKQPM6M4TEVF2E7TILSMUU5ARQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.722246298 (13h41m7.46630316s) [noise=true , address=104.36.82.58:4292 , piece_id=KQ2RNSZDMCU7OED4IJC4ERBTMTYVGHJCRSVOFZUXG24LW7H3KHUQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.722330454 (13h41m7.466387316s) [noise=true , address=64.49.13.169:4705 , piece_id=YJCLDFSO5ENFREIROYPEZQQTQD4GM4A3DFKNJEHSPFDOKDCFEJ7Q, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.722658583 (13h41m7.466715506s) [noise=true , address=79.127.205.234:4597 , piece_id=WETMAYMNBWD6F6JYCXPNAG5KOJXNCXE3KM7ZJ4F5Y3GEBPHLMDEA, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.722717191 (13h41m7.46677405s) [noise=true , address=104.36.82.5:4867 , piece_id=IEDY33GIP4MZRSN2MKKBLDFAXPMKJUPZVPMOXVCEFJ6BUPZJHYHQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.722989701 (13h41m7.467046568s) [noise=true , address=64.49.13.139:4474 , piece_id=LGXAPMGRQOOEXXG7EBNLO7GAPDKFBCQIBWGYID7KOVQ3UNMDZICQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.723061064 (13h41m7.467117927s) [noise=true , address=45.91.116.14:4201 , piece_id=LCOEAQDFX432OULFXCKDH3FQMZMPR5XSDVKK5MAPEQYEABUAYMDA, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.723458393 (13h41m7.467515259s) [noise=true , address=104.36.82.61:4392 , piece_id=EL44CRMZDDTXOW45GURIDIFQF4K65UUJFKR6L3F3WQBMVXSD2XHQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.729867582 (13h41m7.473924444s) [noise=true , address=104.36.82.60:4345 , piece_id=2AKG3BCGW7TFNVG3FQ576YJOTEZV3V2NP5ITSSZ7BO3NO5CRVDVQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.731063643 (13h41m7.475120512s) [noise=true , address=104.36.82.9:4016 , piece_id=VTTRIIBZ4KCGJW2H5DMG7J63AGFIV7QT34MJJTHRUGFIH6JD6EQQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.732350031 (13h41m7.476406894s) [noise=true , address=104.36.82.52:4106 , piece_id=KTACAAM5SP3PCN3VE6CKF3KHMLGYCHONLC7FDUI4YHLFKIEJ65HQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.733989697 (13h41m7.478046556s) [noise=true , address=64.49.13.155:4916 , piece_id=PJ3VBT33DUHLG3WHE6AK7IIYX76IZZFJCAWX7WX3UUBA6BLGB5VA, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.734727605 (13h41m7.478784474s) [noise=true , address=104.36.82.37:4680 , piece_id=2MDE2TCPX2ROXJXK7Q234YXMK4BFHPDQB5EB7GECHLMUJY5W4ITQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.734786637 (13h41m7.478843504s) [noise=true , address=104.36.82.4:4813 , piece_id=IZO235F25YVNCQG2GSR3UPI5OPMONHVRQQAWDEBOZWR2EILWGCCA, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.735460668 (13h41m7.479517532s) [noise=true , address=104.36.82.6:4901 , piece_id=7VX47BXB37LW2CKYFTBTFAJSDF3QLHEHQHRD5ZM5EHZPAZFPXBPQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.735486537 (13h41m7.479543401s) [noise=true , address=104.36.82.16:4323 , piece_id=RQYKNED6IWTV47W7HGZCVVOUKFRMHGO5WSPCW36ROCXMEDEFYM4Q, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.736565943 (13h41m7.480622804s) [noise=true , address=104.36.82.36:4651 , piece_id=5HQECPYGS5S43JV2CEJY3MRS33NUE6CJ5WPXKTCWYLYVXUQDXLXQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.737386328 (13h41m7.48144319s) [noise=true , address=104.36.82.44:4876 , piece_id=3WUHXX7GVGZWOGDA7FETGVT772RLHO4HRV6OCZKNTZIMG3VDPDOA, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:46.299198843 (13h41m8.043255689s) [upload=14082]: Next returned nil segment Aug 7 23:20:46.299217870 (13h41m8.043274733s) [upload=14082]: Waiting for error group managing segments... Aug 7 23:20:46.464881860 (13h41m8.208938721s) [noise=true , address=104.36.82.60:4345 , piece_id=2AKG3BCGW7TFNVG3FQ576YJOTEZV3V2NP5ITSSZ7BO3NO5CRVDVQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.473775294 (13h41m8.217832164s) [noise=true , address=104.36.82.52:4106 , piece_id=KTACAAM5SP3PCN3VE6CKF3KHMLGYCHONLC7FDUI4YHLFKIEJ65HQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.474664669 (13h41m8.218721541s) [noise=true , address=104.36.82.9:4016 , piece_id=VTTRIIBZ4KCGJW2H5DMG7J63AGFIV7QT34MJJTHRUGFIH6JD6EQQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.479688336 (13h41m8.223745197s) [noise=true , address=104.36.82.36:4651 , piece_id=5HQECPYGS5S43JV2CEJY3MRS33NUE6CJ5WPXKTCWYLYVXUQDXLXQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.485171804 (13h41m8.229228658s) [noise=true , address=104.36.82.6:4901 , piece_id=7VX47BXB37LW2CKYFTBTFAJSDF3QLHEHQHRD5ZM5EHZPAZFPXBPQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.486836180 (13h41m8.230893032s) [noise=true , address=104.36.82.61:4392 , piece_id=EL44CRMZDDTXOW45GURIDIFQF4K65UUJFKR6L3F3WQBMVXSD2XHQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.496476947 (13h41m8.240533796s) [noise=true , address=104.36.82.44:4876 , piece_id=3WUHXX7GVGZWOGDA7FETGVT772RLHO4HRV6OCZKNTZIMG3VDPDOA, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.556494206 (13h41m8.300551081s) [noise=true , address=104.36.82.16:4323 , piece_id=RQYKNED6IWTV47W7HGZCVVOUKFRMHGO5WSPCW36ROCXMEDEFYM4Q, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.628743021 (13h41m8.372799882s) [noise=true , address=45.91.116.14:4201 , piece_id=LCOEAQDFX432OULFXCKDH3FQMZMPR5XSDVKK5MAPEQYEABUAYMDA, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.645987246 (13h41m8.390044106s) [noise=true , address=104.36.82.4:4813 , piece_id=IZO235F25YVNCQG2GSR3UPI5OPMONHVRQQAWDEBOZWR2EILWGCCA, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.666511132 (13h41m8.410567992s) [noise=true , address=104.36.82.5:4867 , piece_id=IEDY33GIP4MZRSN2MKKBLDFAXPMKJUPZVPMOXVCEFJ6BUPZJHYHQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.671756371 (13h41m8.41581322s) [noise=true , address=104.36.82.37:4680 , piece_id=2MDE2TCPX2ROXJXK7Q234YXMK4BFHPDQB5EB7GECHLMUJY5W4ITQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.934025878 (13h41m8.678082729s) [noise=true , address=79.127.205.234:4597 , piece_id=WETMAYMNBWD6F6JYCXPNAG5KOJXNCXE3KM7ZJ4F5Y3GEBPHLMDEA, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.970907150 (13h41m8.714964016s) [noise=true , address=64.49.13.139:4474 , piece_id=LGXAPMGRQOOEXXG7EBNLO7GAPDKFBCQIBWGYID7KOVQ3UNMDZICQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.970940415 (13h41m8.714997268s) [seg_pos={0 0}, upload=14082]: Segment reached optimal threshold of 14 pieces. Aug 7 23:20:46.971273207 (13h41m8.715330074s) [noise=true , address=64.49.13.155:4916 , piece_id=PJ3VBT33DUHLG3WHE6AK7IIYX76IZZFJCAWX7WX3UUBA6BLGB5VA, seg_pos={0 0}, upload=14082]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12MD2UDNXEmeGra9fvhEb8eGL2R6vGLHtSKLpSyX3H3tZCe1r5B): protocol: expected piece hash; context canceled; EOF Aug 7 23:20:46.971460895 (13h41m8.715517755s) [noise=true , address=64.49.13.169:4705 , piece_id=YJCLDFSO5ENFREIROYPEZQQTQD4GM4A3DFKNJEHSPFDOKDCFEJ7Q, seg_pos={0 0}, upload=14082]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124uqzVwsF5TM4CwUhkzfJvgxd71jZXgXuFNwDLRW1vrSXbwLsr): protocol: expected piece hash; context canceled; EOF Aug 7 23:20:46.971541847 (13h41m8.715598706s) [noise=true , address=79.127.205.246:4276 , piece_id=6753PO7TQBULISFOSTSIT32I7DSKQPM6M4TEVF2E7TILSMUU5ARQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJTNVtPFN48S9ZrayehLD2HcVckyJWLx2rtEvyH56W22R8Suq): protocol: expected piece hash; context canceled; EOF Aug 7 23:20:46.971599101 (13h41m8.715655962s) [noise=true , address=104.36.82.58:4292 , piece_id=KQ2RNSZDMCU7OED4IJC4ERBTMTYVGHJCRSVOFZUXG24LW7H3KHUQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JeDjqgszkkq2asUBCL5oTvH6X21TnWkkmqGEEzBbz1YctdDXX): protocol: expected piece hash; context canceled; EOF Aug 7 23:20:46.971618718 (13h41m8.715675568s) [seg_pos={0 0}, upload=14082]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:20:46.971698846 (13h41m8.715755698s) [upload=14082]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:20:46.971704848 (13h41m8.71576171s) [upload=14082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:21:07.219673652 (13h41m28.963730505s) [upload=14082]: Done uploading segments... Aug 7 23:21:07.219701493 (13h41m28.963758344s) [upload=14082]: Upload finished. err: metaclient: context deadline exceeded Duration: 3.198585218s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 7 out of 60 is failed (threshold=3) [..........X.XXXXX.X.........................................] duration: 2.89692124s FAIL (2025-08-07 23:49:28 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.896965693s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 7 out of 60 is failed (threshold=3) [.......XX..XXXXX............................................] duration: 859.792162ms FAIL (2025-08-07 23:49:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== :10:03.085332650 (13h30m24.829389513s) [upload=13904]: Done starting upload Aug 7 23:10:03.085414170 (13h30m24.829471025s) [upload=13904]: Uploading segments... Aug 7 23:10:03.085475435 (13h30m24.829532308s) [upload=13904]: Got next segment. Inline: true Aug 7 23:10:03.085480843 (13h30m24.829537706s) [upload=13904]: Waiting for error group managing segments... Aug 7 23:10:03.085544347 (13h30m24.829601211s) [upload=13904]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:03.085552551 (13h30m24.829609424s) [upload=13904]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:03.085555974 (13h30m24.829612844s) [upload=13904]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:10:03.247308964 (13h30m24.991365825s) [upload=13904]: Done uploading segments... Aug 7 23:10:03.247396260 (13h30m24.991453126s) [upload=13904]: Upload finished. err: <nil> Aug 7 23:10:40.522399484 (13h31m2.266456348s) [upload=13915]: Starting upload Aug 7 23:10:40.522487646 (13h31m2.266544496s) [upload=13915]: Done starting upload Aug 7 23:10:40.522571041 (13h31m2.266627902s) [upload=13915]: Uploading segments... Aug 7 23:10:40.522619425 (13h31m2.266676294s) [upload=13915]: Got next segment. Inline: true Aug 7 23:10:40.522626307 (13h31m2.266683183s) [upload=13915]: Waiting for error group managing segments... Aug 7 23:10:40.522684106 (13h31m2.266740971s) [upload=13915]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:40.522691484 (13h31m2.26674835s) [upload=13915]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:40.522696090 (13h31m2.266752961s) [upload=13915]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:40.254283342 (13h32m1.998340192s) [upload=13915]: Done uploading segments... Aug 7 23:11:40.254304548 (13h32m1.998361398s) [upload=13915]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:11:40.667902567 (13h32m2.411959413s) [upload=13933]: Starting upload Aug 7 23:11:40.667961940 (13h32m2.412018788s) [upload=13933]: Done starting upload Aug 7 23:11:40.668020524 (13h32m2.41207737s) [upload=13933]: Uploading segments... Aug 7 23:11:40.668075247 (13h32m2.412132113s) [upload=13933]: Got next segment. Inline: true Aug 7 23:11:40.668080664 (13h32m2.412137525s) [upload=13933]: Waiting for error group managing segments... Aug 7 23:11:40.668121190 (13h32m2.412178036s) [upload=13933]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:40.668125756 (13h32m2.412182616s) [upload=13933]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:40.668128431 (13h32m2.412185291s) [upload=13933]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:40.394243254 (13h33m2.138300118s) [upload=13933]: Done uploading segments... Aug 7 23:12:40.394270270 (13h33m2.138327132s) [upload=13933]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:12:40.798630406 (13h33m2.542687267s) [upload=13953]: Starting upload Aug 7 23:12:40.798725809 (13h33m2.54278266s) [upload=13953]: Done starting upload Aug 7 23:12:40.798811946 (13h33m2.542868796s) [upload=13953]: Uploading segments... Aug 7 23:12:40.798867069 (13h33m2.542923932s) [upload=13953]: Got next segment. Inline: true Aug 7 23:12:40.798873108 (13h33m2.542929959s) [upload=13953]: Waiting for error group managing segments... Aug 7 23:12:40.798918573 (13h33m2.542975422s) [upload=13953]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:40.798924603 (13h33m2.542981463s) [upload=13953]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:40.798939855 (13h33m2.542996716s) [upload=13953]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:40.942390421 (13h33m2.686447272s) [upload=13953]: Done uploading segments... Aug 7 23:12:40.942410212 (13h33m2.686467062s) [upload=13953]: Upload finished. err: <nil> Aug 7 23:13:18.137108974 (13h33m39.881165825s) [upload=13967]: Starting upload Aug 7 23:13:18.137180408 (13h33m39.881237292s) [upload=13967]: Done starting upload Aug 7 23:13:18.137258925 (13h33m39.881315786s) [upload=13967]: Uploading segments... Aug 7 23:13:18.137309635 (13h33m39.881366505s) [upload=13967]: Got next segment. Inline: true Aug 7 23:13:18.137314511 (13h33m39.881371371s) [upload=13967]: Waiting for error group managing segments... Aug 7 23:13:18.137374772 (13h33m39.881431633s) [upload=13967]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:18.137380412 (13h33m39.881437273s) [upload=13967]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:18.137383733 (13h33m39.881440593s) [upload=13967]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:18.301480519 (13h33m40.045537379s) [upload=13967]: Done uploading segments... Aug 7 23:13:18.301498918 (13h33m40.045555767s) [upload=13967]: Upload finished. err: <nil> Aug 7 23:14:13.169029066 (13h34m34.913085913s) [upload=13980]: Starting upload Aug 7 23:14:13.169095254 (13h34m34.913152101s) [upload=13980]: Done starting upload Aug 7 23:14:13.169153338 (13h34m34.913210184s) [upload=13980]: Uploading segments... Aug 7 23:14:13.169225662 (13h34m34.913282508s) [upload=13980]: Got next segment. Inline: true Aug 7 23:14:13.169230885 (13h34m34.913287744s) [upload=13980]: Waiting for error group managing segments... Aug 7 23:14:13.169272521 (13h34m34.913329381s) [upload=13980]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:13.169288952 (13h34m34.913345813s) [upload=13980]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:13.169291623 (13h34m34.913348483s) [upload=13980]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:12.895960606 (13h35m34.640017467s) [upload=13980]: Done uploading segments... Aug 7 23:15:12.895983608 (13h35m34.640040456s) [upload=13980]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:15:13.308146067 (13h35m35.052202918s) [upload=13997]: Starting upload Aug 7 23:15:13.308219271 (13h35m35.052276134s) [upload=13997]: Done starting upload Aug 7 23:15:13.308297319 (13h35m35.052354171s) [upload=13997]: Uploading segments... Aug 7 23:15:13.308360203 (13h35m35.052417072s) [upload=13997]: Got next segment. Inline: true Aug 7 23:15:13.308365548 (13h35m35.052422408s) [upload=13997]: Waiting for error group managing segments... Aug 7 23:15:13.308409794 (13h35m35.052466657s) [upload=13997]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:15:13.308426102 (13h35m35.052482963s) [upload=13997]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:15:13.308429725 (13h35m35.052486585s) [upload=13997]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:13.040589970 (13h36m34.784646843s) [upload=13997]: Done uploading segments... Aug 7 23:16:13.040614428 (13h36m34.784671274s) [upload=13997]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:16:20.767866726 (13h36m42.511923577s) [upload=14018]: Starting upload Aug 7 23:16:20.767927544 (13h36m42.511984398s) [upload=14018]: Done starting upload Aug 7 23:16:20.767993090 (13h36m42.512049942s) [upload=14018]: Uploading segments... Aug 7 23:16:20.768055054 (13h36m42.512111926s) [upload=14018]: Got next segment. Inline: true Aug 7 23:16:20.768061500 (13h36m42.512118351s) [upload=14018]: Waiting for error group managing segments... Aug 7 23:16:20.768130604 (13h36m42.512187577s) [upload=14018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:20.768139327 (13h36m42.512196192s) [upload=14018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:20.768144987 (13h36m42.512201849s) [upload=14018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:20.496845379 (13h37m42.240902231s) [upload=14018]: Done uploading segments... Aug 7 23:17:20.496873778 (13h37m42.240930632s) [upload=14018]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:17:20.916777280 (13h37m42.660834146s) [upload=14032]: Starting upload Aug 7 23:17:20.916877286 (13h37m42.660934149s) [upload=14032]: Done starting upload Aug 7 23:17:20.917002557 (13h37m42.661059408s) [upload=14032]: Uploading segments... Aug 7 23:17:20.917058206 (13h37m42.661115077s) [upload=14032]: Got next segment. Inline: true Aug 7 23:17:20.917064317 (13h37m42.661121178s) [upload=14032]: Waiting for error group managing segments... Aug 7 23:17:20.917116347 (13h37m42.661173214s) [upload=14032]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:17:20.917123150 (13h37m42.661180014s) [upload=14032]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:17:20.917127779 (13h37m42.661184648s) [upload=14032]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:18:20.640884349 (13h38m42.384941214s) [upload=14032]: Done uploading segments... Aug 7 23:18:20.640914749 (13h38m42.384971617s) [upload=14032]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:18:21.050365938 (13h38m42.79442281s) [upload=14051]: Starting upload Aug 7 23:18:21.050484720 (13h38m42.794541587s) [upload=14051]: Done starting upload Aug 7 23:18:21.050589902 (13h38m42.794646752s) [upload=14051]: Uploading segments... Aug 7 23:18:21.050663797 (13h38m42.794720667s) [upload=14051]: Got next segment. Inline: true Aug 7 23:18:21.050670451 (13h38m42.794727311s) [upload=14051]: Waiting for error group managing segments... Aug 7 23:18:21.050719413 (13h38m42.794776275s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:18:21.050725163 (13h38m42.794782023s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:18:21.050728952 (13h38m42.794785811s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:19:20.780076504 (13h39m42.524133366s) [upload=14051]: Done uploading segments... Aug 7 23:19:20.780099469 (13h39m42.52415633s) [upload=14051]: Upload finished. err: metaclient: context deadline exceeded Duration: 859.819896ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 6 out of 60 is failed (threshold=3) [...............XXX...XX.X...................................] duration: 1.101289568s FAIL (2025-08-07 23:49:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.101378168s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 10 out of 60 is failed (threshold=3) [..........XXXXX.XX.XX.X.....................................] duration: 905.577227ms FAIL (2025-08-07 23:49:13 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 905.596361ms core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 7 out of 60 is failed (threshold=3) [...........X.XXXXX.X........................................] duration: 3.46076305s FAIL (2025-08-07 23:49:13 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.460796203s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 322.304006ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 23:49:02 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 322.33637ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............X...X.XX.....................................] duration: 3.169776573s FAIL (2025-08-07 23:49:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== inished. err: context deadline exceeded Aug 7 23:20:07.202369543 (13h40m28.946426394s) [upload=14082]: Starting upload Aug 7 23:20:07.202426657 (13h40m28.946483509s) [upload=14082]: Done starting upload Aug 7 23:20:07.202696950 (13h40m28.946753814s) [upload=14082]: Uploading segments... Aug 7 23:20:07.202761256 (13h40m28.94681811s) [upload=14082]: Got next segment. Inline: false Aug 7 23:20:07.202807046 (13h40m28.946863908s) [upload=14082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:20:07.202814233 (13h40m28.946871102s) [upload=14082]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:20:45.722093118 (13h41m7.466149983s) [seg_pos={0 0}, upload=14082]: Begin upload segment... Aug 7 23:20:45.722177165 (13h41m7.466234018s) [seg_pos={0 0}, upload=14082]: Done begin upload segment. Aug 7 23:20:45.722218538 (13h41m7.4662754s) [noise=true , address=79.127.205.246:4276 , piece_id=6753PO7TQBULISFOSTSIT32I7DSKQPM6M4TEVF2E7TILSMUU5ARQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.722246298 (13h41m7.46630316s) [noise=true , address=104.36.82.58:4292 , piece_id=KQ2RNSZDMCU7OED4IJC4ERBTMTYVGHJCRSVOFZUXG24LW7H3KHUQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.722330454 (13h41m7.466387316s) [noise=true , address=64.49.13.169:4705 , piece_id=YJCLDFSO5ENFREIROYPEZQQTQD4GM4A3DFKNJEHSPFDOKDCFEJ7Q, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.722658583 (13h41m7.466715506s) [noise=true , address=79.127.205.234:4597 , piece_id=WETMAYMNBWD6F6JYCXPNAG5KOJXNCXE3KM7ZJ4F5Y3GEBPHLMDEA, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.722717191 (13h41m7.46677405s) [noise=true , address=104.36.82.5:4867 , piece_id=IEDY33GIP4MZRSN2MKKBLDFAXPMKJUPZVPMOXVCEFJ6BUPZJHYHQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.722989701 (13h41m7.467046568s) [noise=true , address=64.49.13.139:4474 , piece_id=LGXAPMGRQOOEXXG7EBNLO7GAPDKFBCQIBWGYID7KOVQ3UNMDZICQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.723061064 (13h41m7.467117927s) [noise=true , address=45.91.116.14:4201 , piece_id=LCOEAQDFX432OULFXCKDH3FQMZMPR5XSDVKK5MAPEQYEABUAYMDA, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.723458393 (13h41m7.467515259s) [noise=true , address=104.36.82.61:4392 , piece_id=EL44CRMZDDTXOW45GURIDIFQF4K65UUJFKR6L3F3WQBMVXSD2XHQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.729867582 (13h41m7.473924444s) [noise=true , address=104.36.82.60:4345 , piece_id=2AKG3BCGW7TFNVG3FQ576YJOTEZV3V2NP5ITSSZ7BO3NO5CRVDVQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.731063643 (13h41m7.475120512s) [noise=true , address=104.36.82.9:4016 , piece_id=VTTRIIBZ4KCGJW2H5DMG7J63AGFIV7QT34MJJTHRUGFIH6JD6EQQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.732350031 (13h41m7.476406894s) [noise=true , address=104.36.82.52:4106 , piece_id=KTACAAM5SP3PCN3VE6CKF3KHMLGYCHONLC7FDUI4YHLFKIEJ65HQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.733989697 (13h41m7.478046556s) [noise=true , address=64.49.13.155:4916 , piece_id=PJ3VBT33DUHLG3WHE6AK7IIYX76IZZFJCAWX7WX3UUBA6BLGB5VA, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.734727605 (13h41m7.478784474s) [noise=true , address=104.36.82.37:4680 , piece_id=2MDE2TCPX2ROXJXK7Q234YXMK4BFHPDQB5EB7GECHLMUJY5W4ITQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.734786637 (13h41m7.478843504s) [noise=true , address=104.36.82.4:4813 , piece_id=IZO235F25YVNCQG2GSR3UPI5OPMONHVRQQAWDEBOZWR2EILWGCCA, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.735460668 (13h41m7.479517532s) [noise=true , address=104.36.82.6:4901 , piece_id=7VX47BXB37LW2CKYFTBTFAJSDF3QLHEHQHRD5ZM5EHZPAZFPXBPQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.735486537 (13h41m7.479543401s) [noise=true , address=104.36.82.16:4323 , piece_id=RQYKNED6IWTV47W7HGZCVVOUKFRMHGO5WSPCW36ROCXMEDEFYM4Q, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.736565943 (13h41m7.480622804s) [noise=true , address=104.36.82.36:4651 , piece_id=5HQECPYGS5S43JV2CEJY3MRS33NUE6CJ5WPXKTCWYLYVXUQDXLXQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.737386328 (13h41m7.48144319s) [noise=true , address=104.36.82.44:4876 , piece_id=3WUHXX7GVGZWOGDA7FETGVT772RLHO4HRV6OCZKNTZIMG3VDPDOA, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:46.299198843 (13h41m8.043255689s) [upload=14082]: Next returned nil segment Aug 7 23:20:46.299217870 (13h41m8.043274733s) [upload=14082]: Waiting for error group managing segments... Aug 7 23:20:46.464881860 (13h41m8.208938721s) [noise=true , address=104.36.82.60:4345 , piece_id=2AKG3BCGW7TFNVG3FQ576YJOTEZV3V2NP5ITSSZ7BO3NO5CRVDVQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.473775294 (13h41m8.217832164s) [noise=true , address=104.36.82.52:4106 , piece_id=KTACAAM5SP3PCN3VE6CKF3KHMLGYCHONLC7FDUI4YHLFKIEJ65HQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.474664669 (13h41m8.218721541s) [noise=true , address=104.36.82.9:4016 , piece_id=VTTRIIBZ4KCGJW2H5DMG7J63AGFIV7QT34MJJTHRUGFIH6JD6EQQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.479688336 (13h41m8.223745197s) [noise=true , address=104.36.82.36:4651 , piece_id=5HQECPYGS5S43JV2CEJY3MRS33NUE6CJ5WPXKTCWYLYVXUQDXLXQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.485171804 (13h41m8.229228658s) [noise=true , address=104.36.82.6:4901 , piece_id=7VX47BXB37LW2CKYFTBTFAJSDF3QLHEHQHRD5ZM5EHZPAZFPXBPQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.486836180 (13h41m8.230893032s) [noise=true , address=104.36.82.61:4392 , piece_id=EL44CRMZDDTXOW45GURIDIFQF4K65UUJFKR6L3F3WQBMVXSD2XHQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.496476947 (13h41m8.240533796s) [noise=true , address=104.36.82.44:4876 , piece_id=3WUHXX7GVGZWOGDA7FETGVT772RLHO4HRV6OCZKNTZIMG3VDPDOA, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.556494206 (13h41m8.300551081s) [noise=true , address=104.36.82.16:4323 , piece_id=RQYKNED6IWTV47W7HGZCVVOUKFRMHGO5WSPCW36ROCXMEDEFYM4Q, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.628743021 (13h41m8.372799882s) [noise=true , address=45.91.116.14:4201 , piece_id=LCOEAQDFX432OULFXCKDH3FQMZMPR5XSDVKK5MAPEQYEABUAYMDA, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.645987246 (13h41m8.390044106s) [noise=true , address=104.36.82.4:4813 , piece_id=IZO235F25YVNCQG2GSR3UPI5OPMONHVRQQAWDEBOZWR2EILWGCCA, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.666511132 (13h41m8.410567992s) [noise=true , address=104.36.82.5:4867 , piece_id=IEDY33GIP4MZRSN2MKKBLDFAXPMKJUPZVPMOXVCEFJ6BUPZJHYHQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.671756371 (13h41m8.41581322s) [noise=true , address=104.36.82.37:4680 , piece_id=2MDE2TCPX2ROXJXK7Q234YXMK4BFHPDQB5EB7GECHLMUJY5W4ITQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.934025878 (13h41m8.678082729s) [noise=true , address=79.127.205.234:4597 , piece_id=WETMAYMNBWD6F6JYCXPNAG5KOJXNCXE3KM7ZJ4F5Y3GEBPHLMDEA, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.970907150 (13h41m8.714964016s) [noise=true , address=64.49.13.139:4474 , piece_id=LGXAPMGRQOOEXXG7EBNLO7GAPDKFBCQIBWGYID7KOVQ3UNMDZICQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.970940415 (13h41m8.714997268s) [seg_pos={0 0}, upload=14082]: Segment reached optimal threshold of 14 pieces. Aug 7 23:20:46.971273207 (13h41m8.715330074s) [noise=true , address=64.49.13.155:4916 , piece_id=PJ3VBT33DUHLG3WHE6AK7IIYX76IZZFJCAWX7WX3UUBA6BLGB5VA, seg_pos={0 0}, upload=14082]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12MD2UDNXEmeGra9fvhEb8eGL2R6vGLHtSKLpSyX3H3tZCe1r5B): protocol: expected piece hash; context canceled; EOF Aug 7 23:20:46.971460895 (13h41m8.715517755s) [noise=true , address=64.49.13.169:4705 , piece_id=YJCLDFSO5ENFREIROYPEZQQTQD4GM4A3DFKNJEHSPFDOKDCFEJ7Q, seg_pos={0 0}, upload=14082]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124uqzVwsF5TM4CwUhkzfJvgxd71jZXgXuFNwDLRW1vrSXbwLsr): protocol: expected piece hash; context canceled; EOF Aug 7 23:20:46.971541847 (13h41m8.715598706s) [noise=true , address=79.127.205.246:4276 , piece_id=6753PO7TQBULISFOSTSIT32I7DSKQPM6M4TEVF2E7TILSMUU5ARQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJTNVtPFN48S9ZrayehLD2HcVckyJWLx2rtEvyH56W22R8Suq): protocol: expected piece hash; context canceled; EOF Aug 7 23:20:46.971599101 (13h41m8.715655962s) [noise=true , address=104.36.82.58:4292 , piece_id=KQ2RNSZDMCU7OED4IJC4ERBTMTYVGHJCRSVOFZUXG24LW7H3KHUQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JeDjqgszkkq2asUBCL5oTvH6X21TnWkkmqGEEzBbz1YctdDXX): protocol: expected piece hash; context canceled; EOF Aug 7 23:20:46.971618718 (13h41m8.715675568s) [seg_pos={0 0}, upload=14082]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:20:46.971698846 (13h41m8.715755698s) [upload=14082]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:20:46.971704848 (13h41m8.71576171s) [upload=14082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:21:07.219673652 (13h41m28.963730505s) [upload=14082]: Done uploading segments... Aug 7 23:21:07.219701493 (13h41m28.963758344s) [upload=14082]: Upload finished. err: metaclient: context deadline exceeded Duration: 3.169806089s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 8 out of 60 is failed (threshold=3) [..............XXXXXX.XX.....................................] duration: 2.341024359s FAIL (2025-08-07 23:48:59 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.341099723s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [...........XXXXXXXXXX...........X...........................] duration: 2.929722078s FAIL (2025-08-07 23:48:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 0:4517 , piece_id=CIP2H4JR7JNEVTTPEDFI7FVDVPI66H6TNQTP7WDTWO3EFDNJWKXA, seg_pos={0 0}, upload=14216]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Aug 7 23:27:57.305395174 (13h48m19.049452024s) [seg_pos={0 0}, upload=14216]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:27:57.305491984 (13h48m19.049548844s) [upload=14216]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:27:57.305496984 (13h48m19.049553844s) [upload=14216]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:27:57.453740728 (13h48m19.197797589s) [upload=14216]: Done uploading segments... Aug 7 23:27:57.453760898 (13h48m19.197817752s) [upload=14216]: Upload finished. err: <nil> Aug 7 23:28:48.175431324 (13h49m9.919488199s) [upload=14229]: Starting upload Aug 7 23:28:48.175526742 (13h49m9.919583612s) [upload=14229]: Done starting upload Aug 7 23:28:48.175776798 (13h49m9.919833649s) [upload=14229]: Uploading segments... Aug 7 23:28:48.175982529 (13h49m9.920039392s) [upload=14229]: Got next segment. Inline: false Aug 7 23:28:48.176023543 (13h49m9.920080406s) [upload=14229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:28:48.176029551 (13h49m9.920086414s) [upload=14229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:28:48.284809382 (13h49m10.02886623s) [seg_pos={0 0}, upload=14229]: Begin upload segment... Aug 7 23:28:48.284855416 (13h49m10.028912275s) [seg_pos={0 0}, upload=14229]: Done begin upload segment. Aug 7 23:28:48.284878655 (13h49m10.028935518s) [noise=true , address=77.90.31.13:4626 , piece_id=QH4O3MHPFYQ6LH4Y6KGGELCSO5KRAHBPXGX4BWZLPTH2EBUBEBZA, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.285087631 (13h49m10.029144514s) [noise=true , address=77.90.31.17:4772 , piece_id=J6QWEIV73JC5VHU5BYRA7WBN5N7QOVTDIEOI4GQIL5MDDY3F32PQ, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.285200468 (13h49m10.029257328s) [noise=true , address=45.91.116.6:4975 , piece_id=VBM5UD6OVZCMPJ6QF664V2ZACFG6UQH6SOPHJFUE4WMYGZZYGCKQ, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.285437499 (13h49m10.029494358s) [noise=true , address=77.90.31.11:4553 , piece_id=WHCCT2GMH2DRWLK7TCINB4JR5C5SETU3XS2ARSIIWQN52E2EFVKQ, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.285628611 (13h49m10.029685478s) [noise=true , address=45.91.116.14:4190 , piece_id=XVD5ISVMY4722XYPEDTYPOHOGYL7XDNPBDNBMUQZNZJLLLYWIXFQ, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.285699438 (13h49m10.029756306s) [noise=true , address=104.36.82.14:4221 , piece_id=6AFA2RGZEI6YLGAGYE6ZSPPBJHMRGQIMRJH44AVZPG5JKCKK2F7A, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.286010251 (13h49m10.030067111s) [noise=true , address=77.90.31.10:4520 , piece_id=S3FSHHVOQS5UVL27Q7GAXEDB3H2QHQZSL6Z36QP7UABBABT3AKQQ, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.286078142 (13h49m10.030135004s) [noise=true , address=104.36.82.13:4185 , piece_id=EVQM6WWKOM5HCW64MOQHEGEZNCGBPB6KHD7H4G47NHEI6FQUAFZQ, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.286284224 (13h49m10.030341088s) [noise=true , address=77.90.31.12:4590 , piece_id=ZKPV2LZS65UFVUK5JWZAWAAVEQCELDU5FZQTU37PAISE6TWMEVKA, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.286553378 (13h49m10.03061024s) [noise=true , address=77.90.31.16:4733 , piece_id=2W4RBFY3CABLH7JFOCISDF4MP6H4EN6CN5F3GF7HANAJPE26TSHA, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.286568001 (13h49m10.030624861s) [noise=true , address=104.36.82.10:4077 , piece_id=4E23BJAD735PGV45SNNRBJQUFEWVUI4JLJDLZHNUXJXWGIQVQPEQ, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.286799362 (13h49m10.030856223s) [noise=true , address=77.90.31.12:4592 , piece_id=B3J4KUKWAUPNMX775I6UU4PD5XXGWECMLNTHPAVWCMFA4GZZQV2Q, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.287294442 (13h49m10.031351312s) [noise=true , address=45.91.116.7:4010 , piece_id=KFN54772PPRIEMWW35UTAFHODB2YVX3NFFBDH6HVIHXHVLBFUJLA, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.287736301 (13h49m10.031793162s) [noise=true , address=77.90.31.19:4841 , piece_id=ZPYXEINGBZIFNFXDANSECW7S7NLRQKVWOTRFUHXRG7YPIM6VUZYA, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.288128520 (13h49m10.032185379s) [noise=true , address=45.91.116.13:4155 , piece_id=L5FRG63YLCXXLHOO2NGEKCMXW3HQXNGUTPONX76ZEHQDIJG76KEQ, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.288446325 (13h49m10.032503213s) [noise=true , address=77.90.31.19:4843 , piece_id=725EIRIK675YXHGOPGBBGXGK4BMMUDOHQOBL36MCDSYUE5XKTI6Q, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.381456643 (13h49m10.125513496s) [noise=true , address=77.90.31.13:4626 , piece_id=QH4O3MHPFYQ6LH4Y6KGGELCSO5KRAHBPXGX4BWZLPTH2EBUBEBZA, seg_pos={0 0}, upload=14229]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 139.59.144.22:50332->77.90.31.13:4626: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:28:48.394653940 (13h49m10.138710812s) [noise=true , address=77.90.31.16:4733 , piece_id=2W4RBFY3CABLH7JFOCISDF4MP6H4EN6CN5F3GF7HANAJPE26TSHA, seg_pos={0 0}, upload=14229]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 139.59.144.22:43998->77.90.31.16:4733: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:28:48.897346092 (13h49m10.641402943s) [upload=14229]: Next returned nil segment Aug 7 23:28:48.897427052 (13h49m10.641483913s) [upload=14229]: Waiting for error group managing segments... Aug 7 23:28:49.066298229 (13h49m10.810355077s) [noise=true , address=77.90.31.12:4592 , piece_id=B3J4KUKWAUPNMX775I6UU4PD5XXGWECMLNTHPAVWCMFA4GZZQV2Q, seg_pos={0 0}, upload=14229]: Done uploading piece. err: <nil> Aug 7 23:28:49.082629995 (13h49m10.826686848s) [noise=true , address=104.36.82.13:4185 , piece_id=EVQM6WWKOM5HCW64MOQHEGEZNCGBPB6KHD7H4G47NHEI6FQUAFZQ, seg_pos={0 0}, upload=14229]: Done uploading piece. err: <nil> Aug 7 23:28:49.085963999 (13h49m10.830020845s) [noise=true , address=104.36.82.10:4077 , piece_id=4E23BJAD735PGV45SNNRBJQUFEWVUI4JLJDLZHNUXJXWGIQVQPEQ, seg_pos={0 0}, upload=14229]: Done uploading piece. err: <nil> Aug 7 23:28:49.091538387 (13h49m10.83559524s) [noise=true , address=77.90.31.12:4590 , piece_id=ZKPV2LZS65UFVUK5JWZAWAAVEQCELDU5FZQTU37PAISE6TWMEVKA, seg_pos={0 0}, upload=14229]: Done uploading piece. err: <nil> Aug 7 23:28:49.120549301 (13h49m10.864606147s) [noise=true , address=77.90.31.11:4553 , piece_id=WHCCT2GMH2DRWLK7TCINB4JR5C5SETU3XS2ARSIIWQN52E2EFVKQ, seg_pos={0 0}, upload=14229]: Done uploading piece. err: <nil> Aug 7 23:28:49.126986818 (13h49m10.871043668s) [noise=true , address=77.90.31.17:4772 , piece_id=J6QWEIV73JC5VHU5BYRA7WBN5N7QOVTDIEOI4GQIL5MDDY3F32PQ, seg_pos={0 0}, upload=14229]: Done uploading piece. err: <nil> Aug 7 23:28:49.133663068 (13h49m10.877719922s) [noise=true , address=45.91.116.7:4010 , piece_id=KFN54772PPRIEMWW35UTAFHODB2YVX3NFFBDH6HVIHXHVLBFUJLA, seg_pos={0 0}, upload=14229]: Done uploading piece. err: <nil> Aug 7 23:28:49.151613435 (13h49m10.895670288s) [noise=true , address=45.91.116.13:4155 , piece_id=L5FRG63YLCXXLHOO2NGEKCMXW3HQXNGUTPONX76ZEHQDIJG76KEQ, seg_pos={0 0}, upload=14229]: Done uploading piece. err: <nil> Aug 7 23:28:49.179832443 (13h49m10.923889293s) [noise=true , address=45.91.116.14:4190 , piece_id=XVD5ISVMY4722XYPEDTYPOHOGYL7XDNPBDNBMUQZNZJLLLYWIXFQ, seg_pos={0 0}, upload=14229]: Done uploading piece. err: <nil> Aug 7 23:28:49.235424834 (13h49m10.979481694s) [noise=true , address=77.90.31.10:4520 , piece_id=S3FSHHVOQS5UVL27Q7GAXEDB3H2QHQZSL6Z36QP7UABBABT3AKQQ, seg_pos={0 0}, upload=14229]: Done uploading piece. err: <nil> Aug 7 23:28:49.253356810 (13h49m10.997413672s) [noise=true , address=104.36.82.14:4221 , piece_id=6AFA2RGZEI6YLGAGYE6ZSPPBJHMRGQIMRJH44AVZPG5JKCKK2F7A, seg_pos={0 0}, upload=14229]: Done uploading piece. err: <nil> Aug 7 23:28:49.314647975 (13h49m11.058704826s) [noise=true , address=45.91.116.6:4975 , piece_id=VBM5UD6OVZCMPJ6QF664V2ZACFG6UQH6SOPHJFUE4WMYGZZYGCKQ, seg_pos={0 0}, upload=14229]: Done uploading piece. err: <nil> Aug 7 23:29:17.906721377 (13h49m39.650778264s) [noise=true , address=77.90.31.19:4841 , piece_id=ZPYXEINGBZIFNFXDANSECW7S7NLRQKVWOTRFUHXRG7YPIM6VUZYA, seg_pos={0 0}, upload=14229]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:29:17.906810237 (13h49m39.650867112s) [noise=true , address=77.90.31.19:4843 , piece_id=725EIRIK675YXHGOPGBBGXGK4BMMUDOHQOBL36MCDSYUE5XKTI6Q, seg_pos={0 0}, upload=14229]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 139.59.144.22:58688->77.90.31.19:4843: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:29:17.906834622 (13h49m39.650891505s) [seg_pos={0 0}, upload=14229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Aug 7 23:29:18.017467630 (13h49m39.761524483s) [upload=14229]: Done uploading segments... Aug 7 23:29:18.017499122 (13h49m39.761555972s) [upload=14229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.929754537s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 826.548884ms FAIL (2025-08-07 23:48:44 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== pload=14121]: Done uploading segments... Aug 7 23:22:28.505845197 (13h42m50.249902044s) [upload=14121]: Upload finished. err: <nil> Aug 7 23:23:10.377483605 (13h43m32.121540458s) [upload=14132]: Starting upload Aug 7 23:23:10.377535635 (13h43m32.121592495s) [upload=14132]: Done starting upload Aug 7 23:23:10.377603797 (13h43m32.121660647s) [upload=14132]: Uploading segments... Aug 7 23:23:10.377638935 (13h43m32.121695804s) [upload=14132]: Got next segment. Inline: true Aug 7 23:23:10.377643594 (13h43m32.121700454s) [upload=14132]: Waiting for error group managing segments... Aug 7 23:23:10.377688252 (13h43m32.121745104s) [upload=14132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:23:10.377693427 (13h43m32.121750286s) [upload=14132]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:23:10.377696805 (13h43m32.121753666s) [upload=14132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:23:10.562260724 (13h43m32.306317573s) [upload=14132]: Done uploading segments... Aug 7 23:23:10.562279228 (13h43m32.306336078s) [upload=14132]: Upload finished. err: <nil> Aug 7 23:24:07.896404858 (13h44m29.640461721s) [upload=14147]: Starting upload Aug 7 23:24:07.896490216 (13h44m29.640547079s) [upload=14147]: Done starting upload Aug 7 23:24:07.896580488 (13h44m29.640637339s) [upload=14147]: Uploading segments... Aug 7 23:24:07.896655006 (13h44m29.640711874s) [upload=14147]: Got next segment. Inline: true Aug 7 23:24:07.896660179 (13h44m29.640717039s) [upload=14147]: Waiting for error group managing segments... Aug 7 23:24:07.896714840 (13h44m29.6407717s) [upload=14147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:24:07.896719399 (13h44m29.64077626s) [upload=14147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:24:07.896722726 (13h44m29.640779586s) [upload=14147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:24:08.048898130 (13h44m29.792954991s) [upload=14147]: Done uploading segments... Aug 7 23:24:08.048952526 (13h44m29.793009386s) [upload=14147]: Upload finished. err: <nil> Aug 7 23:24:53.794754379 (13h45m15.53881125s) [upload=14162]: Starting upload Aug 7 23:24:53.794842317 (13h45m15.538899177s) [upload=14162]: Done starting upload Aug 7 23:24:53.794935964 (13h45m15.538992814s) [upload=14162]: Uploading segments... Aug 7 23:24:53.795003752 (13h45m15.539060623s) [upload=14162]: Got next segment. Inline: true Aug 7 23:24:53.795010814 (13h45m15.539067676s) [upload=14162]: Waiting for error group managing segments... Aug 7 23:24:53.795066607 (13h45m15.539123488s) [upload=14162]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:24:53.795158449 (13h45m15.5392153s) [upload=14162]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:24:53.795163166 (13h45m15.539220026s) [upload=14162]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:24:53.946994982 (13h45m15.691051833s) [upload=14162]: Done uploading segments... Aug 7 23:24:53.947013862 (13h45m15.691070711s) [upload=14162]: Upload finished. err: <nil> Aug 7 23:25:40.029856966 (13h46m1.773913836s) [upload=14177]: Starting upload Aug 7 23:25:40.029986938 (13h46m1.774043809s) [upload=14177]: Done starting upload Aug 7 23:25:40.030076796 (13h46m1.774133642s) [upload=14177]: Uploading segments... Aug 7 23:25:40.030132567 (13h46m1.774189438s) [upload=14177]: Got next segment. Inline: true Aug 7 23:25:40.030139885 (13h46m1.774196757s) [upload=14177]: Waiting for error group managing segments... Aug 7 23:25:40.030197588 (13h46m1.774254461s) [upload=14177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:25:40.030204146 (13h46m1.774261016s) [upload=14177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:25:40.030207984 (13h46m1.77426485s) [upload=14177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:25:40.184198742 (13h46m1.928255592s) [upload=14177]: Done uploading segments... Aug 7 23:25:40.184219017 (13h46m1.928275867s) [upload=14177]: Upload finished. err: <nil> Aug 7 23:26:54.625649110 (13h47m16.369705971s) [upload=14198]: Starting upload Aug 7 23:26:54.625738563 (13h47m16.369795424s) [upload=14198]: Done starting upload Aug 7 23:26:54.625842691 (13h47m16.369899539s) [upload=14198]: Uploading segments... Aug 7 23:26:54.626016839 (13h47m16.37007371s) [upload=14198]: Got next segment. Inline: true Aug 7 23:26:54.626052112 (13h47m16.370108973s) [upload=14198]: Waiting for error group managing segments... Aug 7 23:26:54.626129780 (13h47m16.370186645s) [upload=14198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:26:54.626137392 (13h47m16.370194264s) [upload=14198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:26:54.626143182 (13h47m16.370200048s) [upload=14198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:26:54.775804171 (13h47m16.519861021s) [upload=14198]: Done uploading segments... Aug 7 23:26:54.775824249 (13h47m16.519881102s) [upload=14198]: Upload finished. err: <nil> Aug 7 23:27:32.263746490 (13h47m54.007803361s) [upload=14208]: Starting upload Aug 7 23:27:32.263833655 (13h47m54.007890501s) [upload=14208]: Done starting upload Aug 7 23:27:32.263976536 (13h47m54.008033385s) [upload=14208]: Uploading segments... Aug 7 23:27:32.264070926 (13h47m54.008127796s) [upload=14208]: Got next segment. Inline: true Aug 7 23:27:32.264080356 (13h47m54.00813733s) [upload=14208]: Waiting for error group managing segments... Aug 7 23:27:32.264154657 (13h47m54.008211526s) [upload=14208]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:27:32.264163587 (13h47m54.008220454s) [upload=14208]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:27:32.264172017 (13h47m54.008228914s) [upload=14208]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:27:32.490188559 (13h47m54.23424541s) [upload=14208]: Done uploading segments... Aug 7 23:27:32.490207326 (13h47m54.234264172s) [upload=14208]: Upload finished. err: <nil> Aug 7 23:27:57.927940477 (13h48m19.671997323s) [upload=14217]: Starting upload Aug 7 23:27:57.928016423 (13h48m19.672073283s) [upload=14217]: Done starting upload Aug 7 23:27:57.928101290 (13h48m19.672158142s) [upload=14217]: Uploading segments... Aug 7 23:27:57.928155692 (13h48m19.672212566s) [upload=14217]: Got next segment. Inline: true Aug 7 23:27:57.928162009 (13h48m19.672218869s) [upload=14217]: Waiting for error group managing segments... Aug 7 23:27:57.928213294 (13h48m19.672270145s) [upload=14217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:27:57.928219212 (13h48m19.672276074s) [upload=14217]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:27:57.928223521 (13h48m19.672280383s) [upload=14217]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:27:58.087544005 (13h48m19.831600874s) [upload=14217]: Done uploading segments... Aug 7 23:27:58.087570226 (13h48m19.831627094s) [upload=14217]: Upload finished. err: <nil> Aug 7 23:29:12.765200935 (13h49m34.509257795s) [upload=14237]: Starting upload Aug 7 23:29:12.765253106 (13h49m34.509309977s) [upload=14237]: Done starting upload Aug 7 23:29:12.765324641 (13h49m34.509381499s) [upload=14237]: Uploading segments... Aug 7 23:29:12.765374187 (13h49m34.509431056s) [upload=14237]: Got next segment. Inline: true Aug 7 23:29:12.765378947 (13h49m34.509435807s) [upload=14237]: Waiting for error group managing segments... Aug 7 23:29:12.765430859 (13h49m34.50948772s) [upload=14237]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:29:12.765438106 (13h49m34.50949497s) [upload=14237]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:29:12.765443127 (13h49m34.509499988s) [upload=14237]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:29:12.926437674 (13h49m34.670494535s) [upload=14237]: Done uploading segments... Aug 7 23:29:12.926457418 (13h49m34.670514268s) [upload=14237]: Upload finished. err: <nil> Aug 7 23:29:42.225760316 (13h50m3.969817177s) [upload=14244]: Starting upload Aug 7 23:29:42.225855063 (13h50m3.969911929s) [upload=14244]: Done starting upload Aug 7 23:29:42.225962489 (13h50m3.970019355s) [upload=14244]: Uploading segments... Aug 7 23:29:42.226078484 (13h50m3.970135331s) [upload=14244]: Got next segment. Inline: true Aug 7 23:29:42.226083996 (13h50m3.970140842s) [upload=14244]: Waiting for error group managing segments... Aug 7 23:29:42.226143067 (13h50m3.970199914s) [upload=14244]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:29:42.226148511 (13h50m3.970205371s) [upload=14244]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:29:42.226153006 (13h50m3.970209865s) [upload=14244]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:29:42.401822947 (13h50m4.14587981s) [upload=14244]: Done uploading segments... Aug 7 23:29:42.401847556 (13h50m4.145904405s) [upload=14244]: Upload finished. err: <nil> Aug 7 23:30:31.529840301 (13h50m53.273897165s) [upload=14254]: Starting upload Aug 7 23:30:31.530002531 (13h50m53.274059378s) [upload=14254]: Done starting upload Aug 7 23:30:31.530201544 (13h50m53.274258391s) [upload=14254]: Uploading segments... Aug 7 23:30:31.530298120 (13h50m53.274354985s) [upload=14254]: Got next segment. Inline: true Aug 7 23:30:31.530303457 (13h50m53.274360317s) [upload=14254]: Waiting for error group managing segments... Aug 7 23:30:31.530353721 (13h50m53.274410567s) [upload=14254]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:30:31.530358927 (13h50m53.274415788s) [upload=14254]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:30:31.530363758 (13h50m53.274420619s) [upload=14254]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:30:31.696350967 (13h50m53.440407828s) [upload=14254]: Done uploading segments... Aug 7 23:30:31.696369460 (13h50m53.440426307s) [upload=14254]: Upload finished. err: <nil> Duration: 826.577852ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [....X..................................X....................] duration: 3.429992267s FAIL (2025-08-07 23:48:43 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== 294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:32:45.172127356 (13h53m6.916184241s) [upload=14294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:32:45.390149151 (13h53m7.134206011s) [seg_pos={0 0}, upload=14294]: Begin upload segment... Aug 7 23:32:45.390215910 (13h53m7.134272761s) [seg_pos={0 0}, upload=14294]: Done begin upload segment. Aug 7 23:32:45.390249193 (13h53m7.134306059s) [noise=true , address=104.36.82.19:4405 , piece_id=MHM3MWZJFTXSJT7PTOFSWHUYO3L6NIBUV6VU66XXZDH3MC3EFR2Q, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390272919 (13h53m7.134329788s) [noise=true , address=77.90.31.12:4593 , piece_id=C3RVQHFS4L7SKPFFIZ52MAQWGD7P4JNG3SNPCNY53SQUD72PZRUA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390658970 (13h53m7.13471583s) [noise=true , address=104.36.82.16:4321 , piece_id=YTF4RWV2SD6Z4UV4XRYPBRGPCTSUTC6TQKL3DHVKU65ALTNA4TEQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390852777 (13h53m7.134909659s) [noise=true , address=104.36.82.34:4566 , piece_id=3YEAUSXEL7WNZ3O7WM24Z2DKY2TDVKMRZSHMEPMNEL2LR7BY7DBQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.390957991 (13h53m7.135014851s) [noise=true , address=104.36.82.47:4523 , piece_id=MJIMHSO66V7OWFFK2UALPKSI7CGGCTHNPZINDXMVIHDLENIW5LFA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.391483291 (13h53m7.135540162s) [noise=true , address=104.36.82.5:4861 , piece_id=GRDGO5LNCETQWRU3M6LACGRW5MNA2KFLXUBT5R2UH6SEGWP6RVRQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.391931094 (13h53m7.135987964s) [noise=true , address=104.36.82.10:4090 , piece_id=V6NPPVIYEH6WLIW7OMT3NDFPR7UDWIT46GW6AFCAWUJTBLC3O55Q, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392250266 (13h53m7.136307136s) [noise=true , address=104.36.82.62:4439 , piece_id=ILRD4TEFW4PSLV4EGPORG3FM6U2HB3WSL5KP3PN566FAOGWODLYQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392611689 (13h53m7.136668559s) [noise=true , address=104.36.82.54:4160 , piece_id=GDBORRYX5QBQGH7ZMMUUAV7AAR5UHZOTK5FTQUPC7YKZC2KD7I3A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.392917410 (13h53m7.136974282s) [noise=true , address=104.36.82.23:4477 , piece_id=HAF6424ZCIBGSR4US5LN5ZWE5JMNMLFGUMQQ4YP2ARP7PQXBH5GQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393270536 (13h53m7.137327398s) [noise=true , address=104.36.82.44:4881 , piece_id=XXVSXG3ILFXG4IVMELHL45TRRHDFVVJMJ5LGOHU5EDHG6XXCDH6A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393582635 (13h53m7.137639498s) [noise=true , address=104.36.82.37:4665 , piece_id=RUKKGC4Y3FIVLZO2GRWJN3J6P36VWP5VSC2HCLEU3MZDDFXODJAQ, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.393856926 (13h53m7.137913788s) [noise=true , address=104.36.82.11:4128 , piece_id=XRPULHI7BYDF2JOIASSN5QKGEAC6FJS2F5C22WIY7EKYILRHZ73A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394196364 (13h53m7.138253227s) [noise=true , address=104.36.82.38:4706 , piece_id=BIHIPQX6LRWBNFN4SDC3K3V22AMGL3RC5B4RVE7QXJFKB4XDN3BA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394484397 (13h53m7.138541257s) [noise=true , address=104.36.82.61:4381 , piece_id=TTZOSRP6YPL6P4C6PY5S4K7Z6CIZJDPX7NUNBCK73CWZPCA3HQHA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.394593709 (13h53m7.138650578s) [noise=true , address=45.91.116.5:4300 , piece_id=MADJCANN2O35MKXT2PIR5PGMJ3PNTAQAJNNGOK6ISJNVAH2L7YIA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.395093013 (13h53m7.139149879s) [noise=true , address=104.36.82.22:4530 , piece_id=HMUUW5GYH6TRHYXP4YXBHD566FWQRILEHI53AVURA5QTQGG6USFA, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.395447945 (13h53m7.139504805s) [noise=true , address=104.36.82.58:4293 , piece_id=I2ETFXYTHYWYJYSHTVOBQTO6ZPGNMA4D3NVKYDE2W34EH4SQE23A, seg_pos={0 0}, upload=14294]: Uploading piece... Aug 7 23:32:45.954469844 (13h53m7.698526694s) [upload=14294]: Next returned nil segment Aug 7 23:32:45.954487953 (13h53m7.69854487s) [upload=14294]: Waiting for error group managing segments... Aug 7 23:32:46.126888627 (13h53m7.870945474s) [noise=true , address=104.36.82.10:4090 , piece_id=V6NPPVIYEH6WLIW7OMT3NDFPR7UDWIT46GW6AFCAWUJTBLC3O55Q, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.132983844 (13h53m7.877040704s) [noise=true , address=104.36.82.16:4321 , piece_id=YTF4RWV2SD6Z4UV4XRYPBRGPCTSUTC6TQKL3DHVKU65ALTNA4TEQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.138277216 (13h53m7.882334067s) [noise=true , address=104.36.82.23:4477 , piece_id=HAF6424ZCIBGSR4US5LN5ZWE5JMNMLFGUMQQ4YP2ARP7PQXBH5GQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.142033326 (13h53m7.886090185s) [noise=true , address=104.36.82.34:4566 , piece_id=3YEAUSXEL7WNZ3O7WM24Z2DKY2TDVKMRZSHMEPMNEL2LR7BY7DBQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.151247206 (13h53m7.895304058s) [noise=true , address=104.36.82.22:4530 , piece_id=HMUUW5GYH6TRHYXP4YXBHD566FWQRILEHI53AVURA5QTQGG6USFA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.151981081 (13h53m7.896037932s) [noise=true , address=104.36.82.58:4293 , piece_id=I2ETFXYTHYWYJYSHTVOBQTO6ZPGNMA4D3NVKYDE2W34EH4SQE23A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.165022882 (13h53m7.909079743s) [noise=true , address=104.36.82.44:4881 , piece_id=XXVSXG3ILFXG4IVMELHL45TRRHDFVVJMJ5LGOHU5EDHG6XXCDH6A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.184418821 (13h53m7.928475667s) [noise=true , address=104.36.82.47:4523 , piece_id=MJIMHSO66V7OWFFK2UALPKSI7CGGCTHNPZINDXMVIHDLENIW5LFA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.184532669 (13h53m7.928589516s) [noise=true , address=45.91.116.5:4300 , piece_id=MADJCANN2O35MKXT2PIR5PGMJ3PNTAQAJNNGOK6ISJNVAH2L7YIA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.193375808 (13h53m7.937432656s) [noise=true , address=104.36.82.54:4160 , piece_id=GDBORRYX5QBQGH7ZMMUUAV7AAR5UHZOTK5FTQUPC7YKZC2KD7I3A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.204247501 (13h53m7.948304372s) [noise=true , address=104.36.82.38:4706 , piece_id=BIHIPQX6LRWBNFN4SDC3K3V22AMGL3RC5B4RVE7QXJFKB4XDN3BA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.235225782 (13h53m7.979282629s) [noise=true , address=104.36.82.62:4439 , piece_id=ILRD4TEFW4PSLV4EGPORG3FM6U2HB3WSL5KP3PN566FAOGWODLYQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.269129867 (13h53m8.013186721s) [noise=true , address=104.36.82.5:4861 , piece_id=GRDGO5LNCETQWRU3M6LACGRW5MNA2KFLXUBT5R2UH6SEGWP6RVRQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.278216912 (13h53m8.022273764s) [noise=true , address=104.36.82.19:4405 , piece_id=MHM3MWZJFTXSJT7PTOFSWHUYO3L6NIBUV6VU66XXZDH3MC3EFR2Q, seg_pos={0 0}, upload=14294]: Done uploading piece. err: <nil> Aug 7 23:32:46.278247847 (13h53m8.022304708s) [seg_pos={0 0}, upload=14294]: Segment reached optimal threshold of 14 pieces. Aug 7 23:32:46.278485612 (13h53m8.022542463s) [noise=true , address=104.36.82.11:4128 , piece_id=XRPULHI7BYDF2JOIASSN5QKGEAC6FJS2F5C22WIY7EKYILRHZ73A, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wABkAtAjoYkKyNoAiaVCxSvY6WgvHvAYFPCjpNojtpiYiGtDS): protocol: expected piece hash; context canceled; EOF Aug 7 23:32:46.278511698 (13h53m8.022568566s) [noise=true , address=77.90.31.12:4593 , piece_id=C3RVQHFS4L7SKPFFIZ52MAQWGD7P4JNG3SNPCNY53SQUD72PZRUA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kSxoZb9nmG6oGNytpFTsoiGh3ZPmebVVe8TZxGzANaw7EfdeF): protocol: expected piece hash; context canceled; EOF Aug 7 23:32:46.278642807 (13h53m8.022699657s) [noise=true , address=104.36.82.61:4381 , piece_id=TTZOSRP6YPL6P4C6PY5S4K7Z6CIZJDPX7NUNBCK73CWZPCA3HQHA, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zU35MWWaaC1npkP5qGcWBXAkfxGsd5UsdkM2nwX9x7wUGCDbY): protocol: write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection; write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42722->104.36.82.61:4381: use of closed network connection Aug 7 23:32:46.278650926 (13h53m8.022707786s) [noise=true , address=104.36.82.37:4665 , piece_id=RUKKGC4Y3FIVLZO2GRWJN3J6P36VWP5VSC2HCLEU3MZDDFXODJAQ, seg_pos={0 0}, upload=14294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jP9JrCmoQSdqrzXZxkNN1ihdFkaUMjGZss6Fh19zbL3YyWj5N): protocol: write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection; write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59450->104.36.82.37:4665: use of closed network connection Aug 7 23:32:46.278674050 (13h53m8.022730911s) [seg_pos={0 0}, upload=14294]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:32:46.278751154 (13h53m8.022808014s) [upload=14294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:32:46.278756168 (13h53m8.022813029s) [upload=14294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:32:46.498660020 (13h53m8.242716882s) [upload=14294]: Done uploading segments... Aug 7 23:32:46.498701263 (13h53m8.242758128s) [upload=14294]: Upload finished. err: <nil> Duration: 3.430018307s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...........X.XX..X..........................................] duration: 2.594966559s FAIL (2025-08-07 23:48:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.594990589s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 12 out of 60 is failed (threshold=3) [...........XXX.XXXXXX..XX.X.................................] duration: 910.593534ms FAIL (2025-08-07 23:48:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 7 23:15:07.832719564 (13h35m29.576776416s) [upload=13995]: Done starting upload Aug 7 23:15:07.832826894 (13h35m29.576883745s) [upload=13995]: Uploading segments... Aug 7 23:15:07.832887307 (13h35m29.576944167s) [upload=13995]: Got next segment. Inline: true Aug 7 23:15:07.832893154 (13h35m29.576950015s) [upload=13995]: Waiting for error group managing segments... Aug 7 23:15:07.832938815 (13h35m29.576995666s) [upload=13995]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:15:07.832943939 (13h35m29.5770008s) [upload=13995]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:15:07.832947902 (13h35m29.577004762s) [upload=13995]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:37.560683179 (13h35m59.304740027s) [upload=13995]: Done uploading segments... Aug 7 23:15:37.560704530 (13h35m59.304761377s) [upload=13995]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:16:13.030124119 (13h36m34.774180979s) [upload=14013]: Starting upload Aug 7 23:16:13.030198850 (13h36m34.774255715s) [upload=14013]: Done starting upload Aug 7 23:16:13.030287975 (13h36m34.774344821s) [upload=14013]: Uploading segments... Aug 7 23:16:13.030344552 (13h36m34.774401422s) [upload=14013]: Got next segment. Inline: true Aug 7 23:16:13.030349665 (13h36m34.774406525s) [upload=14013]: Waiting for error group managing segments... Aug 7 23:16:13.030401514 (13h36m34.774458375s) [upload=14013]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:13.030406613 (13h36m34.774463474s) [upload=14013]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:13.030409548 (13h36m34.774466408s) [upload=14013]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:42.761381352 (13h37m4.505438199s) [upload=14013]: Done uploading segments... Aug 7 23:16:42.761401933 (13h37m4.505458793s) [upload=14013]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:17:23.340430421 (13h37m45.084487291s) [upload=14034]: Starting upload Aug 7 23:17:23.340585398 (13h37m45.084642265s) [upload=14034]: Done starting upload Aug 7 23:17:23.340743782 (13h37m45.084800644s) [upload=14034]: Uploading segments... Aug 7 23:17:23.340847668 (13h37m45.084904539s) [upload=14034]: Got next segment. Inline: true Aug 7 23:17:23.340857086 (13h37m45.084913958s) [upload=14034]: Waiting for error group managing segments... Aug 7 23:17:23.340963706 (13h37m45.085020572s) [upload=14034]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:17:23.340973939 (13h37m45.085031006s) [upload=14034]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:17:23.340979371 (13h37m45.08503624s) [upload=14034]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:53.063547301 (13h38m14.807604164s) [upload=14034]: Done uploading segments... Aug 7 23:17:53.063575766 (13h38m14.807632627s) [upload=14034]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:18:01.235662578 (13h38m22.979719439s) [upload=14043]: Starting upload Aug 7 23:18:01.235769067 (13h38m22.97982594s) [upload=14043]: Done starting upload Aug 7 23:18:01.235873103 (13h38m22.979929971s) [upload=14043]: Uploading segments... Aug 7 23:18:01.235955349 (13h38m22.98001222s) [upload=14043]: Got next segment. Inline: true Aug 7 23:18:01.235976122 (13h38m22.980032984s) [upload=14043]: Waiting for error group managing segments... Aug 7 23:18:01.236032940 (13h38m22.980089801s) [upload=14043]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:18:01.236038848 (13h38m22.980095712s) [upload=14043]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:18:01.236056240 (13h38m22.980113107s) [upload=14043]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:18:30.962606072 (13h38m52.706662932s) [upload=14043]: Done uploading segments... Aug 7 23:18:30.962671417 (13h38m52.706728279s) [upload=14043]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:18:52.160899256 (13h39m13.904956117s) [upload=14060]: Starting upload Aug 7 23:18:52.160970297 (13h39m13.905027157s) [upload=14060]: Done starting upload Aug 7 23:18:52.161065587 (13h39m13.905122439s) [upload=14060]: Uploading segments... Aug 7 23:18:52.161108065 (13h39m13.905164934s) [upload=14060]: Got next segment. Inline: true Aug 7 23:18:52.161113994 (13h39m13.905170844s) [upload=14060]: Waiting for error group managing segments... Aug 7 23:18:52.161171289 (13h39m13.90522814s) [upload=14060]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:18:52.161176753 (13h39m13.905233614s) [upload=14060]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:18:52.161180292 (13h39m13.905237155s) [upload=14060]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:18:52.299645097 (13h39m14.043701961s) [upload=14060]: Done uploading segments... Aug 7 23:18:52.299672156 (13h39m14.043729007s) [upload=14060]: Upload finished. err: <nil> Aug 7 23:19:29.899969190 (13h39m51.64402605s) [upload=14071]: Starting upload Aug 7 23:19:29.900023963 (13h39m51.644080812s) [upload=14071]: Done starting upload Aug 7 23:19:29.900097059 (13h39m51.644153908s) [upload=14071]: Uploading segments... Aug 7 23:19:29.900152738 (13h39m51.644209608s) [upload=14071]: Got next segment. Inline: true Aug 7 23:19:29.900161132 (13h39m51.644218002s) [upload=14071]: Waiting for error group managing segments... Aug 7 23:19:29.900212688 (13h39m51.64426955s) [upload=14071]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:19:29.900219311 (13h39m51.644276178s) [upload=14071]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:19:29.900224731 (13h39m51.64428173s) [upload=14071]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:19:30.208671256 (13h39m51.952728106s) [upload=14071]: Done uploading segments... Aug 7 23:19:30.208715246 (13h39m51.952772096s) [upload=14071]: Upload finished. err: <nil> Aug 7 23:20:17.640185376 (13h40m39.384242236s) [upload=14084]: Starting upload Aug 7 23:20:17.640240796 (13h40m39.384297642s) [upload=14084]: Done starting upload Aug 7 23:20:17.640310076 (13h40m39.384366923s) [upload=14084]: Uploading segments... Aug 7 23:20:17.640362556 (13h40m39.384419425s) [upload=14084]: Got next segment. Inline: true Aug 7 23:20:17.640367233 (13h40m39.384424093s) [upload=14084]: Waiting for error group managing segments... Aug 7 23:20:17.640417842 (13h40m39.384474688s) [upload=14084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:20:17.640422768 (13h40m39.384479628s) [upload=14084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:20:17.640425589 (13h40m39.384482436s) [upload=14084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:20:21.993094963 (13h40m43.737151826s) [upload=14084]: Done uploading segments... Aug 7 23:20:21.993114705 (13h40m43.737171555s) [upload=14084]: Upload finished. err: <nil> Aug 7 23:21:07.375939272 (13h41m29.119996122s) [upload=14096]: Starting upload Aug 7 23:21:07.376022371 (13h41m29.120079233s) [upload=14096]: Done starting upload Aug 7 23:21:07.376131644 (13h41m29.120188494s) [upload=14096]: Uploading segments... Aug 7 23:21:07.376228707 (13h41m29.120285578s) [upload=14096]: Got next segment. Inline: true Aug 7 23:21:07.376236415 (13h41m29.120293285s) [upload=14096]: Waiting for error group managing segments... Aug 7 23:21:07.376314696 (13h41m29.120371567s) [upload=14096]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:21:07.376322308 (13h41m29.120379199s) [upload=14096]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:21:07.376327371 (13h41m29.120384242s) [upload=14096]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:21:37.104686329 (13h41m58.84874319s) [upload=14096]: Done uploading segments... Aug 7 23:21:37.104719076 (13h41m58.848775939s) [upload=14096]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:22:04.330992472 (13h42m26.075049325s) [upload=14111]: Starting upload Aug 7 23:22:04.331290477 (13h42m26.075347342s) [upload=14111]: Done starting upload Aug 7 23:22:04.331454134 (13h42m26.075510996s) [upload=14111]: Uploading segments... Aug 7 23:22:04.331559270 (13h42m26.075616144s) [upload=14111]: Got next segment. Inline: true Aug 7 23:22:04.331568404 (13h42m26.075625271s) [upload=14111]: Waiting for error group managing segments... Aug 7 23:22:04.331647006 (13h42m26.075703873s) [upload=14111]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:22:04.331655756 (13h42m26.075712624s) [upload=14111]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:22:04.331661330 (13h42m26.075718197s) [upload=14111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:22:04.632273425 (13h42m26.376330276s) [upload=14111]: Done uploading segments... Aug 7 23:22:04.632293274 (13h42m26.376350128s) [upload=14111]: Upload finished. err: <nil> Aug 7 23:22:37.291949339 (13h42m59.03600619s) [upload=14122]: Starting upload Aug 7 23:22:37.292013250 (13h42m59.036070113s) [upload=14122]: Done starting upload Aug 7 23:22:37.292085244 (13h42m59.036142097s) [upload=14122]: Uploading segments... Aug 7 23:22:37.292207393 (13h42m59.036264263s) [upload=14122]: Got next segment. Inline: true Aug 7 23:22:37.292216268 (13h42m59.036273132s) [upload=14122]: Waiting for error group managing segments... Aug 7 23:22:37.292274925 (13h42m59.036331788s) [upload=14122]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:22:37.292281983 (13h42m59.036338843s) [upload=14122]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:22:37.292288349 (13h42m59.036345217s) [upload=14122]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:23:07.017248949 (13h43m28.761305819s) [upload=14122]: Done uploading segments... Aug 7 23:23:07.017274459 (13h43m28.761331331s) [upload=14122]: Upload finished. err: metaclient: context deadline exceeded Duration: 910.619411ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 7 out of 60 is failed (threshold=3) [........XX..XXXXX...........................................] duration: 867.189926ms FAIL (2025-08-07 23:48:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== :10:03.085332650 (13h30m24.829389513s) [upload=13904]: Done starting upload Aug 7 23:10:03.085414170 (13h30m24.829471025s) [upload=13904]: Uploading segments... Aug 7 23:10:03.085475435 (13h30m24.829532308s) [upload=13904]: Got next segment. Inline: true Aug 7 23:10:03.085480843 (13h30m24.829537706s) [upload=13904]: Waiting for error group managing segments... Aug 7 23:10:03.085544347 (13h30m24.829601211s) [upload=13904]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:03.085552551 (13h30m24.829609424s) [upload=13904]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:03.085555974 (13h30m24.829612844s) [upload=13904]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:10:03.247308964 (13h30m24.991365825s) [upload=13904]: Done uploading segments... Aug 7 23:10:03.247396260 (13h30m24.991453126s) [upload=13904]: Upload finished. err: <nil> Aug 7 23:10:40.522399484 (13h31m2.266456348s) [upload=13915]: Starting upload Aug 7 23:10:40.522487646 (13h31m2.266544496s) [upload=13915]: Done starting upload Aug 7 23:10:40.522571041 (13h31m2.266627902s) [upload=13915]: Uploading segments... Aug 7 23:10:40.522619425 (13h31m2.266676294s) [upload=13915]: Got next segment. Inline: true Aug 7 23:10:40.522626307 (13h31m2.266683183s) [upload=13915]: Waiting for error group managing segments... Aug 7 23:10:40.522684106 (13h31m2.266740971s) [upload=13915]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:40.522691484 (13h31m2.26674835s) [upload=13915]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:40.522696090 (13h31m2.266752961s) [upload=13915]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:40.254283342 (13h32m1.998340192s) [upload=13915]: Done uploading segments... Aug 7 23:11:40.254304548 (13h32m1.998361398s) [upload=13915]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:11:40.667902567 (13h32m2.411959413s) [upload=13933]: Starting upload Aug 7 23:11:40.667961940 (13h32m2.412018788s) [upload=13933]: Done starting upload Aug 7 23:11:40.668020524 (13h32m2.41207737s) [upload=13933]: Uploading segments... Aug 7 23:11:40.668075247 (13h32m2.412132113s) [upload=13933]: Got next segment. Inline: true Aug 7 23:11:40.668080664 (13h32m2.412137525s) [upload=13933]: Waiting for error group managing segments... Aug 7 23:11:40.668121190 (13h32m2.412178036s) [upload=13933]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:40.668125756 (13h32m2.412182616s) [upload=13933]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:40.668128431 (13h32m2.412185291s) [upload=13933]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:40.394243254 (13h33m2.138300118s) [upload=13933]: Done uploading segments... Aug 7 23:12:40.394270270 (13h33m2.138327132s) [upload=13933]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:12:40.798630406 (13h33m2.542687267s) [upload=13953]: Starting upload Aug 7 23:12:40.798725809 (13h33m2.54278266s) [upload=13953]: Done starting upload Aug 7 23:12:40.798811946 (13h33m2.542868796s) [upload=13953]: Uploading segments... Aug 7 23:12:40.798867069 (13h33m2.542923932s) [upload=13953]: Got next segment. Inline: true Aug 7 23:12:40.798873108 (13h33m2.542929959s) [upload=13953]: Waiting for error group managing segments... Aug 7 23:12:40.798918573 (13h33m2.542975422s) [upload=13953]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:40.798924603 (13h33m2.542981463s) [upload=13953]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:40.798939855 (13h33m2.542996716s) [upload=13953]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:40.942390421 (13h33m2.686447272s) [upload=13953]: Done uploading segments... Aug 7 23:12:40.942410212 (13h33m2.686467062s) [upload=13953]: Upload finished. err: <nil> Aug 7 23:13:18.137108974 (13h33m39.881165825s) [upload=13967]: Starting upload Aug 7 23:13:18.137180408 (13h33m39.881237292s) [upload=13967]: Done starting upload Aug 7 23:13:18.137258925 (13h33m39.881315786s) [upload=13967]: Uploading segments... Aug 7 23:13:18.137309635 (13h33m39.881366505s) [upload=13967]: Got next segment. Inline: true Aug 7 23:13:18.137314511 (13h33m39.881371371s) [upload=13967]: Waiting for error group managing segments... Aug 7 23:13:18.137374772 (13h33m39.881431633s) [upload=13967]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:18.137380412 (13h33m39.881437273s) [upload=13967]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:18.137383733 (13h33m39.881440593s) [upload=13967]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:18.301480519 (13h33m40.045537379s) [upload=13967]: Done uploading segments... Aug 7 23:13:18.301498918 (13h33m40.045555767s) [upload=13967]: Upload finished. err: <nil> Aug 7 23:14:13.169029066 (13h34m34.913085913s) [upload=13980]: Starting upload Aug 7 23:14:13.169095254 (13h34m34.913152101s) [upload=13980]: Done starting upload Aug 7 23:14:13.169153338 (13h34m34.913210184s) [upload=13980]: Uploading segments... Aug 7 23:14:13.169225662 (13h34m34.913282508s) [upload=13980]: Got next segment. Inline: true Aug 7 23:14:13.169230885 (13h34m34.913287744s) [upload=13980]: Waiting for error group managing segments... Aug 7 23:14:13.169272521 (13h34m34.913329381s) [upload=13980]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:13.169288952 (13h34m34.913345813s) [upload=13980]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:13.169291623 (13h34m34.913348483s) [upload=13980]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:12.895960606 (13h35m34.640017467s) [upload=13980]: Done uploading segments... Aug 7 23:15:12.895983608 (13h35m34.640040456s) [upload=13980]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:15:13.308146067 (13h35m35.052202918s) [upload=13997]: Starting upload Aug 7 23:15:13.308219271 (13h35m35.052276134s) [upload=13997]: Done starting upload Aug 7 23:15:13.308297319 (13h35m35.052354171s) [upload=13997]: Uploading segments... Aug 7 23:15:13.308360203 (13h35m35.052417072s) [upload=13997]: Got next segment. Inline: true Aug 7 23:15:13.308365548 (13h35m35.052422408s) [upload=13997]: Waiting for error group managing segments... Aug 7 23:15:13.308409794 (13h35m35.052466657s) [upload=13997]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:15:13.308426102 (13h35m35.052482963s) [upload=13997]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:15:13.308429725 (13h35m35.052486585s) [upload=13997]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:13.040589970 (13h36m34.784646843s) [upload=13997]: Done uploading segments... Aug 7 23:16:13.040614428 (13h36m34.784671274s) [upload=13997]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:16:20.767866726 (13h36m42.511923577s) [upload=14018]: Starting upload Aug 7 23:16:20.767927544 (13h36m42.511984398s) [upload=14018]: Done starting upload Aug 7 23:16:20.767993090 (13h36m42.512049942s) [upload=14018]: Uploading segments... Aug 7 23:16:20.768055054 (13h36m42.512111926s) [upload=14018]: Got next segment. Inline: true Aug 7 23:16:20.768061500 (13h36m42.512118351s) [upload=14018]: Waiting for error group managing segments... Aug 7 23:16:20.768130604 (13h36m42.512187577s) [upload=14018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:20.768139327 (13h36m42.512196192s) [upload=14018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:20.768144987 (13h36m42.512201849s) [upload=14018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:20.496845379 (13h37m42.240902231s) [upload=14018]: Done uploading segments... Aug 7 23:17:20.496873778 (13h37m42.240930632s) [upload=14018]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:17:20.916777280 (13h37m42.660834146s) [upload=14032]: Starting upload Aug 7 23:17:20.916877286 (13h37m42.660934149s) [upload=14032]: Done starting upload Aug 7 23:17:20.917002557 (13h37m42.661059408s) [upload=14032]: Uploading segments... Aug 7 23:17:20.917058206 (13h37m42.661115077s) [upload=14032]: Got next segment. Inline: true Aug 7 23:17:20.917064317 (13h37m42.661121178s) [upload=14032]: Waiting for error group managing segments... Aug 7 23:17:20.917116347 (13h37m42.661173214s) [upload=14032]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:17:20.917123150 (13h37m42.661180014s) [upload=14032]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:17:20.917127779 (13h37m42.661184648s) [upload=14032]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:18:20.640884349 (13h38m42.384941214s) [upload=14032]: Done uploading segments... Aug 7 23:18:20.640914749 (13h38m42.384971617s) [upload=14032]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:18:21.050365938 (13h38m42.79442281s) [upload=14051]: Starting upload Aug 7 23:18:21.050484720 (13h38m42.794541587s) [upload=14051]: Done starting upload Aug 7 23:18:21.050589902 (13h38m42.794646752s) [upload=14051]: Uploading segments... Aug 7 23:18:21.050663797 (13h38m42.794720667s) [upload=14051]: Got next segment. Inline: true Aug 7 23:18:21.050670451 (13h38m42.794727311s) [upload=14051]: Waiting for error group managing segments... Aug 7 23:18:21.050719413 (13h38m42.794776275s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:18:21.050725163 (13h38m42.794782023s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:18:21.050728952 (13h38m42.794785811s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:19:20.780076504 (13h39m42.524133366s) [upload=14051]: Done uploading segments... Aug 7 23:19:20.780099469 (13h39m42.52415633s) [upload=14051]: Upload finished. err: metaclient: context deadline exceeded Duration: 867.230904ms core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 337.190307ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 23:48:40 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 337.217273ms edge gateway vhost us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 6 out of 60 is failed (threshold=3) [................XXX...XX.X..................................] duration: 1.095607654s FAIL (2025-08-07 23:48:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.09563045s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [............X.XX..X.........................................] duration: 2.749163474s FAIL (2025-08-07 23:48:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.749203686s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 7 out of 60 is failed (threshold=3) [............X.XXXXX.X.......................................] duration: 2.849477308s FAIL (2025-08-07 23:48:19 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.849515353s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 10 out of 60 is failed (threshold=3) [...........XXXXX.XX.XX.X....................................] duration: 1.027058771s FAIL (2025-08-07 23:48:15 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.027087622s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 7 out of 60 is failed (threshold=3) [.........XX..XXXXX..........................................] duration: 844.600405ms FAIL (2025-08-07 23:48:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== :10:03.085332650 (13h30m24.829389513s) [upload=13904]: Done starting upload Aug 7 23:10:03.085414170 (13h30m24.829471025s) [upload=13904]: Uploading segments... Aug 7 23:10:03.085475435 (13h30m24.829532308s) [upload=13904]: Got next segment. Inline: true Aug 7 23:10:03.085480843 (13h30m24.829537706s) [upload=13904]: Waiting for error group managing segments... Aug 7 23:10:03.085544347 (13h30m24.829601211s) [upload=13904]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:03.085552551 (13h30m24.829609424s) [upload=13904]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:03.085555974 (13h30m24.829612844s) [upload=13904]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:10:03.247308964 (13h30m24.991365825s) [upload=13904]: Done uploading segments... Aug 7 23:10:03.247396260 (13h30m24.991453126s) [upload=13904]: Upload finished. err: <nil> Aug 7 23:10:40.522399484 (13h31m2.266456348s) [upload=13915]: Starting upload Aug 7 23:10:40.522487646 (13h31m2.266544496s) [upload=13915]: Done starting upload Aug 7 23:10:40.522571041 (13h31m2.266627902s) [upload=13915]: Uploading segments... Aug 7 23:10:40.522619425 (13h31m2.266676294s) [upload=13915]: Got next segment. Inline: true Aug 7 23:10:40.522626307 (13h31m2.266683183s) [upload=13915]: Waiting for error group managing segments... Aug 7 23:10:40.522684106 (13h31m2.266740971s) [upload=13915]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:40.522691484 (13h31m2.26674835s) [upload=13915]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:40.522696090 (13h31m2.266752961s) [upload=13915]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:40.254283342 (13h32m1.998340192s) [upload=13915]: Done uploading segments... Aug 7 23:11:40.254304548 (13h32m1.998361398s) [upload=13915]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:11:40.667902567 (13h32m2.411959413s) [upload=13933]: Starting upload Aug 7 23:11:40.667961940 (13h32m2.412018788s) [upload=13933]: Done starting upload Aug 7 23:11:40.668020524 (13h32m2.41207737s) [upload=13933]: Uploading segments... Aug 7 23:11:40.668075247 (13h32m2.412132113s) [upload=13933]: Got next segment. Inline: true Aug 7 23:11:40.668080664 (13h32m2.412137525s) [upload=13933]: Waiting for error group managing segments... Aug 7 23:11:40.668121190 (13h32m2.412178036s) [upload=13933]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:40.668125756 (13h32m2.412182616s) [upload=13933]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:40.668128431 (13h32m2.412185291s) [upload=13933]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:40.394243254 (13h33m2.138300118s) [upload=13933]: Done uploading segments... Aug 7 23:12:40.394270270 (13h33m2.138327132s) [upload=13933]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:12:40.798630406 (13h33m2.542687267s) [upload=13953]: Starting upload Aug 7 23:12:40.798725809 (13h33m2.54278266s) [upload=13953]: Done starting upload Aug 7 23:12:40.798811946 (13h33m2.542868796s) [upload=13953]: Uploading segments... Aug 7 23:12:40.798867069 (13h33m2.542923932s) [upload=13953]: Got next segment. Inline: true Aug 7 23:12:40.798873108 (13h33m2.542929959s) [upload=13953]: Waiting for error group managing segments... Aug 7 23:12:40.798918573 (13h33m2.542975422s) [upload=13953]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:40.798924603 (13h33m2.542981463s) [upload=13953]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:40.798939855 (13h33m2.542996716s) [upload=13953]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:40.942390421 (13h33m2.686447272s) [upload=13953]: Done uploading segments... Aug 7 23:12:40.942410212 (13h33m2.686467062s) [upload=13953]: Upload finished. err: <nil> Aug 7 23:13:18.137108974 (13h33m39.881165825s) [upload=13967]: Starting upload Aug 7 23:13:18.137180408 (13h33m39.881237292s) [upload=13967]: Done starting upload Aug 7 23:13:18.137258925 (13h33m39.881315786s) [upload=13967]: Uploading segments... Aug 7 23:13:18.137309635 (13h33m39.881366505s) [upload=13967]: Got next segment. Inline: true Aug 7 23:13:18.137314511 (13h33m39.881371371s) [upload=13967]: Waiting for error group managing segments... Aug 7 23:13:18.137374772 (13h33m39.881431633s) [upload=13967]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:18.137380412 (13h33m39.881437273s) [upload=13967]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:18.137383733 (13h33m39.881440593s) [upload=13967]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:18.301480519 (13h33m40.045537379s) [upload=13967]: Done uploading segments... Aug 7 23:13:18.301498918 (13h33m40.045555767s) [upload=13967]: Upload finished. err: <nil> Aug 7 23:14:13.169029066 (13h34m34.913085913s) [upload=13980]: Starting upload Aug 7 23:14:13.169095254 (13h34m34.913152101s) [upload=13980]: Done starting upload Aug 7 23:14:13.169153338 (13h34m34.913210184s) [upload=13980]: Uploading segments... Aug 7 23:14:13.169225662 (13h34m34.913282508s) [upload=13980]: Got next segment. Inline: true Aug 7 23:14:13.169230885 (13h34m34.913287744s) [upload=13980]: Waiting for error group managing segments... Aug 7 23:14:13.169272521 (13h34m34.913329381s) [upload=13980]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:13.169288952 (13h34m34.913345813s) [upload=13980]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:13.169291623 (13h34m34.913348483s) [upload=13980]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:12.895960606 (13h35m34.640017467s) [upload=13980]: Done uploading segments... Aug 7 23:15:12.895983608 (13h35m34.640040456s) [upload=13980]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:15:13.308146067 (13h35m35.052202918s) [upload=13997]: Starting upload Aug 7 23:15:13.308219271 (13h35m35.052276134s) [upload=13997]: Done starting upload Aug 7 23:15:13.308297319 (13h35m35.052354171s) [upload=13997]: Uploading segments... Aug 7 23:15:13.308360203 (13h35m35.052417072s) [upload=13997]: Got next segment. Inline: true Aug 7 23:15:13.308365548 (13h35m35.052422408s) [upload=13997]: Waiting for error group managing segments... Aug 7 23:15:13.308409794 (13h35m35.052466657s) [upload=13997]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:15:13.308426102 (13h35m35.052482963s) [upload=13997]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:15:13.308429725 (13h35m35.052486585s) [upload=13997]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:13.040589970 (13h36m34.784646843s) [upload=13997]: Done uploading segments... Aug 7 23:16:13.040614428 (13h36m34.784671274s) [upload=13997]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:16:20.767866726 (13h36m42.511923577s) [upload=14018]: Starting upload Aug 7 23:16:20.767927544 (13h36m42.511984398s) [upload=14018]: Done starting upload Aug 7 23:16:20.767993090 (13h36m42.512049942s) [upload=14018]: Uploading segments... Aug 7 23:16:20.768055054 (13h36m42.512111926s) [upload=14018]: Got next segment. Inline: true Aug 7 23:16:20.768061500 (13h36m42.512118351s) [upload=14018]: Waiting for error group managing segments... Aug 7 23:16:20.768130604 (13h36m42.512187577s) [upload=14018]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:20.768139327 (13h36m42.512196192s) [upload=14018]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:20.768144987 (13h36m42.512201849s) [upload=14018]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:20.496845379 (13h37m42.240902231s) [upload=14018]: Done uploading segments... Aug 7 23:17:20.496873778 (13h37m42.240930632s) [upload=14018]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:17:20.916777280 (13h37m42.660834146s) [upload=14032]: Starting upload Aug 7 23:17:20.916877286 (13h37m42.660934149s) [upload=14032]: Done starting upload Aug 7 23:17:20.917002557 (13h37m42.661059408s) [upload=14032]: Uploading segments... Aug 7 23:17:20.917058206 (13h37m42.661115077s) [upload=14032]: Got next segment. Inline: true Aug 7 23:17:20.917064317 (13h37m42.661121178s) [upload=14032]: Waiting for error group managing segments... Aug 7 23:17:20.917116347 (13h37m42.661173214s) [upload=14032]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:17:20.917123150 (13h37m42.661180014s) [upload=14032]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:17:20.917127779 (13h37m42.661184648s) [upload=14032]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:18:20.640884349 (13h38m42.384941214s) [upload=14032]: Done uploading segments... Aug 7 23:18:20.640914749 (13h38m42.384971617s) [upload=14032]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:18:21.050365938 (13h38m42.79442281s) [upload=14051]: Starting upload Aug 7 23:18:21.050484720 (13h38m42.794541587s) [upload=14051]: Done starting upload Aug 7 23:18:21.050589902 (13h38m42.794646752s) [upload=14051]: Uploading segments... Aug 7 23:18:21.050663797 (13h38m42.794720667s) [upload=14051]: Got next segment. Inline: true Aug 7 23:18:21.050670451 (13h38m42.794727311s) [upload=14051]: Waiting for error group managing segments... Aug 7 23:18:21.050719413 (13h38m42.794776275s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:18:21.050725163 (13h38m42.794782023s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:18:21.050728952 (13h38m42.794785811s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:19:20.780076504 (13h39m42.524133366s) [upload=14051]: Done uploading segments... Aug 7 23:19:20.780099469 (13h39m42.52415633s) [upload=14051]: Upload finished. err: metaclient: context deadline exceeded Duration: 844.629272ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [................X...X.XX....................................] duration: 2.991847005s FAIL (2025-08-07 23:48:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== inished. err: context deadline exceeded Aug 7 23:20:07.202369543 (13h40m28.946426394s) [upload=14082]: Starting upload Aug 7 23:20:07.202426657 (13h40m28.946483509s) [upload=14082]: Done starting upload Aug 7 23:20:07.202696950 (13h40m28.946753814s) [upload=14082]: Uploading segments... Aug 7 23:20:07.202761256 (13h40m28.94681811s) [upload=14082]: Got next segment. Inline: false Aug 7 23:20:07.202807046 (13h40m28.946863908s) [upload=14082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:20:07.202814233 (13h40m28.946871102s) [upload=14082]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:20:45.722093118 (13h41m7.466149983s) [seg_pos={0 0}, upload=14082]: Begin upload segment... Aug 7 23:20:45.722177165 (13h41m7.466234018s) [seg_pos={0 0}, upload=14082]: Done begin upload segment. Aug 7 23:20:45.722218538 (13h41m7.4662754s) [noise=true , address=79.127.205.246:4276 , piece_id=6753PO7TQBULISFOSTSIT32I7DSKQPM6M4TEVF2E7TILSMUU5ARQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.722246298 (13h41m7.46630316s) [noise=true , address=104.36.82.58:4292 , piece_id=KQ2RNSZDMCU7OED4IJC4ERBTMTYVGHJCRSVOFZUXG24LW7H3KHUQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.722330454 (13h41m7.466387316s) [noise=true , address=64.49.13.169:4705 , piece_id=YJCLDFSO5ENFREIROYPEZQQTQD4GM4A3DFKNJEHSPFDOKDCFEJ7Q, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.722658583 (13h41m7.466715506s) [noise=true , address=79.127.205.234:4597 , piece_id=WETMAYMNBWD6F6JYCXPNAG5KOJXNCXE3KM7ZJ4F5Y3GEBPHLMDEA, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.722717191 (13h41m7.46677405s) [noise=true , address=104.36.82.5:4867 , piece_id=IEDY33GIP4MZRSN2MKKBLDFAXPMKJUPZVPMOXVCEFJ6BUPZJHYHQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.722989701 (13h41m7.467046568s) [noise=true , address=64.49.13.139:4474 , piece_id=LGXAPMGRQOOEXXG7EBNLO7GAPDKFBCQIBWGYID7KOVQ3UNMDZICQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.723061064 (13h41m7.467117927s) [noise=true , address=45.91.116.14:4201 , piece_id=LCOEAQDFX432OULFXCKDH3FQMZMPR5XSDVKK5MAPEQYEABUAYMDA, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.723458393 (13h41m7.467515259s) [noise=true , address=104.36.82.61:4392 , piece_id=EL44CRMZDDTXOW45GURIDIFQF4K65UUJFKR6L3F3WQBMVXSD2XHQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.729867582 (13h41m7.473924444s) [noise=true , address=104.36.82.60:4345 , piece_id=2AKG3BCGW7TFNVG3FQ576YJOTEZV3V2NP5ITSSZ7BO3NO5CRVDVQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.731063643 (13h41m7.475120512s) [noise=true , address=104.36.82.9:4016 , piece_id=VTTRIIBZ4KCGJW2H5DMG7J63AGFIV7QT34MJJTHRUGFIH6JD6EQQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.732350031 (13h41m7.476406894s) [noise=true , address=104.36.82.52:4106 , piece_id=KTACAAM5SP3PCN3VE6CKF3KHMLGYCHONLC7FDUI4YHLFKIEJ65HQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.733989697 (13h41m7.478046556s) [noise=true , address=64.49.13.155:4916 , piece_id=PJ3VBT33DUHLG3WHE6AK7IIYX76IZZFJCAWX7WX3UUBA6BLGB5VA, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.734727605 (13h41m7.478784474s) [noise=true , address=104.36.82.37:4680 , piece_id=2MDE2TCPX2ROXJXK7Q234YXMK4BFHPDQB5EB7GECHLMUJY5W4ITQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.734786637 (13h41m7.478843504s) [noise=true , address=104.36.82.4:4813 , piece_id=IZO235F25YVNCQG2GSR3UPI5OPMONHVRQQAWDEBOZWR2EILWGCCA, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.735460668 (13h41m7.479517532s) [noise=true , address=104.36.82.6:4901 , piece_id=7VX47BXB37LW2CKYFTBTFAJSDF3QLHEHQHRD5ZM5EHZPAZFPXBPQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.735486537 (13h41m7.479543401s) [noise=true , address=104.36.82.16:4323 , piece_id=RQYKNED6IWTV47W7HGZCVVOUKFRMHGO5WSPCW36ROCXMEDEFYM4Q, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.736565943 (13h41m7.480622804s) [noise=true , address=104.36.82.36:4651 , piece_id=5HQECPYGS5S43JV2CEJY3MRS33NUE6CJ5WPXKTCWYLYVXUQDXLXQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.737386328 (13h41m7.48144319s) [noise=true , address=104.36.82.44:4876 , piece_id=3WUHXX7GVGZWOGDA7FETGVT772RLHO4HRV6OCZKNTZIMG3VDPDOA, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:46.299198843 (13h41m8.043255689s) [upload=14082]: Next returned nil segment Aug 7 23:20:46.299217870 (13h41m8.043274733s) [upload=14082]: Waiting for error group managing segments... Aug 7 23:20:46.464881860 (13h41m8.208938721s) [noise=true , address=104.36.82.60:4345 , piece_id=2AKG3BCGW7TFNVG3FQ576YJOTEZV3V2NP5ITSSZ7BO3NO5CRVDVQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.473775294 (13h41m8.217832164s) [noise=true , address=104.36.82.52:4106 , piece_id=KTACAAM5SP3PCN3VE6CKF3KHMLGYCHONLC7FDUI4YHLFKIEJ65HQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.474664669 (13h41m8.218721541s) [noise=true , address=104.36.82.9:4016 , piece_id=VTTRIIBZ4KCGJW2H5DMG7J63AGFIV7QT34MJJTHRUGFIH6JD6EQQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.479688336 (13h41m8.223745197s) [noise=true , address=104.36.82.36:4651 , piece_id=5HQECPYGS5S43JV2CEJY3MRS33NUE6CJ5WPXKTCWYLYVXUQDXLXQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.485171804 (13h41m8.229228658s) [noise=true , address=104.36.82.6:4901 , piece_id=7VX47BXB37LW2CKYFTBTFAJSDF3QLHEHQHRD5ZM5EHZPAZFPXBPQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.486836180 (13h41m8.230893032s) [noise=true , address=104.36.82.61:4392 , piece_id=EL44CRMZDDTXOW45GURIDIFQF4K65UUJFKR6L3F3WQBMVXSD2XHQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.496476947 (13h41m8.240533796s) [noise=true , address=104.36.82.44:4876 , piece_id=3WUHXX7GVGZWOGDA7FETGVT772RLHO4HRV6OCZKNTZIMG3VDPDOA, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.556494206 (13h41m8.300551081s) [noise=true , address=104.36.82.16:4323 , piece_id=RQYKNED6IWTV47W7HGZCVVOUKFRMHGO5WSPCW36ROCXMEDEFYM4Q, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.628743021 (13h41m8.372799882s) [noise=true , address=45.91.116.14:4201 , piece_id=LCOEAQDFX432OULFXCKDH3FQMZMPR5XSDVKK5MAPEQYEABUAYMDA, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.645987246 (13h41m8.390044106s) [noise=true , address=104.36.82.4:4813 , piece_id=IZO235F25YVNCQG2GSR3UPI5OPMONHVRQQAWDEBOZWR2EILWGCCA, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.666511132 (13h41m8.410567992s) [noise=true , address=104.36.82.5:4867 , piece_id=IEDY33GIP4MZRSN2MKKBLDFAXPMKJUPZVPMOXVCEFJ6BUPZJHYHQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.671756371 (13h41m8.41581322s) [noise=true , address=104.36.82.37:4680 , piece_id=2MDE2TCPX2ROXJXK7Q234YXMK4BFHPDQB5EB7GECHLMUJY5W4ITQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.934025878 (13h41m8.678082729s) [noise=true , address=79.127.205.234:4597 , piece_id=WETMAYMNBWD6F6JYCXPNAG5KOJXNCXE3KM7ZJ4F5Y3GEBPHLMDEA, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.970907150 (13h41m8.714964016s) [noise=true , address=64.49.13.139:4474 , piece_id=LGXAPMGRQOOEXXG7EBNLO7GAPDKFBCQIBWGYID7KOVQ3UNMDZICQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.970940415 (13h41m8.714997268s) [seg_pos={0 0}, upload=14082]: Segment reached optimal threshold of 14 pieces. Aug 7 23:20:46.971273207 (13h41m8.715330074s) [noise=true , address=64.49.13.155:4916 , piece_id=PJ3VBT33DUHLG3WHE6AK7IIYX76IZZFJCAWX7WX3UUBA6BLGB5VA, seg_pos={0 0}, upload=14082]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12MD2UDNXEmeGra9fvhEb8eGL2R6vGLHtSKLpSyX3H3tZCe1r5B): protocol: expected piece hash; context canceled; EOF Aug 7 23:20:46.971460895 (13h41m8.715517755s) [noise=true , address=64.49.13.169:4705 , piece_id=YJCLDFSO5ENFREIROYPEZQQTQD4GM4A3DFKNJEHSPFDOKDCFEJ7Q, seg_pos={0 0}, upload=14082]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124uqzVwsF5TM4CwUhkzfJvgxd71jZXgXuFNwDLRW1vrSXbwLsr): protocol: expected piece hash; context canceled; EOF Aug 7 23:20:46.971541847 (13h41m8.715598706s) [noise=true , address=79.127.205.246:4276 , piece_id=6753PO7TQBULISFOSTSIT32I7DSKQPM6M4TEVF2E7TILSMUU5ARQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJTNVtPFN48S9ZrayehLD2HcVckyJWLx2rtEvyH56W22R8Suq): protocol: expected piece hash; context canceled; EOF Aug 7 23:20:46.971599101 (13h41m8.715655962s) [noise=true , address=104.36.82.58:4292 , piece_id=KQ2RNSZDMCU7OED4IJC4ERBTMTYVGHJCRSVOFZUXG24LW7H3KHUQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JeDjqgszkkq2asUBCL5oTvH6X21TnWkkmqGEEzBbz1YctdDXX): protocol: expected piece hash; context canceled; EOF Aug 7 23:20:46.971618718 (13h41m8.715675568s) [seg_pos={0 0}, upload=14082]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:20:46.971698846 (13h41m8.715755698s) [upload=14082]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:20:46.971704848 (13h41m8.71576171s) [upload=14082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:21:07.219673652 (13h41m28.963730505s) [upload=14082]: Done uploading segments... Aug 7 23:21:07.219701493 (13h41m28.963758344s) [upload=14082]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.991882073s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 1.020276974s FAIL (2025-08-07 23:48:03 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== pload=14121]: Done uploading segments... Aug 7 23:22:28.505845197 (13h42m50.249902044s) [upload=14121]: Upload finished. err: <nil> Aug 7 23:23:10.377483605 (13h43m32.121540458s) [upload=14132]: Starting upload Aug 7 23:23:10.377535635 (13h43m32.121592495s) [upload=14132]: Done starting upload Aug 7 23:23:10.377603797 (13h43m32.121660647s) [upload=14132]: Uploading segments... Aug 7 23:23:10.377638935 (13h43m32.121695804s) [upload=14132]: Got next segment. Inline: true Aug 7 23:23:10.377643594 (13h43m32.121700454s) [upload=14132]: Waiting for error group managing segments... Aug 7 23:23:10.377688252 (13h43m32.121745104s) [upload=14132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:23:10.377693427 (13h43m32.121750286s) [upload=14132]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:23:10.377696805 (13h43m32.121753666s) [upload=14132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:23:10.562260724 (13h43m32.306317573s) [upload=14132]: Done uploading segments... Aug 7 23:23:10.562279228 (13h43m32.306336078s) [upload=14132]: Upload finished. err: <nil> Aug 7 23:24:07.896404858 (13h44m29.640461721s) [upload=14147]: Starting upload Aug 7 23:24:07.896490216 (13h44m29.640547079s) [upload=14147]: Done starting upload Aug 7 23:24:07.896580488 (13h44m29.640637339s) [upload=14147]: Uploading segments... Aug 7 23:24:07.896655006 (13h44m29.640711874s) [upload=14147]: Got next segment. Inline: true Aug 7 23:24:07.896660179 (13h44m29.640717039s) [upload=14147]: Waiting for error group managing segments... Aug 7 23:24:07.896714840 (13h44m29.6407717s) [upload=14147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:24:07.896719399 (13h44m29.64077626s) [upload=14147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:24:07.896722726 (13h44m29.640779586s) [upload=14147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:24:08.048898130 (13h44m29.792954991s) [upload=14147]: Done uploading segments... Aug 7 23:24:08.048952526 (13h44m29.793009386s) [upload=14147]: Upload finished. err: <nil> Aug 7 23:24:53.794754379 (13h45m15.53881125s) [upload=14162]: Starting upload Aug 7 23:24:53.794842317 (13h45m15.538899177s) [upload=14162]: Done starting upload Aug 7 23:24:53.794935964 (13h45m15.538992814s) [upload=14162]: Uploading segments... Aug 7 23:24:53.795003752 (13h45m15.539060623s) [upload=14162]: Got next segment. Inline: true Aug 7 23:24:53.795010814 (13h45m15.539067676s) [upload=14162]: Waiting for error group managing segments... Aug 7 23:24:53.795066607 (13h45m15.539123488s) [upload=14162]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:24:53.795158449 (13h45m15.5392153s) [upload=14162]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:24:53.795163166 (13h45m15.539220026s) [upload=14162]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:24:53.946994982 (13h45m15.691051833s) [upload=14162]: Done uploading segments... Aug 7 23:24:53.947013862 (13h45m15.691070711s) [upload=14162]: Upload finished. err: <nil> Aug 7 23:25:40.029856966 (13h46m1.773913836s) [upload=14177]: Starting upload Aug 7 23:25:40.029986938 (13h46m1.774043809s) [upload=14177]: Done starting upload Aug 7 23:25:40.030076796 (13h46m1.774133642s) [upload=14177]: Uploading segments... Aug 7 23:25:40.030132567 (13h46m1.774189438s) [upload=14177]: Got next segment. Inline: true Aug 7 23:25:40.030139885 (13h46m1.774196757s) [upload=14177]: Waiting for error group managing segments... Aug 7 23:25:40.030197588 (13h46m1.774254461s) [upload=14177]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:25:40.030204146 (13h46m1.774261016s) [upload=14177]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:25:40.030207984 (13h46m1.77426485s) [upload=14177]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:25:40.184198742 (13h46m1.928255592s) [upload=14177]: Done uploading segments... Aug 7 23:25:40.184219017 (13h46m1.928275867s) [upload=14177]: Upload finished. err: <nil> Aug 7 23:26:54.625649110 (13h47m16.369705971s) [upload=14198]: Starting upload Aug 7 23:26:54.625738563 (13h47m16.369795424s) [upload=14198]: Done starting upload Aug 7 23:26:54.625842691 (13h47m16.369899539s) [upload=14198]: Uploading segments... Aug 7 23:26:54.626016839 (13h47m16.37007371s) [upload=14198]: Got next segment. Inline: true Aug 7 23:26:54.626052112 (13h47m16.370108973s) [upload=14198]: Waiting for error group managing segments... Aug 7 23:26:54.626129780 (13h47m16.370186645s) [upload=14198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:26:54.626137392 (13h47m16.370194264s) [upload=14198]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:26:54.626143182 (13h47m16.370200048s) [upload=14198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:26:54.775804171 (13h47m16.519861021s) [upload=14198]: Done uploading segments... Aug 7 23:26:54.775824249 (13h47m16.519881102s) [upload=14198]: Upload finished. err: <nil> Aug 7 23:27:32.263746490 (13h47m54.007803361s) [upload=14208]: Starting upload Aug 7 23:27:32.263833655 (13h47m54.007890501s) [upload=14208]: Done starting upload Aug 7 23:27:32.263976536 (13h47m54.008033385s) [upload=14208]: Uploading segments... Aug 7 23:27:32.264070926 (13h47m54.008127796s) [upload=14208]: Got next segment. Inline: true Aug 7 23:27:32.264080356 (13h47m54.00813733s) [upload=14208]: Waiting for error group managing segments... Aug 7 23:27:32.264154657 (13h47m54.008211526s) [upload=14208]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:27:32.264163587 (13h47m54.008220454s) [upload=14208]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:27:32.264172017 (13h47m54.008228914s) [upload=14208]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:27:32.490188559 (13h47m54.23424541s) [upload=14208]: Done uploading segments... Aug 7 23:27:32.490207326 (13h47m54.234264172s) [upload=14208]: Upload finished. err: <nil> Aug 7 23:27:57.927940477 (13h48m19.671997323s) [upload=14217]: Starting upload Aug 7 23:27:57.928016423 (13h48m19.672073283s) [upload=14217]: Done starting upload Aug 7 23:27:57.928101290 (13h48m19.672158142s) [upload=14217]: Uploading segments... Aug 7 23:27:57.928155692 (13h48m19.672212566s) [upload=14217]: Got next segment. Inline: true Aug 7 23:27:57.928162009 (13h48m19.672218869s) [upload=14217]: Waiting for error group managing segments... Aug 7 23:27:57.928213294 (13h48m19.672270145s) [upload=14217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:27:57.928219212 (13h48m19.672276074s) [upload=14217]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:27:57.928223521 (13h48m19.672280383s) [upload=14217]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:27:58.087544005 (13h48m19.831600874s) [upload=14217]: Done uploading segments... Aug 7 23:27:58.087570226 (13h48m19.831627094s) [upload=14217]: Upload finished. err: <nil> Aug 7 23:29:12.765200935 (13h49m34.509257795s) [upload=14237]: Starting upload Aug 7 23:29:12.765253106 (13h49m34.509309977s) [upload=14237]: Done starting upload Aug 7 23:29:12.765324641 (13h49m34.509381499s) [upload=14237]: Uploading segments... Aug 7 23:29:12.765374187 (13h49m34.509431056s) [upload=14237]: Got next segment. Inline: true Aug 7 23:29:12.765378947 (13h49m34.509435807s) [upload=14237]: Waiting for error group managing segments... Aug 7 23:29:12.765430859 (13h49m34.50948772s) [upload=14237]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:29:12.765438106 (13h49m34.50949497s) [upload=14237]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:29:12.765443127 (13h49m34.509499988s) [upload=14237]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:29:12.926437674 (13h49m34.670494535s) [upload=14237]: Done uploading segments... Aug 7 23:29:12.926457418 (13h49m34.670514268s) [upload=14237]: Upload finished. err: <nil> Aug 7 23:29:42.225760316 (13h50m3.969817177s) [upload=14244]: Starting upload Aug 7 23:29:42.225855063 (13h50m3.969911929s) [upload=14244]: Done starting upload Aug 7 23:29:42.225962489 (13h50m3.970019355s) [upload=14244]: Uploading segments... Aug 7 23:29:42.226078484 (13h50m3.970135331s) [upload=14244]: Got next segment. Inline: true Aug 7 23:29:42.226083996 (13h50m3.970140842s) [upload=14244]: Waiting for error group managing segments... Aug 7 23:29:42.226143067 (13h50m3.970199914s) [upload=14244]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:29:42.226148511 (13h50m3.970205371s) [upload=14244]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:29:42.226153006 (13h50m3.970209865s) [upload=14244]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:29:42.401822947 (13h50m4.14587981s) [upload=14244]: Done uploading segments... Aug 7 23:29:42.401847556 (13h50m4.145904405s) [upload=14244]: Upload finished. err: <nil> Aug 7 23:30:31.529840301 (13h50m53.273897165s) [upload=14254]: Starting upload Aug 7 23:30:31.530002531 (13h50m53.274059378s) [upload=14254]: Done starting upload Aug 7 23:30:31.530201544 (13h50m53.274258391s) [upload=14254]: Uploading segments... Aug 7 23:30:31.530298120 (13h50m53.274354985s) [upload=14254]: Got next segment. Inline: true Aug 7 23:30:31.530303457 (13h50m53.274360317s) [upload=14254]: Waiting for error group managing segments... Aug 7 23:30:31.530353721 (13h50m53.274410567s) [upload=14254]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:30:31.530358927 (13h50m53.274415788s) [upload=14254]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:30:31.530363758 (13h50m53.274420619s) [upload=14254]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:30:31.696350967 (13h50m53.440407828s) [upload=14254]: Done uploading segments... Aug 7 23:30:31.696369460 (13h50m53.440426307s) [upload=14254]: Upload finished. err: <nil> Duration: 1.020312502s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.............X.XX..X........................................] duration: 2.846707166s FAIL (2025-08-07 23:47:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.846736211s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [............XXXXXXXXXX...........X..........................] duration: 2.574928729s FAIL (2025-08-07 23:47:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 0:4517 , piece_id=CIP2H4JR7JNEVTTPEDFI7FVDVPI66H6TNQTP7WDTWO3EFDNJWKXA, seg_pos={0 0}, upload=14216]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Aug 7 23:27:57.305395174 (13h48m19.049452024s) [seg_pos={0 0}, upload=14216]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:27:57.305491984 (13h48m19.049548844s) [upload=14216]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:27:57.305496984 (13h48m19.049553844s) [upload=14216]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:27:57.453740728 (13h48m19.197797589s) [upload=14216]: Done uploading segments... Aug 7 23:27:57.453760898 (13h48m19.197817752s) [upload=14216]: Upload finished. err: <nil> Aug 7 23:28:48.175431324 (13h49m9.919488199s) [upload=14229]: Starting upload Aug 7 23:28:48.175526742 (13h49m9.919583612s) [upload=14229]: Done starting upload Aug 7 23:28:48.175776798 (13h49m9.919833649s) [upload=14229]: Uploading segments... Aug 7 23:28:48.175982529 (13h49m9.920039392s) [upload=14229]: Got next segment. Inline: false Aug 7 23:28:48.176023543 (13h49m9.920080406s) [upload=14229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:28:48.176029551 (13h49m9.920086414s) [upload=14229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:28:48.284809382 (13h49m10.02886623s) [seg_pos={0 0}, upload=14229]: Begin upload segment... Aug 7 23:28:48.284855416 (13h49m10.028912275s) [seg_pos={0 0}, upload=14229]: Done begin upload segment. Aug 7 23:28:48.284878655 (13h49m10.028935518s) [noise=true , address=77.90.31.13:4626 , piece_id=QH4O3MHPFYQ6LH4Y6KGGELCSO5KRAHBPXGX4BWZLPTH2EBUBEBZA, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.285087631 (13h49m10.029144514s) [noise=true , address=77.90.31.17:4772 , piece_id=J6QWEIV73JC5VHU5BYRA7WBN5N7QOVTDIEOI4GQIL5MDDY3F32PQ, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.285200468 (13h49m10.029257328s) [noise=true , address=45.91.116.6:4975 , piece_id=VBM5UD6OVZCMPJ6QF664V2ZACFG6UQH6SOPHJFUE4WMYGZZYGCKQ, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.285437499 (13h49m10.029494358s) [noise=true , address=77.90.31.11:4553 , piece_id=WHCCT2GMH2DRWLK7TCINB4JR5C5SETU3XS2ARSIIWQN52E2EFVKQ, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.285628611 (13h49m10.029685478s) [noise=true , address=45.91.116.14:4190 , piece_id=XVD5ISVMY4722XYPEDTYPOHOGYL7XDNPBDNBMUQZNZJLLLYWIXFQ, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.285699438 (13h49m10.029756306s) [noise=true , address=104.36.82.14:4221 , piece_id=6AFA2RGZEI6YLGAGYE6ZSPPBJHMRGQIMRJH44AVZPG5JKCKK2F7A, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.286010251 (13h49m10.030067111s) [noise=true , address=77.90.31.10:4520 , piece_id=S3FSHHVOQS5UVL27Q7GAXEDB3H2QHQZSL6Z36QP7UABBABT3AKQQ, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.286078142 (13h49m10.030135004s) [noise=true , address=104.36.82.13:4185 , piece_id=EVQM6WWKOM5HCW64MOQHEGEZNCGBPB6KHD7H4G47NHEI6FQUAFZQ, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.286284224 (13h49m10.030341088s) [noise=true , address=77.90.31.12:4590 , piece_id=ZKPV2LZS65UFVUK5JWZAWAAVEQCELDU5FZQTU37PAISE6TWMEVKA, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.286553378 (13h49m10.03061024s) [noise=true , address=77.90.31.16:4733 , piece_id=2W4RBFY3CABLH7JFOCISDF4MP6H4EN6CN5F3GF7HANAJPE26TSHA, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.286568001 (13h49m10.030624861s) [noise=true , address=104.36.82.10:4077 , piece_id=4E23BJAD735PGV45SNNRBJQUFEWVUI4JLJDLZHNUXJXWGIQVQPEQ, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.286799362 (13h49m10.030856223s) [noise=true , address=77.90.31.12:4592 , piece_id=B3J4KUKWAUPNMX775I6UU4PD5XXGWECMLNTHPAVWCMFA4GZZQV2Q, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.287294442 (13h49m10.031351312s) [noise=true , address=45.91.116.7:4010 , piece_id=KFN54772PPRIEMWW35UTAFHODB2YVX3NFFBDH6HVIHXHVLBFUJLA, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.287736301 (13h49m10.031793162s) [noise=true , address=77.90.31.19:4841 , piece_id=ZPYXEINGBZIFNFXDANSECW7S7NLRQKVWOTRFUHXRG7YPIM6VUZYA, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.288128520 (13h49m10.032185379s) [noise=true , address=45.91.116.13:4155 , piece_id=L5FRG63YLCXXLHOO2NGEKCMXW3HQXNGUTPONX76ZEHQDIJG76KEQ, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.288446325 (13h49m10.032503213s) [noise=true , address=77.90.31.19:4843 , piece_id=725EIRIK675YXHGOPGBBGXGK4BMMUDOHQOBL36MCDSYUE5XKTI6Q, seg_pos={0 0}, upload=14229]: Uploading piece... Aug 7 23:28:48.381456643 (13h49m10.125513496s) [noise=true , address=77.90.31.13:4626 , piece_id=QH4O3MHPFYQ6LH4Y6KGGELCSO5KRAHBPXGX4BWZLPTH2EBUBEBZA, seg_pos={0 0}, upload=14229]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 139.59.144.22:50332->77.90.31.13:4626: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:28:48.394653940 (13h49m10.138710812s) [noise=true , address=77.90.31.16:4733 , piece_id=2W4RBFY3CABLH7JFOCISDF4MP6H4EN6CN5F3GF7HANAJPE26TSHA, seg_pos={0 0}, upload=14229]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 139.59.144.22:43998->77.90.31.16:4733: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:28:48.897346092 (13h49m10.641402943s) [upload=14229]: Next returned nil segment Aug 7 23:28:48.897427052 (13h49m10.641483913s) [upload=14229]: Waiting for error group managing segments... Aug 7 23:28:49.066298229 (13h49m10.810355077s) [noise=true , address=77.90.31.12:4592 , piece_id=B3J4KUKWAUPNMX775I6UU4PD5XXGWECMLNTHPAVWCMFA4GZZQV2Q, seg_pos={0 0}, upload=14229]: Done uploading piece. err: <nil> Aug 7 23:28:49.082629995 (13h49m10.826686848s) [noise=true , address=104.36.82.13:4185 , piece_id=EVQM6WWKOM5HCW64MOQHEGEZNCGBPB6KHD7H4G47NHEI6FQUAFZQ, seg_pos={0 0}, upload=14229]: Done uploading piece. err: <nil> Aug 7 23:28:49.085963999 (13h49m10.830020845s) [noise=true , address=104.36.82.10:4077 , piece_id=4E23BJAD735PGV45SNNRBJQUFEWVUI4JLJDLZHNUXJXWGIQVQPEQ, seg_pos={0 0}, upload=14229]: Done uploading piece. err: <nil> Aug 7 23:28:49.091538387 (13h49m10.83559524s) [noise=true , address=77.90.31.12:4590 , piece_id=ZKPV2LZS65UFVUK5JWZAWAAVEQCELDU5FZQTU37PAISE6TWMEVKA, seg_pos={0 0}, upload=14229]: Done uploading piece. err: <nil> Aug 7 23:28:49.120549301 (13h49m10.864606147s) [noise=true , address=77.90.31.11:4553 , piece_id=WHCCT2GMH2DRWLK7TCINB4JR5C5SETU3XS2ARSIIWQN52E2EFVKQ, seg_pos={0 0}, upload=14229]: Done uploading piece. err: <nil> Aug 7 23:28:49.126986818 (13h49m10.871043668s) [noise=true , address=77.90.31.17:4772 , piece_id=J6QWEIV73JC5VHU5BYRA7WBN5N7QOVTDIEOI4GQIL5MDDY3F32PQ, seg_pos={0 0}, upload=14229]: Done uploading piece. err: <nil> Aug 7 23:28:49.133663068 (13h49m10.877719922s) [noise=true , address=45.91.116.7:4010 , piece_id=KFN54772PPRIEMWW35UTAFHODB2YVX3NFFBDH6HVIHXHVLBFUJLA, seg_pos={0 0}, upload=14229]: Done uploading piece. err: <nil> Aug 7 23:28:49.151613435 (13h49m10.895670288s) [noise=true , address=45.91.116.13:4155 , piece_id=L5FRG63YLCXXLHOO2NGEKCMXW3HQXNGUTPONX76ZEHQDIJG76KEQ, seg_pos={0 0}, upload=14229]: Done uploading piece. err: <nil> Aug 7 23:28:49.179832443 (13h49m10.923889293s) [noise=true , address=45.91.116.14:4190 , piece_id=XVD5ISVMY4722XYPEDTYPOHOGYL7XDNPBDNBMUQZNZJLLLYWIXFQ, seg_pos={0 0}, upload=14229]: Done uploading piece. err: <nil> Aug 7 23:28:49.235424834 (13h49m10.979481694s) [noise=true , address=77.90.31.10:4520 , piece_id=S3FSHHVOQS5UVL27Q7GAXEDB3H2QHQZSL6Z36QP7UABBABT3AKQQ, seg_pos={0 0}, upload=14229]: Done uploading piece. err: <nil> Aug 7 23:28:49.253356810 (13h49m10.997413672s) [noise=true , address=104.36.82.14:4221 , piece_id=6AFA2RGZEI6YLGAGYE6ZSPPBJHMRGQIMRJH44AVZPG5JKCKK2F7A, seg_pos={0 0}, upload=14229]: Done uploading piece. err: <nil> Aug 7 23:28:49.314647975 (13h49m11.058704826s) [noise=true , address=45.91.116.6:4975 , piece_id=VBM5UD6OVZCMPJ6QF664V2ZACFG6UQH6SOPHJFUE4WMYGZZYGCKQ, seg_pos={0 0}, upload=14229]: Done uploading piece. err: <nil> Aug 7 23:29:17.906721377 (13h49m39.650778264s) [noise=true , address=77.90.31.19:4841 , piece_id=ZPYXEINGBZIFNFXDANSECW7S7NLRQKVWOTRFUHXRG7YPIM6VUZYA, seg_pos={0 0}, upload=14229]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:29:17.906810237 (13h49m39.650867112s) [noise=true , address=77.90.31.19:4843 , piece_id=725EIRIK675YXHGOPGBBGXGK4BMMUDOHQOBL36MCDSYUE5XKTI6Q, seg_pos={0 0}, upload=14229]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 139.59.144.22:58688->77.90.31.19:4843: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:29:17.906834622 (13h49m39.650891505s) [seg_pos={0 0}, upload=14229]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Aug 7 23:29:18.017467630 (13h49m39.761524483s) [upload=14229]: Done uploading segments... Aug 7 23:29:18.017499122 (13h49m39.761555972s) [upload=14229]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.57496548s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.................X...X.XX...................................] duration: 3.069739423s FAIL (2025-08-07 23:47:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== inished. err: context deadline exceeded Aug 7 23:20:07.202369543 (13h40m28.946426394s) [upload=14082]: Starting upload Aug 7 23:20:07.202426657 (13h40m28.946483509s) [upload=14082]: Done starting upload Aug 7 23:20:07.202696950 (13h40m28.946753814s) [upload=14082]: Uploading segments... Aug 7 23:20:07.202761256 (13h40m28.94681811s) [upload=14082]: Got next segment. Inline: false Aug 7 23:20:07.202807046 (13h40m28.946863908s) [upload=14082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:20:07.202814233 (13h40m28.946871102s) [upload=14082]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:20:45.722093118 (13h41m7.466149983s) [seg_pos={0 0}, upload=14082]: Begin upload segment... Aug 7 23:20:45.722177165 (13h41m7.466234018s) [seg_pos={0 0}, upload=14082]: Done begin upload segment. Aug 7 23:20:45.722218538 (13h41m7.4662754s) [noise=true , address=79.127.205.246:4276 , piece_id=6753PO7TQBULISFOSTSIT32I7DSKQPM6M4TEVF2E7TILSMUU5ARQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.722246298 (13h41m7.46630316s) [noise=true , address=104.36.82.58:4292 , piece_id=KQ2RNSZDMCU7OED4IJC4ERBTMTYVGHJCRSVOFZUXG24LW7H3KHUQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.722330454 (13h41m7.466387316s) [noise=true , address=64.49.13.169:4705 , piece_id=YJCLDFSO5ENFREIROYPEZQQTQD4GM4A3DFKNJEHSPFDOKDCFEJ7Q, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.722658583 (13h41m7.466715506s) [noise=true , address=79.127.205.234:4597 , piece_id=WETMAYMNBWD6F6JYCXPNAG5KOJXNCXE3KM7ZJ4F5Y3GEBPHLMDEA, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.722717191 (13h41m7.46677405s) [noise=true , address=104.36.82.5:4867 , piece_id=IEDY33GIP4MZRSN2MKKBLDFAXPMKJUPZVPMOXVCEFJ6BUPZJHYHQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.722989701 (13h41m7.467046568s) [noise=true , address=64.49.13.139:4474 , piece_id=LGXAPMGRQOOEXXG7EBNLO7GAPDKFBCQIBWGYID7KOVQ3UNMDZICQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.723061064 (13h41m7.467117927s) [noise=true , address=45.91.116.14:4201 , piece_id=LCOEAQDFX432OULFXCKDH3FQMZMPR5XSDVKK5MAPEQYEABUAYMDA, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.723458393 (13h41m7.467515259s) [noise=true , address=104.36.82.61:4392 , piece_id=EL44CRMZDDTXOW45GURIDIFQF4K65UUJFKR6L3F3WQBMVXSD2XHQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.729867582 (13h41m7.473924444s) [noise=true , address=104.36.82.60:4345 , piece_id=2AKG3BCGW7TFNVG3FQ576YJOTEZV3V2NP5ITSSZ7BO3NO5CRVDVQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.731063643 (13h41m7.475120512s) [noise=true , address=104.36.82.9:4016 , piece_id=VTTRIIBZ4KCGJW2H5DMG7J63AGFIV7QT34MJJTHRUGFIH6JD6EQQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.732350031 (13h41m7.476406894s) [noise=true , address=104.36.82.52:4106 , piece_id=KTACAAM5SP3PCN3VE6CKF3KHMLGYCHONLC7FDUI4YHLFKIEJ65HQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.733989697 (13h41m7.478046556s) [noise=true , address=64.49.13.155:4916 , piece_id=PJ3VBT33DUHLG3WHE6AK7IIYX76IZZFJCAWX7WX3UUBA6BLGB5VA, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.734727605 (13h41m7.478784474s) [noise=true , address=104.36.82.37:4680 , piece_id=2MDE2TCPX2ROXJXK7Q234YXMK4BFHPDQB5EB7GECHLMUJY5W4ITQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.734786637 (13h41m7.478843504s) [noise=true , address=104.36.82.4:4813 , piece_id=IZO235F25YVNCQG2GSR3UPI5OPMONHVRQQAWDEBOZWR2EILWGCCA, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.735460668 (13h41m7.479517532s) [noise=true , address=104.36.82.6:4901 , piece_id=7VX47BXB37LW2CKYFTBTFAJSDF3QLHEHQHRD5ZM5EHZPAZFPXBPQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.735486537 (13h41m7.479543401s) [noise=true , address=104.36.82.16:4323 , piece_id=RQYKNED6IWTV47W7HGZCVVOUKFRMHGO5WSPCW36ROCXMEDEFYM4Q, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.736565943 (13h41m7.480622804s) [noise=true , address=104.36.82.36:4651 , piece_id=5HQECPYGS5S43JV2CEJY3MRS33NUE6CJ5WPXKTCWYLYVXUQDXLXQ, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:45.737386328 (13h41m7.48144319s) [noise=true , address=104.36.82.44:4876 , piece_id=3WUHXX7GVGZWOGDA7FETGVT772RLHO4HRV6OCZKNTZIMG3VDPDOA, seg_pos={0 0}, upload=14082]: Uploading piece... Aug 7 23:20:46.299198843 (13h41m8.043255689s) [upload=14082]: Next returned nil segment Aug 7 23:20:46.299217870 (13h41m8.043274733s) [upload=14082]: Waiting for error group managing segments... Aug 7 23:20:46.464881860 (13h41m8.208938721s) [noise=true , address=104.36.82.60:4345 , piece_id=2AKG3BCGW7TFNVG3FQ576YJOTEZV3V2NP5ITSSZ7BO3NO5CRVDVQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.473775294 (13h41m8.217832164s) [noise=true , address=104.36.82.52:4106 , piece_id=KTACAAM5SP3PCN3VE6CKF3KHMLGYCHONLC7FDUI4YHLFKIEJ65HQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.474664669 (13h41m8.218721541s) [noise=true , address=104.36.82.9:4016 , piece_id=VTTRIIBZ4KCGJW2H5DMG7J63AGFIV7QT34MJJTHRUGFIH6JD6EQQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.479688336 (13h41m8.223745197s) [noise=true , address=104.36.82.36:4651 , piece_id=5HQECPYGS5S43JV2CEJY3MRS33NUE6CJ5WPXKTCWYLYVXUQDXLXQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.485171804 (13h41m8.229228658s) [noise=true , address=104.36.82.6:4901 , piece_id=7VX47BXB37LW2CKYFTBTFAJSDF3QLHEHQHRD5ZM5EHZPAZFPXBPQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.486836180 (13h41m8.230893032s) [noise=true , address=104.36.82.61:4392 , piece_id=EL44CRMZDDTXOW45GURIDIFQF4K65UUJFKR6L3F3WQBMVXSD2XHQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.496476947 (13h41m8.240533796s) [noise=true , address=104.36.82.44:4876 , piece_id=3WUHXX7GVGZWOGDA7FETGVT772RLHO4HRV6OCZKNTZIMG3VDPDOA, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.556494206 (13h41m8.300551081s) [noise=true , address=104.36.82.16:4323 , piece_id=RQYKNED6IWTV47W7HGZCVVOUKFRMHGO5WSPCW36ROCXMEDEFYM4Q, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.628743021 (13h41m8.372799882s) [noise=true , address=45.91.116.14:4201 , piece_id=LCOEAQDFX432OULFXCKDH3FQMZMPR5XSDVKK5MAPEQYEABUAYMDA, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.645987246 (13h41m8.390044106s) [noise=true , address=104.36.82.4:4813 , piece_id=IZO235F25YVNCQG2GSR3UPI5OPMONHVRQQAWDEBOZWR2EILWGCCA, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.666511132 (13h41m8.410567992s) [noise=true , address=104.36.82.5:4867 , piece_id=IEDY33GIP4MZRSN2MKKBLDFAXPMKJUPZVPMOXVCEFJ6BUPZJHYHQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.671756371 (13h41m8.41581322s) [noise=true , address=104.36.82.37:4680 , piece_id=2MDE2TCPX2ROXJXK7Q234YXMK4BFHPDQB5EB7GECHLMUJY5W4ITQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.934025878 (13h41m8.678082729s) [noise=true , address=79.127.205.234:4597 , piece_id=WETMAYMNBWD6F6JYCXPNAG5KOJXNCXE3KM7ZJ4F5Y3GEBPHLMDEA, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.970907150 (13h41m8.714964016s) [noise=true , address=64.49.13.139:4474 , piece_id=LGXAPMGRQOOEXXG7EBNLO7GAPDKFBCQIBWGYID7KOVQ3UNMDZICQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: <nil> Aug 7 23:20:46.970940415 (13h41m8.714997268s) [seg_pos={0 0}, upload=14082]: Segment reached optimal threshold of 14 pieces. Aug 7 23:20:46.971273207 (13h41m8.715330074s) [noise=true , address=64.49.13.155:4916 , piece_id=PJ3VBT33DUHLG3WHE6AK7IIYX76IZZFJCAWX7WX3UUBA6BLGB5VA, seg_pos={0 0}, upload=14082]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12MD2UDNXEmeGra9fvhEb8eGL2R6vGLHtSKLpSyX3H3tZCe1r5B): protocol: expected piece hash; context canceled; EOF Aug 7 23:20:46.971460895 (13h41m8.715517755s) [noise=true , address=64.49.13.169:4705 , piece_id=YJCLDFSO5ENFREIROYPEZQQTQD4GM4A3DFKNJEHSPFDOKDCFEJ7Q, seg_pos={0 0}, upload=14082]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124uqzVwsF5TM4CwUhkzfJvgxd71jZXgXuFNwDLRW1vrSXbwLsr): protocol: expected piece hash; context canceled; EOF Aug 7 23:20:46.971541847 (13h41m8.715598706s) [noise=true , address=79.127.205.246:4276 , piece_id=6753PO7TQBULISFOSTSIT32I7DSKQPM6M4TEVF2E7TILSMUU5ARQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJTNVtPFN48S9ZrayehLD2HcVckyJWLx2rtEvyH56W22R8Suq): protocol: expected piece hash; context canceled; EOF Aug 7 23:20:46.971599101 (13h41m8.715655962s) [noise=true , address=104.36.82.58:4292 , piece_id=KQ2RNSZDMCU7OED4IJC4ERBTMTYVGHJCRSVOFZUXG24LW7H3KHUQ, seg_pos={0 0}, upload=14082]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JeDjqgszkkq2asUBCL5oTvH6X21TnWkkmqGEEzBbz1YctdDXX): protocol: expected piece hash; context canceled; EOF Aug 7 23:20:46.971618718 (13h41m8.715675568s) [seg_pos={0 0}, upload=14082]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:20:46.971698846 (13h41m8.715755698s) [upload=14082]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:20:46.971704848 (13h41m8.71576171s) [upload=14082]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:21:07.219673652 (13h41m28.963730505s) [upload=14082]: Done uploading segments... Aug 7 23:21:07.219701493 (13h41m28.963758344s) [upload=14082]: Upload finished. err: metaclient: context deadline exceeded Duration: 3.069796558s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 8 out of 60 is failed (threshold=3) [...............XXXXXX.XX....................................] duration: 2.120281882s FAIL (2025-08-07 23:47:52 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.120318039s core uplink us1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)