Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 21 out of 60 is failed (threshold=3) [.................................X......XXXXXXXXXXXXXXXXXXXX] duration: 30.107058581s FAIL (2025-07-26 23:21:15 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 17681s) [upload=57294]: Done uploading segments... Jul 26 23:20:44.956048254 (59h45m16.146346836s) [upload=57294]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Jul 26 23:20:45.363912339 (59h45m16.554210936s) [upload=57305]: Starting upload Jul 26 23:20:45.364022863 (59h45m16.55432149s) [upload=57305]: Done starting upload Jul 26 23:20:45.364363900 (59h45m16.554662491s) [upload=57305]: Uploading segments... Jul 26 23:20:45.364649037 (59h45m16.554947622s) [upload=57305]: Got next segment. Inline: false Jul 26 23:20:45.364723928 (59h45m16.555022517s) [upload=57305]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 23:20:45.364730552 (59h45m16.555029153s) [upload=57305]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 23:20:45.475526914 (59h45m16.665825493s) [seg_pos={0 0}, upload=57305]: Begin upload segment... Jul 26 23:20:45.475597758 (59h45m16.665896331s) [seg_pos={0 0}, upload=57305]: Done begin upload segment. Jul 26 23:20:45.475646434 (59h45m16.665945019s) [noise=true , address=104.36.82.3:4767 , piece_id=XMNQSDVPR3TURUB4GNXDXULH64T6NNGUQEG23OBHFMVBYLS677QQ, seg_pos={0 0}, upload=57305]: Uploading piece... Jul 26 23:20:45.476135108 (59h45m16.666433694s) [noise=true , address=45.91.116.7:4011 , piece_id=GMTJGF2UZGMWELZ2KGDKPFBCZ7HWCTALABRWLFH4OTDAWM4E3XSA, seg_pos={0 0}, upload=57305]: Uploading piece... Jul 26 23:20:45.476570372 (59h45m16.666868955s) [noise=true , address=104.36.82.9:4005 , piece_id=4OXE5EXMPEODKPE6ICNNSHXXVEDKIBMY44JXHI3XBSSG6YJMSZNA, seg_pos={0 0}, upload=57305]: Uploading piece... Jul 26 23:20:45.476804530 (59h45m16.667103116s) [noise=true , address=104.36.82.5:4838 , piece_id=EE5OB2OIF24FKOUC3ZJI3FU2AMSWE6STIST2LK5TLK5FFJ7BDARA, seg_pos={0 0}, upload=57305]: Uploading piece... Jul 26 23:20:45.477348363 (59h45m16.667646949s) [noise=true , address=77.90.31.14:4661 , piece_id=MTAE7Z3V3EO6DHH4JTMWZRNMKZCYGKHC5PKB3FY7NFIJ3EZZPX7Q, seg_pos={0 0}, upload=57305]: Uploading piece... Jul 26 23:20:45.477419932 (59h45m16.667718514s) [noise=true , address=45.91.116.8:4045 , piece_id=KYQ3YF4HAXZY3EVMB3YLXFN2MZNJPX4MY4KBSGSTTMBJDGWFTH3Q, seg_pos={0 0}, upload=57305]: Uploading piece... Jul 26 23:20:45.478037013 (59h45m16.668335592s) [noise=true , address=104.36.82.12:4151 , piece_id=GTRPQ2R66XRLIAYTQTTH2T56O2WHTTCCKA2MUTDPI26QNYXGFR7A, seg_pos={0 0}, upload=57305]: Uploading piece... Jul 26 23:20:45.478911881 (59h45m16.669210463s) [noise=true , address=104.36.82.15:4259 , piece_id=SKGC4C5VXKXJ5ZVZKVFSO4OFRPYSVEKRGQUUXVIZ763ZKKHWTXSA, seg_pos={0 0}, upload=57305]: Uploading piece... Jul 26 23:20:45.479232943 (59h45m16.669531525s) [noise=true , address=104.36.82.7:4912 , piece_id=R3JRWNWTRADXO4IUIQU6QC7YG3HO4SKWWWTXNICT2XNI62KHOE6A, seg_pos={0 0}, upload=57305]: Uploading piece... Jul 26 23:20:45.479567945 (59h45m16.669866527s) [noise=true , address=104.36.82.2:4731 , piece_id=QQJUZ6QJQ56OBZ3DQFAWLRXHCDLVOZ6RC4SS64GNMCGGR3EME2PQ, seg_pos={0 0}, upload=57305]: Uploading piece... Jul 26 23:20:45.481296426 (59h45m16.671595011s) [noise=true , address=104.36.82.9:4006 , piece_id=KMALSV7TEVTY75E5WAFNL2NUOQBBKOLWIIRYMXD2B4P35I3QGDUQ, seg_pos={0 0}, upload=57305]: Uploading piece... Jul 26 23:20:45.481746327 (59h45m16.672044909s) [noise=true , address=104.36.82.13:4187 , piece_id=XN3QIZDJRZZ532EOSODXRE62QQPO7DMBRO4HE64KS7EAGF3TIMJA, seg_pos={0 0}, upload=57305]: Uploading piece... Jul 26 23:20:45.482324745 (59h45m16.672623328s) [noise=true , address=45.91.116.5:4297 , piece_id=53VCGVFINRQPCFNLE5RVCIECFOEYWFNVGLPZLGAM6KPTOACTQHWQ, seg_pos={0 0}, upload=57305]: Uploading piece... Jul 26 23:20:45.483132027 (59h45m16.673430612s) [noise=true , address=104.36.82.4:4803 , piece_id=UIKAOMBH7ITDBQBWIZEIIQ7ZT7ADP4QUYGNL2ZHCH5SYL3W7JQ5A, seg_pos={0 0}, upload=57305]: Uploading piece... Jul 26 23:20:45.484011898 (59h45m16.674310481s) [noise=true , address=104.36.82.6:4873 , piece_id=53VKRVBNTPPUUNVTMLHA4DIWEBEUUAUKJDKETJFEGLCLKJNXIUZQ, seg_pos={0 0}, upload=57305]: Uploading piece... Jul 26 23:20:45.485081662 (59h45m16.675380246s) [noise=true , address=45.91.116.13:4154 , piece_id=IGLN65PPSQU66S6K7W36I4FPJKTL74BJUXZPV56TWQ47MJL5OF4A, seg_pos={0 0}, upload=57305]: Uploading piece... Jul 26 23:20:46.095774610 (59h45m17.2860732s) [upload=57305]: Next returned nil segment Jul 26 23:20:46.095796539 (59h45m17.286095127s) [upload=57305]: Waiting for error group managing segments... Jul 26 23:20:46.227575412 (59h45m17.417873997s) [noise=true , address=45.91.116.7:4011 , piece_id=GMTJGF2UZGMWELZ2KGDKPFBCZ7HWCTALABRWLFH4OTDAWM4E3XSA, seg_pos={0 0}, upload=57305]: Done uploading piece. err: <nil> Jul 26 23:20:46.242344844 (59h45m17.432643424s) [noise=true , address=77.90.31.14:4661 , piece_id=MTAE7Z3V3EO6DHH4JTMWZRNMKZCYGKHC5PKB3FY7NFIJ3EZZPX7Q, seg_pos={0 0}, upload=57305]: Done uploading piece. err: <nil> Jul 26 23:20:46.244138036 (59h45m17.434436609s) [noise=true , address=45.91.116.13:4154 , piece_id=IGLN65PPSQU66S6K7W36I4FPJKTL74BJUXZPV56TWQ47MJL5OF4A, seg_pos={0 0}, upload=57305]: Done uploading piece. err: <nil> Jul 26 23:20:46.330464002 (59h45m17.520762568s) [noise=true , address=45.91.116.8:4045 , piece_id=KYQ3YF4HAXZY3EVMB3YLXFN2MZNJPX4MY4KBSGSTTMBJDGWFTH3Q, seg_pos={0 0}, upload=57305]: Done uploading piece. err: <nil> Jul 26 23:20:46.391610125 (59h45m17.581908695s) [noise=true , address=45.91.116.5:4297 , piece_id=53VCGVFINRQPCFNLE5RVCIECFOEYWFNVGLPZLGAM6KPTOACTQHWQ, seg_pos={0 0}, upload=57305]: Done uploading piece. err: <nil> Jul 26 23:20:49.367181845 (59h45m20.55748041s) [noise=true , address=104.36.82.3:4767 , piece_id=XMNQSDVPR3TURUB4GNXDXULH64T6NNGUQEG23OBHFMVBYLS677QQ, seg_pos={0 0}, upload=57305]: Done uploading piece. err: <nil> Jul 26 23:20:52.051109950 (59h45m23.241408522s) [noise=true , address=104.36.82.5:4838 , piece_id=EE5OB2OIF24FKOUC3ZJI3FU2AMSWE6STIST2LK5TLK5FFJ7BDARA, seg_pos={0 0}, upload=57305]: Done uploading piece. err: <nil> Jul 26 23:20:54.652850923 (59h45m25.843149505s) [noise=true , address=104.36.82.6:4873 , piece_id=53VKRVBNTPPUUNVTMLHA4DIWEBEUUAUKJDKETJFEGLCLKJNXIUZQ, seg_pos={0 0}, upload=57305]: Done uploading piece. err: <nil> Jul 26 23:21:03.470547226 (59h45m34.660845799s) [noise=true , address=104.36.82.13:4187 , piece_id=XN3QIZDJRZZ532EOSODXRE62QQPO7DMBRO4HE64KS7EAGF3TIMJA, seg_pos={0 0}, upload=57305]: Done uploading piece. err: <nil> Jul 26 23:21:07.973630144 (59h45m39.163928725s) [noise=true , address=104.36.82.9:4005 , piece_id=4OXE5EXMPEODKPE6ICNNSHXXVEDKIBMY44JXHI3XBSSG6YJMSZNA, seg_pos={0 0}, upload=57305]: Done uploading piece. err: <nil> Jul 26 23:21:08.286686993 (59h45m39.476985559s) [noise=true , address=104.36.82.9:4006 , piece_id=KMALSV7TEVTY75E5WAFNL2NUOQBBKOLWIIRYMXD2B4P35I3QGDUQ, seg_pos={0 0}, upload=57305]: Done uploading piece. err: <nil> Jul 26 23:21:15.086199196 (59h45m46.276497775s) [noise=true , address=104.36.82.4:4803 , piece_id=UIKAOMBH7ITDBQBWIZEIIQ7ZT7ADP4QUYGNL2ZHCH5SYL3W7JQ5A, seg_pos={0 0}, upload=57305]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 23:21:15.086318056 (59h45m46.276616635s) [noise=true , address=104.36.82.15:4259 , piece_id=SKGC4C5VXKXJ5ZVZKVFSO4OFRPYSVEKRGQUUXVIZ763ZKKHWTXSA, seg_pos={0 0}, upload=57305]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:104.36.82.15:4259): protocol: expected piece hash; write tcp 139.59.144.22:51064->104.36.82.15:4259: use of closed network connection; write tcp 139.59.144.22:51064->104.36.82.15:4259: use of closed network connection; EOF Jul 26 23:21:15.086574098 (59h45m46.276872678s) [noise=true , address=104.36.82.12:4151 , piece_id=GTRPQ2R66XRLIAYTQTTH2T56O2WHTTCCKA2MUTDPI26QNYXGFR7A, seg_pos={0 0}, upload=57305]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 139.59.144.22:33738->104.36.82.12:4151: use of closed network connection; write tcp 139.59.144.22:33738->104.36.82.12:4151: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33738->104.36.82.12:4151: use of closed network connection Jul 26 23:21:15.086678406 (59h45m46.276976985s) [noise=true , address=104.36.82.7:4912 , piece_id=R3JRWNWTRADXO4IUIQU6QC7YG3HO4SKWWWTXNICT2XNI62KHOE6A, seg_pos={0 0}, upload=57305]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:104.36.82.7:4912): protocol: write tcp 139.59.144.22:48440->104.36.82.7:4912: use of closed network connection; write tcp 139.59.144.22:48440->104.36.82.7:4912: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48440->104.36.82.7:4912: use of closed network connection Jul 26 23:21:15.086744132 (59h45m46.277042711s) [noise=true , address=104.36.82.2:4731 , piece_id=QQJUZ6QJQ56OBZ3DQFAWLRXHCDLVOZ6RC4SS64GNMCGGR3EME2PQ, seg_pos={0 0}, upload=57305]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:104.36.82.2:4731): protocol: write tcp 139.59.144.22:38538->104.36.82.2:4731: use of closed network connection; write tcp 139.59.144.22:38538->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38538->104.36.82.2:4731: use of closed network connection Jul 26 23:21:15.086761187 (59h45m46.277059761s) [seg_pos={0 0}, upload=57305]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 23:21:15.191876285 (59h45m46.382174855s) [upload=57305]: Done uploading segments... Jul 26 23:21:15.191899730 (59h45m46.382198308s) [upload=57305]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.107078229s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 14 out of 60 is failed (threshold=3) [X...X..X.XX......X.X.......X.......X........XX....X....X.X..] duration: 55.026473085s FAIL (2025-07-26 23:20:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 55.0265063s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 22 out of 60 is failed (threshold=3) [..............X..................X.....XXXXXX.XXXXXXXXXXXXXX] duration: 30.110157686s FAIL (2025-07-26 23:20:49 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 9) Duration: 30.110174982s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 41 out of 60 is failed (threshold=3) [X.XXXX.XXXXX..XXX.....XXX.X.X.XX.X.XXXXXXXXX.XX..X.XXXXXXXX.] duration: 25.842163064s FAIL (2025-07-26 23:20:48 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== pieces (needed at least 14 but got 13) Jul 26 23:19:22.694730691 (59h43m53.885029271s) [upload=57281]: Starting upload Jul 26 23:19:22.694807623 (59h43m53.885106192s) [upload=57281]: Done starting upload Jul 26 23:19:22.695118272 (59h43m53.885416852s) [upload=57281]: Uploading segments... Jul 26 23:19:22.695178146 (59h43m53.885476734s) [upload=57281]: Got next segment. Inline: false Jul 26 23:19:22.695221277 (59h43m53.885519856s) [upload=57281]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 23:19:22.695227525 (59h43m53.885526104s) [upload=57281]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 23:19:22.915508910 (59h43m54.105807499s) [seg_pos={0 0}, upload=57281]: Begin upload segment... Jul 26 23:19:22.915585559 (59h43m54.105884149s) [seg_pos={0 0}, upload=57281]: Done begin upload segment. Jul 26 23:19:22.915634986 (59h43m54.105933576s) [noise=true , address=104.36.82.47:4532 , piece_id=DAOGJMTOL2PBFP37IDO42PFC2YP7YLLJFNHAQ4XKPBYQ5FLBJGSQ, seg_pos={0 0}, upload=57281]: Uploading piece... Jul 26 23:19:22.916294913 (59h43m54.106593501s) [noise=true , address=64.49.13.139:4478 , piece_id=3M532Q2HM6RRRZSVSEMWM4YXHCG3ONTR6CKJEU6HBJQMOQHCOEZQ, seg_pos={0 0}, upload=57281]: Uploading piece... Jul 26 23:19:22.916416403 (59h43m54.106714993s) [noise=true , address=104.36.82.3:4781 , piece_id=F5KGLJ4ZPMGE7MUVS77H7RVOSV3J26TKF2XEBFEZKHULPQB2HGNA, seg_pos={0 0}, upload=57281]: Uploading piece... Jul 26 23:19:22.916842103 (59h43m54.10714069s) [noise=true , address=104.36.82.52:4103 , piece_id=XPEK2IHCAHKMVGZCXEUAEAXFLNPJB5KTPS5AJABYIYRSDMWB32NQ, seg_pos={0 0}, upload=57281]: Uploading piece... Jul 26 23:19:22.917246389 (59h43m54.107544983s) [noise=true , address=104.36.82.46:4502 , piece_id=75S6GUD3IEYAHJEHO7FNIBN2AXRKQ357L34DEIGUZZIQSCYTXICA, seg_pos={0 0}, upload=57281]: Uploading piece... Jul 26 23:19:22.917621008 (59h43m54.107919597s) [noise=true , address=156.146.43.225:4448 , piece_id=MKCOBEIHKWUO4JSM5OOOUPBERZLRXYK3HI3OCOOCRZFRQOL5YQXA, seg_pos={0 0}, upload=57281]: Uploading piece... Jul 26 23:19:22.918014803 (59h43m54.108313395s) [noise=true , address=45.91.116.9:4288 , piece_id=WDLGBZ2YSJPCMFHSPWBMOUBAAUPJJRMLXDVBEGTRUGZF3LFLRTJQ, seg_pos={0 0}, upload=57281]: Uploading piece... Jul 26 23:19:22.918385156 (59h43m54.108683748s) [noise=true , address=77.90.31.12:4620 , piece_id=AOTEUZPIUSSAKG4HC3UQWJAYZCX2JXLVW5QLMRNAE5IO6V6WTN4A, seg_pos={0 0}, upload=57281]: Uploading piece... Jul 26 23:19:22.918459466 (59h43m54.108758273s) [noise=true , address=104.36.82.40:4972 , piece_id=E2DSLG4HG3VVZXCC75AX235Z2ETGCGLN5X5XPE3G7BB5UL55MOYA, seg_pos={0 0}, upload=57281]: Uploading piece... Jul 26 23:19:22.918785249 (59h43m54.109083838s) [noise=true , address=79.127.205.246:4272 , piece_id=HEZQFODPTEZWYT7P2JBQJG7WMOCMY2R245WDEHTFFPB7JO26BXAA, seg_pos={0 0}, upload=57281]: Uploading piece... Jul 26 23:19:22.918801231 (59h43m54.10909981s) [noise=true , address=79.127.205.235:4666 , piece_id=RWT3WAD3YRNBFXBPUWWDFZDIB3HINTFMK5YQVCQJN5HWF6FDNUZA, seg_pos={0 0}, upload=57281]: Uploading piece... Jul 26 23:19:22.918849080 (59h43m54.109147662s) [noise=true , address=104.36.82.55:4195 , piece_id=NDMUDLMHRFT7VHDORHUQIPFJUQMF5U7BPEK63HHKMNA3CEUV7FBQ, seg_pos={0 0}, upload=57281]: Uploading piece... Jul 26 23:19:22.919144112 (59h43m54.109442761s) [noise=true , address=79.127.205.233:4623 , piece_id=3LG7D3U2RXXCBAJ2ZJRRJX2FQHPNAEKDLW3XHZ2NXO2IS5AXKA7Q, seg_pos={0 0}, upload=57281]: Uploading piece... Jul 26 23:19:22.919455304 (59h43m54.109753892s) [noise=true , address=77.90.31.11:4557 , piece_id=YSWXFTL2FY4M4GK2NXQWK4OYK44L2QIACCUEFNIK6S2ECDCWLH7A, seg_pos={0 0}, upload=57281]: Uploading piece... Jul 26 23:19:22.919818250 (59h43m54.110116838s) [noise=true , address=104.36.82.56:4243 , piece_id=Y5EOPEIDPNFQM37YBHIPKK5LMM6QKPVN7UVJ43M7TJZGG6VEDIAA, seg_pos={0 0}, upload=57281]: Uploading piece... Jul 26 23:19:22.920120752 (59h43m54.110419331s) [noise=true , address=104.36.82.54:4161 , piece_id=M3V2J2TL2ONZAIZGCBG4NR6UNTPMYRB4STTBHXS45CV4NM7IIYCA, seg_pos={0 0}, upload=57281]: Uploading piece... Jul 26 23:19:22.920583479 (59h43m54.110882081s) [noise=true , address=104.36.82.53:4129 , piece_id=DX7ML2I6TR4RJTDG2DQN5SZL3WHDNSXEQLXGURCR36Z67562AREQ, seg_pos={0 0}, upload=57281]: Uploading piece... Jul 26 23:19:22.925170438 (59h43m54.115469005s) [noise=true , address=104.36.82.13:4188 , piece_id=WN2PJBLY22UCRL2HWCZHBOQAY6O6DHPPUJGUCUQB3DR3BFJFMYFA, seg_pos={0 0}, upload=57281]: Uploading piece... Jul 26 23:19:23.494980104 (59h43m54.685278676s) [upload=57281]: Next returned nil segment Jul 26 23:19:23.495021038 (59h43m54.685319619s) [upload=57281]: Waiting for error group managing segments... Jul 26 23:19:23.590568776 (59h43m54.780867365s) [noise=true , address=64.49.13.139:4478 , piece_id=3M532Q2HM6RRRZSVSEMWM4YXHCG3ONTR6CKJEU6HBJQMOQHCOEZQ, seg_pos={0 0}, upload=57281]: Done uploading piece. err: ecclient: upload failed (node:1kmGWoxQSmnEF7KAt2JqGDcmwTYnu82wpaCQUFaBAYUshTY98J, address:64.49.13.139:4478): protocol: storage node overloaded, request limit: 2000 Jul 26 23:19:23.671411384 (59h43m54.861709956s) [noise=true , address=77.90.31.12:4620 , piece_id=AOTEUZPIUSSAKG4HC3UQWJAYZCX2JXLVW5QLMRNAE5IO6V6WTN4A, seg_pos={0 0}, upload=57281]: Done uploading piece. err: <nil> Jul 26 23:19:24.232215279 (59h43m55.422513863s) [noise=true , address=79.127.205.235:4666 , piece_id=RWT3WAD3YRNBFXBPUWWDFZDIB3HINTFMK5YQVCQJN5HWF6FDNUZA, seg_pos={0 0}, upload=57281]: Done uploading piece. err: <nil> Jul 26 23:19:24.261563102 (59h43m55.451861683s) [noise=true , address=79.127.205.246:4272 , piece_id=HEZQFODPTEZWYT7P2JBQJG7WMOCMY2R245WDEHTFFPB7JO26BXAA, seg_pos={0 0}, upload=57281]: Done uploading piece. err: <nil> Jul 26 23:19:24.317576181 (59h43m55.507874768s) [noise=true , address=79.127.205.233:4623 , piece_id=3LG7D3U2RXXCBAJ2ZJRRJX2FQHPNAEKDLW3XHZ2NXO2IS5AXKA7Q, seg_pos={0 0}, upload=57281]: Done uploading piece. err: <nil> Jul 26 23:19:26.086384153 (59h43m57.276682719s) [noise=true , address=45.91.116.9:4288 , piece_id=WDLGBZ2YSJPCMFHSPWBMOUBAAUPJJRMLXDVBEGTRUGZF3LFLRTJQ, seg_pos={0 0}, upload=57281]: Done uploading piece. err: <nil> Jul 26 23:19:26.341677091 (59h43m57.53197566s) [noise=true , address=156.146.43.225:4448 , piece_id=MKCOBEIHKWUO4JSM5OOOUPBERZLRXYK3HI3OCOOCRZFRQOL5YQXA, seg_pos={0 0}, upload=57281]: Done uploading piece. err: <nil> Jul 26 23:19:29.044536453 (59h44m0.234835044s) [noise=true , address=77.90.31.11:4557 , piece_id=YSWXFTL2FY4M4GK2NXQWK4OYK44L2QIACCUEFNIK6S2ECDCWLH7A, seg_pos={0 0}, upload=57281]: Done uploading piece. err: <nil> Jul 26 23:19:37.817773676 (59h44m9.008072242s) [noise=true , address=104.36.82.54:4161 , piece_id=M3V2J2TL2ONZAIZGCBG4NR6UNTPMYRB4STTBHXS45CV4NM7IIYCA, seg_pos={0 0}, upload=57281]: Done uploading piece. err: <nil> Jul 26 23:19:45.991730700 (59h44m17.182029272s) [noise=true , address=104.36.82.56:4243 , piece_id=Y5EOPEIDPNFQM37YBHIPKK5LMM6QKPVN7UVJ43M7TJZGG6VEDIAA, seg_pos={0 0}, upload=57281]: Done uploading piece. err: <nil> Jul 26 23:19:50.447699041 (59h44m21.637997613s) [noise=true , address=104.36.82.52:4103 , piece_id=XPEK2IHCAHKMVGZCXEUAEAXFLNPJB5KTPS5AJABYIYRSDMWB32NQ, seg_pos={0 0}, upload=57281]: Done uploading piece. err: <nil> Jul 26 23:19:50.690387694 (59h44m21.880686261s) [noise=true , address=104.36.82.40:4972 , piece_id=E2DSLG4HG3VVZXCC75AX235Z2ETGCGLN5X5XPE3G7BB5UL55MOYA, seg_pos={0 0}, upload=57281]: Done uploading piece. err: <nil> Jul 26 23:20:10.957824766 (59h44m42.148123339s) [noise=true , address=104.36.82.55:4195 , piece_id=NDMUDLMHRFT7VHDORHUQIPFJUQMF5U7BPEK63HHKMNA3CEUV7FBQ, seg_pos={0 0}, upload=57281]: Done uploading piece. err: <nil> Jul 26 23:20:17.659119651 (59h44m48.849418242s) [noise=true , address=104.36.82.53:4129 , piece_id=DX7ML2I6TR4RJTDG2DQN5SZL3WHDNSXEQLXGURCR36Z67562AREQ, seg_pos={0 0}, upload=57281]: Done uploading piece. err: <nil> Jul 26 23:20:22.424851652 (59h44m53.615150231s) [noise=true , address=104.36.82.3:4781 , piece_id=F5KGLJ4ZPMGE7MUVS77H7RVOSV3J26TKF2XEBFEZKHULPQB2HGNA, seg_pos={0 0}, upload=57281]: Done uploading piece. err: ecclient: upload failed (node:1Y6iagayTdoSQtXrAdeBrvrck8qMNRAmLogi5jUG8bcLrBEswn, address:104.36.82.3:4781): protocol: expected piece hash; write tcp 139.59.144.22:42158->104.36.82.3:4781: use of closed network connection; write tcp 139.59.144.22:42158->104.36.82.3:4781: use of closed network connection; EOF Jul 26 23:20:22.424929301 (59h44m53.615227885s) [noise=true , address=104.36.82.13:4188 , piece_id=WN2PJBLY22UCRL2HWCZHBOQAY6O6DHPPUJGUCUQB3DR3BFJFMYFA, seg_pos={0 0}, upload=57281]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:104.36.82.13:4188): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 23:20:22.425062699 (59h44m53.615361281s) [noise=true , address=104.36.82.47:4532 , piece_id=DAOGJMTOL2PBFP37IDO42PFC2YP7YLLJFNHAQ4XKPBYQ5FLBJGSQ, seg_pos={0 0}, upload=57281]: Done uploading piece. err: ecclient: upload failed (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL, address:104.36.82.47:4532): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 23:20:22.425187374 (59h44m53.615485954s) [noise=true , address=104.36.82.46:4502 , piece_id=75S6GUD3IEYAHJEHO7FNIBN2AXRKQ357L34DEIGUZZIQSCYTXICA, seg_pos={0 0}, upload=57281]: Done uploading piece. err: ecclient: upload failed (node:12HP5xuRAGgTsL11UQh7VRgfuFzwNdEMXgc8zPVv4QknStyv6Ta, address:104.36.82.46:4502): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 23:20:22.425233933 (59h44m53.615532525s) [seg_pos={0 0}, upload=57281]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 26 23:20:22.534153187 (59h44m53.724451759s) [upload=57281]: Done uploading segments... Jul 26 23:20:22.534183900 (59h44m53.724482488s) [upload=57281]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 25.842205929s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 20 out of 60 is failed (threshold=3) [..................................X......XXXXXXXXXXXXXXXXXXX] duration: 30.113408774s FAIL (2025-07-26 23:20:44 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ZXJ2WUDXHA5ZHS2HUPI7A, seg_pos={0 0}, upload=57286]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: read tcp 139.59.144.22:36954->104.36.82.6:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 23:20:14.584753080 (59h44m45.775051669s) [seg_pos={0 0}, upload=57286]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Jul 26 23:20:14.694921960 (59h44m45.885220545s) [upload=57286]: Done uploading segments... Jul 26 23:20:14.694966542 (59h44m45.885265124s) [upload=57286]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Jul 26 23:20:15.118900596 (59h44m46.309199177s) [upload=57294]: Starting upload Jul 26 23:20:15.119034788 (59h44m46.309333371s) [upload=57294]: Done starting upload Jul 26 23:20:15.119216268 (59h44m46.309514865s) [upload=57294]: Uploading segments... Jul 26 23:20:15.119455994 (59h44m46.309754576s) [upload=57294]: Got next segment. Inline: false Jul 26 23:20:15.119549295 (59h44m46.309847875s) [upload=57294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 23:20:15.119556909 (59h44m46.309855488s) [upload=57294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 23:20:15.228281795 (59h44m46.418580374s) [seg_pos={0 0}, upload=57294]: Begin upload segment... Jul 26 23:20:15.228530645 (59h44m46.418829228s) [seg_pos={0 0}, upload=57294]: Done begin upload segment. Jul 26 23:20:15.228577566 (59h44m46.418876154s) [noise=true , address=104.36.82.14:4222 , piece_id=YA4WPW57QNNMJARTYUSNOKAHVF6VT5X4VZ43K6DVDXQVUNLG2CGQ, seg_pos={0 0}, upload=57294]: Uploading piece... Jul 26 23:20:15.229034941 (59h44m46.419333521s) [noise=true , address=104.36.82.4:4802 , piece_id=CAZRW7CHNRQ4FIJZVDZSN3JNYNJTPNWQEYYZGIUHH76RRKVILKBA, seg_pos={0 0}, upload=57294]: Uploading piece... Jul 26 23:20:15.229305973 (59h44m46.419604552s) [noise=true , address=104.36.82.11:4115 , piece_id=USZBT4SR6GPFJ3EWHDD5S4ZYWEIP5O6SPAXMGCOHDS4UUNOKQZCQ, seg_pos={0 0}, upload=57294]: Uploading piece... Jul 26 23:20:15.229583804 (59h44m46.419882386s) [noise=true , address=104.36.82.16:4294 , piece_id=LUIMNU35INR5C75TD2WW2NMMYWPD3SXF2WWYY6ITYGKXQ6BUO4YQ, seg_pos={0 0}, upload=57294]: Uploading piece... Jul 26 23:20:15.229852324 (59h44m46.420150903s) [noise=true , address=45.91.116.6:4975 , piece_id=4ZJKXHRLFGUXDKBN5MT6A3OAZG3A23TXMCMJJJ44KLMHLOEYTBGQ, seg_pos={0 0}, upload=57294]: Uploading piece... Jul 26 23:20:15.230159975 (59h44m46.420458554s) [noise=true , address=45.91.116.4:4939 , piece_id=LIWUMLFRHVD6XH4ZH3W2LH6JQL4QC6CUYRJPD6P6KQT3MYCCGSUA, seg_pos={0 0}, upload=57294]: Uploading piece... Jul 26 23:20:15.230417750 (59h44m46.42071633s) [noise=true , address=104.36.82.13:4186 , piece_id=LB6VJ65CRKTOXVQILHWUSCQAXYIG54FD2LLATPHDWFYJWY7AVF4A, seg_pos={0 0}, upload=57294]: Uploading piece... Jul 26 23:20:15.230708232 (59h44m46.421006811s) [noise=true , address=45.91.116.13:4155 , piece_id=Y7I5PZFXRETHQYNXYBU3JK5ZLYY4427J3RF4I3UPT6BBRMAHZD2Q, seg_pos={0 0}, upload=57294]: Uploading piece... Jul 26 23:20:15.230964604 (59h44m46.421263182s) [noise=true , address=45.91.116.5:4297 , piece_id=RHMQNVJH3EE4GTNIGXT4L7J53NP6EUHPJV5LA4HHYTYQ7ADXJIVA, seg_pos={0 0}, upload=57294]: Uploading piece... Jul 26 23:20:15.231239024 (59h44m46.421537608s) [noise=true , address=104.36.82.12:4151 , piece_id=PFFZEMYHOYVC5SZ47NKO55XVKWM74K7TW4SGZNQAJARLLLQEW6DQ, seg_pos={0 0}, upload=57294]: Uploading piece... Jul 26 23:20:15.231543377 (59h44m46.421841956s) [noise=true , address=45.91.116.4:4938 , piece_id=7NJGAEMQNRN65ZKNK672GSGLD7EX6JB6K5M6UY3SB5BWW73WOZ3A, seg_pos={0 0}, upload=57294]: Uploading piece... Jul 26 23:20:15.231828505 (59h44m46.422127086s) [noise=true , address=45.91.116.13:4153 , piece_id=B54ZFENPAVN4AQIH3PLBRSLGYA7AGDQUV5HBLGT6NIEQBCYDYDLQ, seg_pos={0 0}, upload=57294]: Uploading piece... Jul 26 23:20:15.232128092 (59h44m46.422426671s) [noise=true , address=45.91.116.7:4009 , piece_id=YE5U7BR7FGR73S6HIYBAORIFBNA7W77QUB67LK2JVZJOESNSD2YQ, seg_pos={0 0}, upload=57294]: Uploading piece... Jul 26 23:20:15.232461744 (59h44m46.422760328s) [noise=true , address=77.90.31.14:4661 , piece_id=I4JTPBRE7IPN774L6HTL5TZVH7HZC7IYZ57IDKFFQRE2LZFWFFDA, seg_pos={0 0}, upload=57294]: Uploading piece... Jul 26 23:20:15.232844945 (59h44m46.423143528s) [noise=true , address=45.91.116.10:4081 , piece_id=K5LGT3CADUUKZ3FAFSRRWHPG2UBBF4NETDV4GCVHW3XZZH7OCNQQ, seg_pos={0 0}, upload=57294]: Uploading piece... Jul 26 23:20:15.233173553 (59h44m46.423472134s) [noise=true , address=45.91.116.12:4227 , piece_id=BLY3SJ7V4HW3YEOIWM67KHFRBAL5E47FGLGYMEKJWSV3H637NOWQ, seg_pos={0 0}, upload=57294]: Uploading piece... Jul 26 23:20:15.865033002 (59h44m47.055331568s) [upload=57294]: Next returned nil segment Jul 26 23:20:15.865063681 (59h44m47.05536226s) [upload=57294]: Waiting for error group managing segments... Jul 26 23:20:15.994931321 (59h44m47.185229909s) [noise=true , address=45.91.116.4:4938 , piece_id=7NJGAEMQNRN65ZKNK672GSGLD7EX6JB6K5M6UY3SB5BWW73WOZ3A, seg_pos={0 0}, upload=57294]: Done uploading piece. err: <nil> Jul 26 23:20:15.998664316 (59h44m47.188962903s) [noise=true , address=45.91.116.10:4081 , piece_id=K5LGT3CADUUKZ3FAFSRRWHPG2UBBF4NETDV4GCVHW3XZZH7OCNQQ, seg_pos={0 0}, upload=57294]: Done uploading piece. err: <nil> Jul 26 23:20:15.999488453 (59h44m47.189787045s) [noise=true , address=77.90.31.14:4661 , piece_id=I4JTPBRE7IPN774L6HTL5TZVH7HZC7IYZ57IDKFFQRE2LZFWFFDA, seg_pos={0 0}, upload=57294]: Done uploading piece. err: <nil> Jul 26 23:20:15.999925460 (59h44m47.19022403s) [noise=true , address=45.91.116.4:4939 , piece_id=LIWUMLFRHVD6XH4ZH3W2LH6JQL4QC6CUYRJPD6P6KQT3MYCCGSUA, seg_pos={0 0}, upload=57294]: Done uploading piece. err: <nil> Jul 26 23:20:16.000036472 (59h44m47.190335071s) [noise=true , address=45.91.116.13:4153 , piece_id=B54ZFENPAVN4AQIH3PLBRSLGYA7AGDQUV5HBLGT6NIEQBCYDYDLQ, seg_pos={0 0}, upload=57294]: Done uploading piece. err: <nil> Jul 26 23:20:16.000790676 (59h44m47.191089263s) [noise=true , address=45.91.116.13:4155 , piece_id=Y7I5PZFXRETHQYNXYBU3JK5ZLYY4427J3RF4I3UPT6BBRMAHZD2Q, seg_pos={0 0}, upload=57294]: Done uploading piece. err: <nil> Jul 26 23:20:16.001625387 (59h44m47.191923981s) [noise=true , address=45.91.116.6:4975 , piece_id=4ZJKXHRLFGUXDKBN5MT6A3OAZG3A23TXMCMJJJ44KLMHLOEYTBGQ, seg_pos={0 0}, upload=57294]: Done uploading piece. err: <nil> Jul 26 23:20:16.019852479 (59h44m47.210151059s) [noise=true , address=45.91.116.12:4227 , piece_id=BLY3SJ7V4HW3YEOIWM67KHFRBAL5E47FGLGYMEKJWSV3H637NOWQ, seg_pos={0 0}, upload=57294]: Done uploading piece. err: <nil> Jul 26 23:20:16.046056930 (59h44m47.236355516s) [noise=true , address=45.91.116.7:4009 , piece_id=YE5U7BR7FGR73S6HIYBAORIFBNA7W77QUB67LK2JVZJOESNSD2YQ, seg_pos={0 0}, upload=57294]: Done uploading piece. err: <nil> Jul 26 23:20:16.594307691 (59h44m47.784606261s) [noise=true , address=45.91.116.5:4297 , piece_id=RHMQNVJH3EE4GTNIGXT4L7J53NP6EUHPJV5LA4HHYTYQ7ADXJIVA, seg_pos={0 0}, upload=57294]: Done uploading piece. err: <nil> Jul 26 23:20:35.854225098 (59h45m7.044523667s) [noise=true , address=104.36.82.14:4222 , piece_id=YA4WPW57QNNMJARTYUSNOKAHVF6VT5X4VZ43K6DVDXQVUNLG2CGQ, seg_pos={0 0}, upload=57294]: Done uploading piece. err: <nil> Jul 26 23:20:40.173999106 (59h45m11.364297686s) [noise=true , address=104.36.82.4:4802 , piece_id=CAZRW7CHNRQ4FIJZVDZSN3JNYNJTPNWQEYYZGIUHH76RRKVILKBA, seg_pos={0 0}, upload=57294]: Done uploading piece. err: <nil> Jul 26 23:20:44.844317296 (59h45m16.034615877s) [noise=true , address=104.36.82.13:4186 , piece_id=LB6VJ65CRKTOXVQILHWUSCQAXYIG54FD2LLATPHDWFYJWY7AVF4A, seg_pos={0 0}, upload=57294]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 23:20:44.844817948 (59h45m16.035116528s) [noise=true , address=104.36.82.16:4294 , piece_id=LUIMNU35INR5C75TD2WW2NMMYWPD3SXF2WWYY6ITYGKXQ6BUO4YQ, seg_pos={0 0}, upload=57294]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:104.36.82.16:4294): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 23:20:44.844998962 (59h45m16.035297553s) [noise=true , address=104.36.82.11:4115 , piece_id=USZBT4SR6GPFJ3EWHDD5S4ZYWEIP5O6SPAXMGCOHDS4UUNOKQZCQ, seg_pos={0 0}, upload=57294]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:104.36.82.11:4115): protocol: write tcp 139.59.144.22:58186->104.36.82.11:4115: use of closed network connection; write tcp 139.59.144.22:58186->104.36.82.11:4115: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58186->104.36.82.11:4115: use of closed network connection Jul 26 23:20:44.845175768 (59h45m16.035474354s) [noise=true , address=104.36.82.12:4151 , piece_id=PFFZEMYHOYVC5SZ47NKO55XVKWM74K7TW4SGZNQAJARLLLQEW6DQ, seg_pos={0 0}, upload=57294]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 139.59.144.22:39834->104.36.82.12:4151: use of closed network connection; write tcp 139.59.144.22:39834->104.36.82.12:4151: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39834->104.36.82.12:4151: use of closed network connection Jul 26 23:20:44.845199792 (59h45m16.035498383s) [seg_pos={0 0}, upload=57294]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 23:20:44.956019109 (59h45m16.146317681s) [upload=57294]: Done uploading segments... Jul 26 23:20:44.956048254 (59h45m16.146346836s) [upload=57294]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.113441811s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [..........................................X.XX...XX..X..XXXX] duration: 30.000822819s FAIL (2025-07-26 23:20:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000847774s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 42 out of 60 is failed (threshold=3) [XX.XXXX.XXXXX..XXX.....XXX.X.X.XX.X.XXXXXXXXX.XX..X.XXXXXXXX] duration: 1m0.111253151s FAIL (2025-07-26 23:20:22 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== pieces (needed at least 14 but got 13) Jul 26 23:19:22.694730691 (59h43m53.885029271s) [upload=57281]: Starting upload Jul 26 23:19:22.694807623 (59h43m53.885106192s) [upload=57281]: Done starting upload Jul 26 23:19:22.695118272 (59h43m53.885416852s) [upload=57281]: Uploading segments... Jul 26 23:19:22.695178146 (59h43m53.885476734s) [upload=57281]: Got next segment. Inline: false Jul 26 23:19:22.695221277 (59h43m53.885519856s) [upload=57281]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 23:19:22.695227525 (59h43m53.885526104s) [upload=57281]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 23:19:22.915508910 (59h43m54.105807499s) [seg_pos={0 0}, upload=57281]: Begin upload segment... Jul 26 23:19:22.915585559 (59h43m54.105884149s) [seg_pos={0 0}, upload=57281]: Done begin upload segment. Jul 26 23:19:22.915634986 (59h43m54.105933576s) [noise=true , address=104.36.82.47:4532 , piece_id=DAOGJMTOL2PBFP37IDO42PFC2YP7YLLJFNHAQ4XKPBYQ5FLBJGSQ, seg_pos={0 0}, upload=57281]: Uploading piece... Jul 26 23:19:22.916294913 (59h43m54.106593501s) [noise=true , address=64.49.13.139:4478 , piece_id=3M532Q2HM6RRRZSVSEMWM4YXHCG3ONTR6CKJEU6HBJQMOQHCOEZQ, seg_pos={0 0}, upload=57281]: Uploading piece... Jul 26 23:19:22.916416403 (59h43m54.106714993s) [noise=true , address=104.36.82.3:4781 , piece_id=F5KGLJ4ZPMGE7MUVS77H7RVOSV3J26TKF2XEBFEZKHULPQB2HGNA, seg_pos={0 0}, upload=57281]: Uploading piece... Jul 26 23:19:22.916842103 (59h43m54.10714069s) [noise=true , address=104.36.82.52:4103 , piece_id=XPEK2IHCAHKMVGZCXEUAEAXFLNPJB5KTPS5AJABYIYRSDMWB32NQ, seg_pos={0 0}, upload=57281]: Uploading piece... Jul 26 23:19:22.917246389 (59h43m54.107544983s) [noise=true , address=104.36.82.46:4502 , piece_id=75S6GUD3IEYAHJEHO7FNIBN2AXRKQ357L34DEIGUZZIQSCYTXICA, seg_pos={0 0}, upload=57281]: Uploading piece... Jul 26 23:19:22.917621008 (59h43m54.107919597s) [noise=true , address=156.146.43.225:4448 , piece_id=MKCOBEIHKWUO4JSM5OOOUPBERZLRXYK3HI3OCOOCRZFRQOL5YQXA, seg_pos={0 0}, upload=57281]: Uploading piece... Jul 26 23:19:22.918014803 (59h43m54.108313395s) [noise=true , address=45.91.116.9:4288 , piece_id=WDLGBZ2YSJPCMFHSPWBMOUBAAUPJJRMLXDVBEGTRUGZF3LFLRTJQ, seg_pos={0 0}, upload=57281]: Uploading piece... Jul 26 23:19:22.918385156 (59h43m54.108683748s) [noise=true , address=77.90.31.12:4620 , piece_id=AOTEUZPIUSSAKG4HC3UQWJAYZCX2JXLVW5QLMRNAE5IO6V6WTN4A, seg_pos={0 0}, upload=57281]: Uploading piece... Jul 26 23:19:22.918459466 (59h43m54.108758273s) [noise=true , address=104.36.82.40:4972 , piece_id=E2DSLG4HG3VVZXCC75AX235Z2ETGCGLN5X5XPE3G7BB5UL55MOYA, seg_pos={0 0}, upload=57281]: Uploading piece... Jul 26 23:19:22.918785249 (59h43m54.109083838s) [noise=true , address=79.127.205.246:4272 , piece_id=HEZQFODPTEZWYT7P2JBQJG7WMOCMY2R245WDEHTFFPB7JO26BXAA, seg_pos={0 0}, upload=57281]: Uploading piece... Jul 26 23:19:22.918801231 (59h43m54.10909981s) [noise=true , address=79.127.205.235:4666 , piece_id=RWT3WAD3YRNBFXBPUWWDFZDIB3HINTFMK5YQVCQJN5HWF6FDNUZA, seg_pos={0 0}, upload=57281]: Uploading piece... Jul 26 23:19:22.918849080 (59h43m54.109147662s) [noise=true , address=104.36.82.55:4195 , piece_id=NDMUDLMHRFT7VHDORHUQIPFJUQMF5U7BPEK63HHKMNA3CEUV7FBQ, seg_pos={0 0}, upload=57281]: Uploading piece... Jul 26 23:19:22.919144112 (59h43m54.109442761s) [noise=true , address=79.127.205.233:4623 , piece_id=3LG7D3U2RXXCBAJ2ZJRRJX2FQHPNAEKDLW3XHZ2NXO2IS5AXKA7Q, seg_pos={0 0}, upload=57281]: Uploading piece... Jul 26 23:19:22.919455304 (59h43m54.109753892s) [noise=true , address=77.90.31.11:4557 , piece_id=YSWXFTL2FY4M4GK2NXQWK4OYK44L2QIACCUEFNIK6S2ECDCWLH7A, seg_pos={0 0}, upload=57281]: Uploading piece... Jul 26 23:19:22.919818250 (59h43m54.110116838s) [noise=true , address=104.36.82.56:4243 , piece_id=Y5EOPEIDPNFQM37YBHIPKK5LMM6QKPVN7UVJ43M7TJZGG6VEDIAA, seg_pos={0 0}, upload=57281]: Uploading piece... Jul 26 23:19:22.920120752 (59h43m54.110419331s) [noise=true , address=104.36.82.54:4161 , piece_id=M3V2J2TL2ONZAIZGCBG4NR6UNTPMYRB4STTBHXS45CV4NM7IIYCA, seg_pos={0 0}, upload=57281]: Uploading piece... Jul 26 23:19:22.920583479 (59h43m54.110882081s) [noise=true , address=104.36.82.53:4129 , piece_id=DX7ML2I6TR4RJTDG2DQN5SZL3WHDNSXEQLXGURCR36Z67562AREQ, seg_pos={0 0}, upload=57281]: Uploading piece... Jul 26 23:19:22.925170438 (59h43m54.115469005s) [noise=true , address=104.36.82.13:4188 , piece_id=WN2PJBLY22UCRL2HWCZHBOQAY6O6DHPPUJGUCUQB3DR3BFJFMYFA, seg_pos={0 0}, upload=57281]: Uploading piece... Jul 26 23:19:23.494980104 (59h43m54.685278676s) [upload=57281]: Next returned nil segment Jul 26 23:19:23.495021038 (59h43m54.685319619s) [upload=57281]: Waiting for error group managing segments... Jul 26 23:19:23.590568776 (59h43m54.780867365s) [noise=true , address=64.49.13.139:4478 , piece_id=3M532Q2HM6RRRZSVSEMWM4YXHCG3ONTR6CKJEU6HBJQMOQHCOEZQ, seg_pos={0 0}, upload=57281]: Done uploading piece. err: ecclient: upload failed (node:1kmGWoxQSmnEF7KAt2JqGDcmwTYnu82wpaCQUFaBAYUshTY98J, address:64.49.13.139:4478): protocol: storage node overloaded, request limit: 2000 Jul 26 23:19:23.671411384 (59h43m54.861709956s) [noise=true , address=77.90.31.12:4620 , piece_id=AOTEUZPIUSSAKG4HC3UQWJAYZCX2JXLVW5QLMRNAE5IO6V6WTN4A, seg_pos={0 0}, upload=57281]: Done uploading piece. err: <nil> Jul 26 23:19:24.232215279 (59h43m55.422513863s) [noise=true , address=79.127.205.235:4666 , piece_id=RWT3WAD3YRNBFXBPUWWDFZDIB3HINTFMK5YQVCQJN5HWF6FDNUZA, seg_pos={0 0}, upload=57281]: Done uploading piece. err: <nil> Jul 26 23:19:24.261563102 (59h43m55.451861683s) [noise=true , address=79.127.205.246:4272 , piece_id=HEZQFODPTEZWYT7P2JBQJG7WMOCMY2R245WDEHTFFPB7JO26BXAA, seg_pos={0 0}, upload=57281]: Done uploading piece. err: <nil> Jul 26 23:19:24.317576181 (59h43m55.507874768s) [noise=true , address=79.127.205.233:4623 , piece_id=3LG7D3U2RXXCBAJ2ZJRRJX2FQHPNAEKDLW3XHZ2NXO2IS5AXKA7Q, seg_pos={0 0}, upload=57281]: Done uploading piece. err: <nil> Jul 26 23:19:26.086384153 (59h43m57.276682719s) [noise=true , address=45.91.116.9:4288 , piece_id=WDLGBZ2YSJPCMFHSPWBMOUBAAUPJJRMLXDVBEGTRUGZF3LFLRTJQ, seg_pos={0 0}, upload=57281]: Done uploading piece. err: <nil> Jul 26 23:19:26.341677091 (59h43m57.53197566s) [noise=true , address=156.146.43.225:4448 , piece_id=MKCOBEIHKWUO4JSM5OOOUPBERZLRXYK3HI3OCOOCRZFRQOL5YQXA, seg_pos={0 0}, upload=57281]: Done uploading piece. err: <nil> Jul 26 23:19:29.044536453 (59h44m0.234835044s) [noise=true , address=77.90.31.11:4557 , piece_id=YSWXFTL2FY4M4GK2NXQWK4OYK44L2QIACCUEFNIK6S2ECDCWLH7A, seg_pos={0 0}, upload=57281]: Done uploading piece. err: <nil> Jul 26 23:19:37.817773676 (59h44m9.008072242s) [noise=true , address=104.36.82.54:4161 , piece_id=M3V2J2TL2ONZAIZGCBG4NR6UNTPMYRB4STTBHXS45CV4NM7IIYCA, seg_pos={0 0}, upload=57281]: Done uploading piece. err: <nil> Jul 26 23:19:45.991730700 (59h44m17.182029272s) [noise=true , address=104.36.82.56:4243 , piece_id=Y5EOPEIDPNFQM37YBHIPKK5LMM6QKPVN7UVJ43M7TJZGG6VEDIAA, seg_pos={0 0}, upload=57281]: Done uploading piece. err: <nil> Jul 26 23:19:50.447699041 (59h44m21.637997613s) [noise=true , address=104.36.82.52:4103 , piece_id=XPEK2IHCAHKMVGZCXEUAEAXFLNPJB5KTPS5AJABYIYRSDMWB32NQ, seg_pos={0 0}, upload=57281]: Done uploading piece. err: <nil> Jul 26 23:19:50.690387694 (59h44m21.880686261s) [noise=true , address=104.36.82.40:4972 , piece_id=E2DSLG4HG3VVZXCC75AX235Z2ETGCGLN5X5XPE3G7BB5UL55MOYA, seg_pos={0 0}, upload=57281]: Done uploading piece. err: <nil> Jul 26 23:20:10.957824766 (59h44m42.148123339s) [noise=true , address=104.36.82.55:4195 , piece_id=NDMUDLMHRFT7VHDORHUQIPFJUQMF5U7BPEK63HHKMNA3CEUV7FBQ, seg_pos={0 0}, upload=57281]: Done uploading piece. err: <nil> Jul 26 23:20:17.659119651 (59h44m48.849418242s) [noise=true , address=104.36.82.53:4129 , piece_id=DX7ML2I6TR4RJTDG2DQN5SZL3WHDNSXEQLXGURCR36Z67562AREQ, seg_pos={0 0}, upload=57281]: Done uploading piece. err: <nil> Jul 26 23:20:22.424851652 (59h44m53.615150231s) [noise=true , address=104.36.82.3:4781 , piece_id=F5KGLJ4ZPMGE7MUVS77H7RVOSV3J26TKF2XEBFEZKHULPQB2HGNA, seg_pos={0 0}, upload=57281]: Done uploading piece. err: ecclient: upload failed (node:1Y6iagayTdoSQtXrAdeBrvrck8qMNRAmLogi5jUG8bcLrBEswn, address:104.36.82.3:4781): protocol: expected piece hash; write tcp 139.59.144.22:42158->104.36.82.3:4781: use of closed network connection; write tcp 139.59.144.22:42158->104.36.82.3:4781: use of closed network connection; EOF Jul 26 23:20:22.424929301 (59h44m53.615227885s) [noise=true , address=104.36.82.13:4188 , piece_id=WN2PJBLY22UCRL2HWCZHBOQAY6O6DHPPUJGUCUQB3DR3BFJFMYFA, seg_pos={0 0}, upload=57281]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:104.36.82.13:4188): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 23:20:22.425062699 (59h44m53.615361281s) [noise=true , address=104.36.82.47:4532 , piece_id=DAOGJMTOL2PBFP37IDO42PFC2YP7YLLJFNHAQ4XKPBYQ5FLBJGSQ, seg_pos={0 0}, upload=57281]: Done uploading piece. err: ecclient: upload failed (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL, address:104.36.82.47:4532): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 23:20:22.425187374 (59h44m53.615485954s) [noise=true , address=104.36.82.46:4502 , piece_id=75S6GUD3IEYAHJEHO7FNIBN2AXRKQ357L34DEIGUZZIQSCYTXICA, seg_pos={0 0}, upload=57281]: Done uploading piece. err: ecclient: upload failed (node:12HP5xuRAGgTsL11UQh7VRgfuFzwNdEMXgc8zPVv4QknStyv6Ta, address:104.36.82.46:4502): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 23:20:22.425233933 (59h44m53.615532525s) [seg_pos={0 0}, upload=57281]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 26 23:20:22.534153187 (59h44m53.724451759s) [upload=57281]: Done uploading segments... Jul 26 23:20:22.534183900 (59h44m53.724482488s) [upload=57281]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1m0.111275926s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 21 out of 60 is failed (threshold=3) [...............X..................X.....XXXXXX.XXXXXXXXXXXXX] duration: 30.104923619s FAIL (2025-07-26 23:20:18 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.104941353s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 19 out of 60 is failed (threshold=3) [...................................X......XXXXXXXXXXXXXXXXXX] duration: 30.113895034s FAIL (2025-07-26 23:20:14 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== BY63Z7WBP5TUC4LILFZO73CHX7LCYKAIM63HBQIKSFHQ, seg_pos={0 0}, upload=57286]: Uploading piece... Jul 26 23:19:44.971244687 (59h44m16.161543256s) [noise=true , address=104.36.82.13:4186 , piece_id=YIPEKKFLS7GGD6TLU5PBWAZ3U65GSKJSS2PJ523QXTDIMZK5UOJA, seg_pos={0 0}, upload=57286]: Uploading piece... Jul 26 23:19:44.971460986 (59h44m16.161759575s) [noise=true , address=104.36.82.15:4258 , piece_id=TDOTDUY52UWLXUYUPL5EL74PX4YFYCDKYF2KQ7YQ4PC3V6Y5M7YA, seg_pos={0 0}, upload=57286]: Uploading piece... Jul 26 23:19:44.971570261 (59h44m16.161868844s) [noise=true , address=104.36.82.7:4909 , piece_id=NWOHVTNS6ZHDQCTYIRA2BGMBIC2VZMVZ37EP4R5M2BSKGCXNLPRQ, seg_pos={0 0}, upload=57286]: Uploading piece... Jul 26 23:19:44.971876677 (59h44m16.162175259s) [noise=true , address=104.36.82.6:4876 , piece_id=CWG4ERBGWRSWQS4Z7UBXNX7XONYJNF4ZXJ2WUDXHA5ZHS2HUPI7A, seg_pos={0 0}, upload=57286]: Uploading piece... Jul 26 23:19:44.972209699 (59h44m16.162508287s) [noise=true , address=104.36.82.9:4006 , piece_id=3CW52PQ22PX5K5A2WXC4J2NSHXU6FWFAGNIPAXSVJKR5QRLJP27A, seg_pos={0 0}, upload=57286]: Uploading piece... Jul 26 23:19:44.973294787 (59h44m16.163593377s) [noise=true , address=104.36.82.2:4729 , piece_id=NOVKZX57RMMBBU34SHVRICWY5F664GZFLJIFZEVTYGQVWAZ5Z3HQ, seg_pos={0 0}, upload=57286]: Uploading piece... Jul 26 23:19:44.974471924 (59h44m16.164770515s) [noise=true , address=104.36.82.10:4078 , piece_id=BS3AVDAASXN2ICX6NSV25H2ZE3G2RPJFYNFJFQITX2BY3GKI6CRQ, seg_pos={0 0}, upload=57286]: Uploading piece... Jul 26 23:19:44.974857808 (59h44m16.16515639s) [noise=true , address=104.36.82.13:4187 , piece_id=DWARQELZOYYIIPTQNTYAGYHGYDBD4D6I47JN5OBC6KMLEXJXZHCQ, seg_pos={0 0}, upload=57286]: Uploading piece... Jul 26 23:19:44.975197232 (59h44m16.165495811s) [noise=true , address=45.91.116.7:4010 , piece_id=CEO7BJZOATN7IWBFGYVK4S6SY3RAUYGZWXKKLDPBIVNATYPN6ZKQ, seg_pos={0 0}, upload=57286]: Uploading piece... Jul 26 23:19:44.975512061 (59h44m16.165810641s) [noise=true , address=45.91.116.7:4009 , piece_id=YN23AHINLLO4AET664OTVQ6HPLM3YX25VAIWA3TJOQ5IV3M4XBDA, seg_pos={0 0}, upload=57286]: Uploading piece... Jul 26 23:19:44.975783121 (59h44m16.166081699s) [noise=true , address=77.90.31.11:4553 , piece_id=2FFRGZO6TS3FTPZU6V2KOONSUHJZB47XGQWLTDA3B2RDTX76QC7Q, seg_pos={0 0}, upload=57286]: Uploading piece... Jul 26 23:19:44.976178330 (59h44m16.166476922s) [noise=true , address=45.91.116.6:4975 , piece_id=37X4STF2N2S2HNTOO6WU73MYJJYOC622GYH5GQA4KNDC7JPB6KKQ, seg_pos={0 0}, upload=57286]: Uploading piece... Jul 26 23:19:44.976630988 (59h44m16.166929577s) [noise=true , address=104.36.82.10:4077 , piece_id=MW5TGXH4PEHR7HQOALERNUXGZJ7WXVN3WHMZKYAIEYXCBVZSIAZA, seg_pos={0 0}, upload=57286]: Uploading piece... Jul 26 23:19:44.977206096 (59h44m16.167504686s) [noise=true , address=45.91.116.4:4937 , piece_id=G5CXG7DHHC5MNEX5QDCTF7SUVDBBJBIXLO4KGLVVZ5Q3JKP5SBIA, seg_pos={0 0}, upload=57286]: Uploading piece... Jul 26 23:19:45.590567983 (59h44m16.780866569s) [upload=57286]: Next returned nil segment Jul 26 23:19:45.590592156 (59h44m16.780890746s) [upload=57286]: Waiting for error group managing segments... Jul 26 23:19:45.730560291 (59h44m16.920858882s) [noise=true , address=45.91.116.7:4009 , piece_id=YN23AHINLLO4AET664OTVQ6HPLM3YX25VAIWA3TJOQ5IV3M4XBDA, seg_pos={0 0}, upload=57286]: Done uploading piece. err: <nil> Jul 26 23:19:45.730757128 (59h44m16.921055699s) [noise=true , address=45.91.116.10:4082 , piece_id=BTSRFN7W6CPBEVEOWNSZMPKA5UG44X7DMDILNVYVYPDWGYA3MCPQ, seg_pos={0 0}, upload=57286]: Done uploading piece. err: <nil> Jul 26 23:19:45.731260202 (59h44m16.921558785s) [noise=true , address=45.91.116.6:4975 , piece_id=37X4STF2N2S2HNTOO6WU73MYJJYOC622GYH5GQA4KNDC7JPB6KKQ, seg_pos={0 0}, upload=57286]: Done uploading piece. err: <nil> Jul 26 23:19:45.737150860 (59h44m16.927449428s) [noise=true , address=45.91.116.8:4048 , piece_id=Z5Y4DWXGBY63Z7WBP5TUC4LILFZO73CHX7LCYKAIM63HBQIKSFHQ, seg_pos={0 0}, upload=57286]: Done uploading piece. err: <nil> Jul 26 23:19:45.740190358 (59h44m16.930488926s) [noise=true , address=45.91.116.4:4937 , piece_id=G5CXG7DHHC5MNEX5QDCTF7SUVDBBJBIXLO4KGLVVZ5Q3JKP5SBIA, seg_pos={0 0}, upload=57286]: Done uploading piece. err: <nil> Jul 26 23:19:45.740279774 (59h44m16.930578356s) [noise=true , address=45.91.116.7:4010 , piece_id=CEO7BJZOATN7IWBFGYVK4S6SY3RAUYGZWXKKLDPBIVNATYPN6ZKQ, seg_pos={0 0}, upload=57286]: Done uploading piece. err: <nil> Jul 26 23:19:45.769365074 (59h44m16.959663654s) [noise=true , address=77.90.31.11:4553 , piece_id=2FFRGZO6TS3FTPZU6V2KOONSUHJZB47XGQWLTDA3B2RDTX76QC7Q, seg_pos={0 0}, upload=57286]: Done uploading piece. err: <nil> Jul 26 23:20:14.582620635 (59h44m45.77291922s) [noise=true , address=104.36.82.10:4078 , piece_id=BS3AVDAASXN2ICX6NSV25H2ZE3G2RPJFYNFJFQITX2BY3GKI6CRQ, seg_pos={0 0}, upload=57286]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 23:20:14.582796540 (59h44m45.773095122s) [noise=true , address=104.36.82.10:4077 , piece_id=MW5TGXH4PEHR7HQOALERNUXGZJ7WXVN3WHMZKYAIEYXCBVZSIAZA, seg_pos={0 0}, upload=57286]: Done uploading piece. err: ecclient: upload failed (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk, address:104.36.82.10:4077): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 23:20:14.583680782 (59h44m45.773979366s) [noise=true , address=104.36.82.7:4909 , piece_id=NWOHVTNS6ZHDQCTYIRA2BGMBIC2VZMVZ37EP4R5M2BSKGCXNLPRQ, seg_pos={0 0}, upload=57286]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: write tcp 139.59.144.22:39018->104.36.82.7:4909: use of closed network connection; write tcp 139.59.144.22:39018->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39018->104.36.82.7:4909: use of closed network connection Jul 26 23:20:14.584112388 (59h44m45.774410978s) [noise=true , address=104.36.82.13:4187 , piece_id=DWARQELZOYYIIPTQNTYAGYHGYDBD4D6I47JN5OBC6KMLEXJXZHCQ, seg_pos={0 0}, upload=57286]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: write tcp 139.59.144.22:60650->104.36.82.13:4187: use of closed network connection; write tcp 139.59.144.22:60650->104.36.82.13:4187: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60650->104.36.82.13:4187: use of closed network connection Jul 26 23:20:14.584235668 (59h44m45.774534251s) [noise=true , address=104.36.82.13:4186 , piece_id=YIPEKKFLS7GGD6TLU5PBWAZ3U65GSKJSS2PJ523QXTDIMZK5UOJA, seg_pos={0 0}, upload=57286]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 139.59.144.22:38092->104.36.82.13:4186: use of closed network connection; write tcp 139.59.144.22:38092->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38092->104.36.82.13:4186: use of closed network connection Jul 26 23:20:14.584441152 (59h44m45.774739738s) [noise=true , address=104.36.82.15:4258 , piece_id=TDOTDUY52UWLXUYUPL5EL74PX4YFYCDKYF2KQ7YQ4PC3V6Y5M7YA, seg_pos={0 0}, upload=57286]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:104.36.82.15:4258): protocol: write tcp 139.59.144.22:42768->104.36.82.15:4258: use of closed network connection; write tcp 139.59.144.22:42768->104.36.82.15:4258: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42768->104.36.82.15:4258: use of closed network connection Jul 26 23:20:14.584540642 (59h44m45.774839223s) [noise=true , address=104.36.82.9:4006 , piece_id=3CW52PQ22PX5K5A2WXC4J2NSHXU6FWFAGNIPAXSVJKR5QRLJP27A, seg_pos={0 0}, upload=57286]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:104.36.82.9:4006): protocol: write tcp 139.59.144.22:46422->104.36.82.9:4006: use of closed network connection; write tcp 139.59.144.22:46422->104.36.82.9:4006: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46422->104.36.82.9:4006: use of closed network connection Jul 26 23:20:14.584621200 (59h44m45.774919786s) [noise=true , address=104.36.82.2:4729 , piece_id=NOVKZX57RMMBBU34SHVRICWY5F664GZFLJIFZEVTYGQVWAZ5Z3HQ, seg_pos={0 0}, upload=57286]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:104.36.82.2:4729): protocol: write tcp 139.59.144.22:42738->104.36.82.2:4729: use of closed network connection; write tcp 139.59.144.22:42738->104.36.82.2:4729: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42738->104.36.82.2:4729: use of closed network connection Jul 26 23:20:14.584728013 (59h44m45.775026593s) [noise=true , address=104.36.82.6:4876 , piece_id=CWG4ERBGWRSWQS4Z7UBXNX7XONYJNF4ZXJ2WUDXHA5ZHS2HUPI7A, seg_pos={0 0}, upload=57286]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: read tcp 139.59.144.22:36954->104.36.82.6:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 23:20:14.584753080 (59h44m45.775051669s) [seg_pos={0 0}, upload=57286]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Jul 26 23:20:14.694921960 (59h44m45.885220545s) [upload=57286]: Done uploading segments... Jul 26 23:20:14.694966542 (59h44m45.885265124s) [upload=57286]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 30.113917399s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 15 out of 60 is failed (threshold=3) [XX...X..X.XX......X.X.......X.......X........XX....X....X.X.] duration: 31.967963658s FAIL (2025-07-26 23:19:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 31.967980452s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 20 out of 60 is failed (threshold=3) [................X..................X.....XXXXXX.XXXXXXXXXXXX] duration: 30.115435828s FAIL (2025-07-26 23:19:48 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 8) Duration: 30.115466403s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 18 out of 60 is failed (threshold=3) [....................................X......XXXXXXXXXXXXXXXXX] duration: 30.119492532s FAIL (2025-07-26 23:19:44 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== YYLGSPR2J5MJSWZS35WS7TGF24UZOG5PA, seg_pos={0 0}, upload=57277]: Uploading piece... Jul 26 23:19:14.723913388 (59h43m45.914211973s) [noise=true , address=77.90.31.17:4772 , piece_id=JAOUF2MHIX7TFHZLBPJEMJ4RWUT2XD3TQHUGCI3YRNO6656TQCNQ, seg_pos={0 0}, upload=57277]: Uploading piece... Jul 26 23:19:14.724711035 (59h43m45.915009617s) [noise=true , address=45.91.116.10:4083 , piece_id=KKQWEIZHGH3REWR2BJJ6JNUA37A5K2ZGOMMOY7W4LNPSGTHJJWVA, seg_pos={0 0}, upload=57277]: Uploading piece... Jul 26 23:19:14.724809078 (59h43m45.915107666s) [noise=true , address=77.90.31.15:4698 , piece_id=2XLMXIRZ7OJPJCZ6GDL5SGQX4YKSAMKFROFDDW53S6XKSIKOIZ3Q, seg_pos={0 0}, upload=57277]: Uploading piece... Jul 26 23:19:14.725429680 (59h43m45.915728266s) [noise=true , address=104.36.82.7:4911 , piece_id=IUCCIFJY4ZJ3SDFV5OEEHRV4A3ZOFOTZ7P37SIASKBMX5JSICADA, seg_pos={0 0}, upload=57277]: Uploading piece... Jul 26 23:19:14.725465647 (59h43m45.915764235s) [noise=true , address=104.36.82.6:4874 , piece_id=RGRJ6PXRFUEXWY6VSCGJT3S6NFTT7FWFBVO5FWJRSBU7UPTE5TAA, seg_pos={0 0}, upload=57277]: Uploading piece... Jul 26 23:19:14.726081309 (59h43m45.916379892s) [noise=true , address=104.36.82.10:4078 , piece_id=DFKYKBJWXVU25ELRR3KMTAO2TGNEPCD3UFMPIV77PDUW5O6O4FVQ, seg_pos={0 0}, upload=57277]: Uploading piece... Jul 26 23:19:14.726264810 (59h43m45.916563399s) [noise=true , address=104.36.82.15:4257 , piece_id=QWGHCPLHM63UTORKQ6RJQV4JF7SUCEVTG3OFXYED7RTRE2BMOWLA, seg_pos={0 0}, upload=57277]: Uploading piece... Jul 26 23:19:14.726822283 (59h43m45.917120872s) [noise=true , address=104.36.82.3:4765 , piece_id=CE2ICNFGBLLDMNENDF2E77X3H5QH3I52SL6Y2GHEOUH66MQKMKZQ, seg_pos={0 0}, upload=57277]: Uploading piece... Jul 26 23:19:14.726996794 (59h43m45.917295383s) [noise=true , address=104.36.82.13:4185 , piece_id=VGEDBAATCGDIJZ76PM5YG5Y67FOIEVKAKDV6LZ7MDWO64MIJQGVA, seg_pos={0 0}, upload=57277]: Uploading piece... Jul 26 23:19:14.727515375 (59h43m45.917813955s) [noise=true , address=45.91.116.8:4047 , piece_id=3435Q4HIXJXSPNEVPGVLTGFSXUXKU6W2BUAUNCV3ORXEVT73TQPA, seg_pos={0 0}, upload=57277]: Uploading piece... Jul 26 23:19:14.727831554 (59h43m45.918130142s) [noise=true , address=104.36.82.4:4804 , piece_id=DVH3BYGDIPUKCBDEPZU5MPXT2SRPACHYCD7PSSDGODTU53GKE7QQ, seg_pos={0 0}, upload=57277]: Uploading piece... Jul 26 23:19:14.729564770 (59h43m45.919863359s) [noise=true , address=45.91.116.4:4939 , piece_id=5K4S6TIH54RAUBGLZL2UZAKJCTNPXCM6YDOJPBR4D6TMCPUWA6NQ, seg_pos={0 0}, upload=57277]: Uploading piece... Jul 26 23:19:14.731642411 (59h43m45.921941029s) [noise=true , address=45.91.116.7:4011 , piece_id=YZNHJGFIZSKARR34YFUPXWXG2IGD24N276WSF663UEKPWN3AIDMQ, seg_pos={0 0}, upload=57277]: Uploading piece... Jul 26 23:19:15.346677622 (59h43m46.536976188s) [upload=57277]: Next returned nil segment Jul 26 23:19:15.346696440 (59h43m46.536995022s) [upload=57277]: Waiting for error group managing segments... Jul 26 23:19:15.486719166 (59h43m46.677017747s) [noise=true , address=45.91.116.7:4011 , piece_id=YZNHJGFIZSKARR34YFUPXWXG2IGD24N276WSF663UEKPWN3AIDMQ, seg_pos={0 0}, upload=57277]: Done uploading piece. err: <nil> Jul 26 23:19:15.492060143 (59h43m46.682358723s) [noise=true , address=45.91.116.8:4047 , piece_id=3435Q4HIXJXSPNEVPGVLTGFSXUXKU6W2BUAUNCV3ORXEVT73TQPA, seg_pos={0 0}, upload=57277]: Done uploading piece. err: <nil> Jul 26 23:19:15.492448346 (59h43m46.682746916s) [noise=true , address=45.91.116.10:4083 , piece_id=KKQWEIZHGH3REWR2BJJ6JNUA37A5K2ZGOMMOY7W4LNPSGTHJJWVA, seg_pos={0 0}, upload=57277]: Done uploading piece. err: <nil> Jul 26 23:19:15.492726348 (59h43m46.683024913s) [noise=true , address=45.91.116.7:4009 , piece_id=CIB3U4R3H3K3ONPVQZF3S6KIRZYT5PAAXRQM754ETQINSTB4OP7Q, seg_pos={0 0}, upload=57277]: Done uploading piece. err: <nil> Jul 26 23:19:15.495873885 (59h43m46.686172457s) [noise=true , address=45.91.116.4:4939 , piece_id=5K4S6TIH54RAUBGLZL2UZAKJCTNPXCM6YDOJPBR4D6TMCPUWA6NQ, seg_pos={0 0}, upload=57277]: Done uploading piece. err: <nil> Jul 26 23:19:25.873698268 (59h43m57.063996834s) [noise=true , address=77.90.31.15:4698 , piece_id=2XLMXIRZ7OJPJCZ6GDL5SGQX4YKSAMKFROFDDW53S6XKSIKOIZ3Q, seg_pos={0 0}, upload=57277]: Done uploading piece. err: <nil> Jul 26 23:19:44.320107428 (59h44m15.510406012s) [noise=true , address=104.36.82.4:4804 , piece_id=DVH3BYGDIPUKCBDEPZU5MPXT2SRPACHYCD7PSSDGODTU53GKE7QQ, seg_pos={0 0}, upload=57277]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 23:19:44.320480205 (59h44m15.510778784s) [noise=true , address=104.36.82.7:4911 , piece_id=IUCCIFJY4ZJ3SDFV5OEEHRV4A3ZOFOTZ7P37SIASKBMX5JSICADA, seg_pos={0 0}, upload=57277]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:104.36.82.7:4911): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 23:19:44.320920956 (59h44m15.511219535s) [noise=true , address=104.36.82.10:4078 , piece_id=DFKYKBJWXVU25ELRR3KMTAO2TGNEPCD3UFMPIV77PDUW5O6O4FVQ, seg_pos={0 0}, upload=57277]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: write tcp 139.59.144.22:59464->104.36.82.10:4078: use of closed network connection; write tcp 139.59.144.22:59464->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59464->104.36.82.10:4078: use of closed network connection Jul 26 23:19:44.320987461 (59h44m15.51128604s) [noise=true , address=104.36.82.13:4185 , piece_id=VGEDBAATCGDIJZ76PM5YG5Y67FOIEVKAKDV6LZ7MDWO64MIJQGVA, seg_pos={0 0}, upload=57277]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:104.36.82.13:4185): protocol: write tcp 139.59.144.22:45246->104.36.82.13:4185: use of closed network connection; write tcp 139.59.144.22:45246->104.36.82.13:4185: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:45246->104.36.82.13:4185: use of closed network connection Jul 26 23:19:44.321039076 (59h44m15.511337655s) [noise=true , address=104.36.82.3:4765 , piece_id=CE2ICNFGBLLDMNENDF2E77X3H5QH3I52SL6Y2GHEOUH66MQKMKZQ, seg_pos={0 0}, upload=57277]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:104.36.82.3:4765): protocol: write tcp 139.59.144.22:41180->104.36.82.3:4765: use of closed network connection; write tcp 139.59.144.22:41180->104.36.82.3:4765: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41180->104.36.82.3:4765: use of closed network connection Jul 26 23:19:44.321102391 (59h44m15.51140097s) [noise=true , address=104.36.82.9:4006 , piece_id=7LGPWVQCLW6SSJMGXMDW6NQ7DYIFBK7D7LJNYA3LECKX4KFJTPUQ, seg_pos={0 0}, upload=57277]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:104.36.82.9:4006): protocol: write tcp 139.59.144.22:44678->104.36.82.9:4006: use of closed network connection; write tcp 139.59.144.22:44678->104.36.82.9:4006: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44678->104.36.82.9:4006: use of closed network connection Jul 26 23:19:44.321152394 (59h44m15.511450973s) [noise=true , address=104.36.82.7:4910 , piece_id=IKPUZSOGJFH4HPX6M3HYYLGSPR2J5MJSWZS35WS7TGF24UZOG5PA, seg_pos={0 0}, upload=57277]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: write tcp 139.59.144.22:45824->104.36.82.7:4910: use of closed network connection; write tcp 139.59.144.22:45824->104.36.82.7:4910: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:45824->104.36.82.7:4910: use of closed network connection Jul 26 23:19:44.321213006 (59h44m15.511511585s) [noise=true , address=104.36.82.15:4257 , piece_id=QWGHCPLHM63UTORKQ6RJQV4JF7SUCEVTG3OFXYED7RTRE2BMOWLA, seg_pos={0 0}, upload=57277]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: write tcp 139.59.144.22:35702->104.36.82.15:4257: use of closed network connection; write tcp 139.59.144.22:35702->104.36.82.15:4257: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35702->104.36.82.15:4257: use of closed network connection Jul 26 23:19:44.321266321 (59h44m15.5115649s) [noise=true , address=77.90.31.17:4772 , piece_id=JAOUF2MHIX7TFHZLBPJEMJ4RWUT2XD3TQHUGCI3YRNO6656TQCNQ, seg_pos={0 0}, upload=57277]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 23:19:44.321312008 (59h44m15.511610591s) [noise=true , address=104.36.82.6:4874 , piece_id=RGRJ6PXRFUEXWY6VSCGJT3S6NFTT7FWFBVO5FWJRSBU7UPTE5TAA, seg_pos={0 0}, upload=57277]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: read tcp 139.59.144.22:48600->104.36.82.6:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 23:19:44.321330128 (59h44m15.511628698s) [seg_pos={0 0}, upload=57277]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 26 23:19:44.437844568 (59h44m15.628143169s) [upload=57277]: Done uploading segments... Jul 26 23:19:44.437896025 (59h44m15.628194603s) [upload=57277]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 30.119520549s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [...........................................X.XX...XX..X..XXX] duration: 30.000488453s FAIL (2025-07-26 23:19:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.00052166s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 42 out of 60 is failed (threshold=3) [XXX.XXXX.XXXXX..XXX.....XXX.X.X.XX.X.XXXXXXXXX.XX..X.XXXXXXX] duration: 1m0.112230578s FAIL (2025-07-26 23:19:22 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== load=57251]: Done uploading segments... Jul 26 23:18:22.024661646 (59h42m53.21496023s) [upload=57251]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Jul 26 23:18:22.445339550 (59h42m53.63563813s) [upload=57265]: Starting upload Jul 26 23:18:22.445406998 (59h42m53.635705578s) [upload=57265]: Done starting upload Jul 26 23:18:22.445572242 (59h42m53.635870821s) [upload=57265]: Uploading segments... Jul 26 23:18:22.445802617 (59h42m53.636101183s) [upload=57265]: Got next segment. Inline: false Jul 26 23:18:22.445859392 (59h42m53.636157972s) [upload=57265]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 23:18:22.445864362 (59h42m53.636162941s) [upload=57265]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 23:18:22.649578327 (59h42m53.839876915s) [seg_pos={0 0}, upload=57265]: Begin upload segment... Jul 26 23:18:22.649648972 (59h42m53.839947562s) [seg_pos={0 0}, upload=57265]: Done begin upload segment. Jul 26 23:18:22.649778543 (59h42m53.840077126s) [noise=true , address=45.91.116.12:4230 , piece_id=7OGTVSKUHS7RZZMZWQ5YEMIEVGG63EDZ7SGOK4LZBIHS3YHZZ5VA, seg_pos={0 0}, upload=57265]: Uploading piece... Jul 26 23:18:22.649835971 (59h42m53.840134554s) [noise=true , address=45.91.116.11:4124 , piece_id=Z3TSCRITM7BDFXEKPXT57KSFGFDQK4GLBV4FDZQLNE55GTPVOPPA, seg_pos={0 0}, upload=57265]: Uploading piece... Jul 26 23:18:22.650175294 (59h42m53.840473878s) [noise=true , address=77.90.31.19:4871 , piece_id=5TSWLPP4QY5XG3YESQ62NRGQMRAFB57LTJNE5M3J36AXRSV74ESQ, seg_pos={0 0}, upload=57265]: Uploading piece... Jul 26 23:18:22.650993538 (59h42m53.841292117s) [noise=true , address=104.36.82.54:4189 , piece_id=UNHERSX42IHQB2ZK62ENVCWO7YZQOX7KUUDXDK5DC7O3IJV4J3DA, seg_pos={0 0}, upload=57265]: Uploading piece... Jul 26 23:18:22.651785412 (59h42m53.842083998s) [noise=true , address=104.36.82.53:4125 , piece_id=7L5HHC7SVXVMOJO4WCUGRMHTMJWEL6MEJ2G4KSR3OOEVIGV2CPQA, seg_pos={0 0}, upload=57265]: Uploading piece... Jul 26 23:18:22.652530671 (59h42m53.842829256s) [noise=true , address=79.127.219.38:4007 , piece_id=WUOWIXVQ2YFFIBDP47MBOQ3C6KS246SGKHLWTCIBWLSPVEN5QMHQ, seg_pos={0 0}, upload=57265]: Uploading piece... Jul 26 23:18:22.652948216 (59h42m53.843246798s) [noise=true , address=77.90.31.13:4649 , piece_id=RPSRAX5ZJPNXKKBXYU52PSELLAALXHHB7WPL2XI3RHGMNWTEQ4BA, seg_pos={0 0}, upload=57265]: Uploading piece... Jul 26 23:18:22.653313310 (59h42m53.843611893s) [noise=true , address=79.127.201.215:4347 , piece_id=OL46BT6WG3AVUCDTGNBZWOH2LIU6IF73DL3YQE23COTFXUEGR5VA, seg_pos={0 0}, upload=57265]: Uploading piece... Jul 26 23:18:22.653841596 (59h42m53.844140188s) [noise=true , address=104.36.82.46:4497 , piece_id=XZ37P6QATKZ5LOFHL74FVODPHKCN7PVI7KZB26KT6PFYCVVG7VNA, seg_pos={0 0}, upload=57265]: Uploading piece... Jul 26 23:18:22.654444024 (59h42m53.844742612s) [noise=true , address=77.90.31.16:4766 , piece_id=GX3CSCOQSKIX2I5RSHVVD3Q4CFFK6OJ7WZXQ32WEU4CJSIGYVLOA, seg_pos={0 0}, upload=57265]: Uploading piece... Jul 26 23:18:22.655012322 (59h42m53.845310903s) [noise=true , address=79.127.205.246:4284 , piece_id=2MDFJAN4STW2IWQ7LXLPLUEENS5EURP5HPZ46I6SCQ5F5EIAIFEQ, seg_pos={0 0}, upload=57265]: Uploading piece... Jul 26 23:18:22.655561135 (59h42m53.845859717s) [noise=true , address=104.36.82.54:4175 , piece_id=O73MBOO6TJL53M3G2ZOTPEYAPNIL7LOQI6TFHLGLWYTAXFCJ3T4A, seg_pos={0 0}, upload=57265]: Uploading piece... Jul 26 23:18:22.656170816 (59h42m53.846469405s) [noise=true , address=79.127.205.235:4685 , piece_id=4TYXR5RG5EHGNPQ2QGEX2ZB66AIVXLN3QUM552F3ZW3PO2OTNZHQ, seg_pos={0 0}, upload=57265]: Uploading piece... Jul 26 23:18:22.658503574 (59h42m53.848802158s) [noise=true , address=104.36.82.47:4540 , piece_id=5JFVP3E5U6IXTX7VVLYVOKRDFUJR5FWNP3GUVTV3GVB4ELWBUZ5A, seg_pos={0 0}, upload=57265]: Uploading piece... Jul 26 23:18:22.659032750 (59h42m53.849331334s) [noise=true , address=104.36.82.52:4102 , piece_id=NF7HNQRYH6I4YIFVRQEYMQYMEELHHAXMVDFLPLIBYXZ2WZCBM4TA, seg_pos={0 0}, upload=57265]: Uploading piece... Jul 26 23:18:22.659503887 (59h42m53.849802468s) [noise=true , address=64.49.13.140:4508 , piece_id=JAOPGACQT6XOP33FBY47QVD4YN3AFPGP6WZQEW2JMSFVDUFTWELA, seg_pos={0 0}, upload=57265]: Uploading piece... Jul 26 23:18:22.659978188 (59h42m53.85027677s) [noise=true , address=45.91.116.14:4194 , piece_id=4F2RNWI4EXD2WAUK2KGOOKATBPW73L6ZMGAAPCOQN332CRC4BH6Q, seg_pos={0 0}, upload=57265]: Uploading piece... Jul 26 23:18:22.660768731 (59h42m53.851067322s) [noise=true , address=104.36.82.55:4193 , piece_id=DTG5DK5BY6SJK35SUAJDZQISBEGO6OD6DRJ7NEW4T6V2QV75HM6A, seg_pos={0 0}, upload=57265]: Uploading piece... Jul 26 23:18:23.031753425 (59h42m54.222052007s) [noise=true , address=77.90.31.19:4871 , piece_id=5TSWLPP4QY5XG3YESQ62NRGQMRAFB57LTJNE5M3J36AXRSV74ESQ, seg_pos={0 0}, upload=57265]: Done uploading piece. err: ecclient: upload failed (node:12hrM59APBvzYUrwWMvSUQVMSkzNPxaNFDWEZn2J96xYTdv4Ezm, address:77.90.31.19:4871): protocol: storage node overloaded, request limit: 2000 Jul 26 23:18:23.211667805 (59h42m54.401966374s) [upload=57265]: Next returned nil segment Jul 26 23:18:23.212107349 (59h42m54.402405937s) [upload=57265]: Waiting for error group managing segments... Jul 26 23:18:23.424872010 (59h42m54.615170598s) [noise=true , address=104.36.82.54:4189 , piece_id=UNHERSX42IHQB2ZK62ENVCWO7YZQOX7KUUDXDK5DC7O3IJV4J3DA, seg_pos={0 0}, upload=57265]: Done uploading piece. err: <nil> Jul 26 23:18:23.445850643 (59h42m54.636149216s) [noise=true , address=77.90.31.13:4649 , piece_id=RPSRAX5ZJPNXKKBXYU52PSELLAALXHHB7WPL2XI3RHGMNWTEQ4BA, seg_pos={0 0}, upload=57265]: Done uploading piece. err: <nil> Jul 26 23:18:23.470857188 (59h42m54.661155761s) [noise=true , address=104.36.82.54:4175 , piece_id=O73MBOO6TJL53M3G2ZOTPEYAPNIL7LOQI6TFHLGLWYTAXFCJ3T4A, seg_pos={0 0}, upload=57265]: Done uploading piece. err: <nil> Jul 26 23:18:23.477270905 (59h42m54.667569478s) [noise=true , address=79.127.219.38:4007 , piece_id=WUOWIXVQ2YFFIBDP47MBOQ3C6KS246SGKHLWTCIBWLSPVEN5QMHQ, seg_pos={0 0}, upload=57265]: Done uploading piece. err: <nil> Jul 26 23:18:23.478446022 (59h42m54.668744594s) [noise=true , address=45.91.116.12:4230 , piece_id=7OGTVSKUHS7RZZMZWQ5YEMIEVGG63EDZ7SGOK4LZBIHS3YHZZ5VA, seg_pos={0 0}, upload=57265]: Done uploading piece. err: <nil> Jul 26 23:18:24.060304456 (59h42m55.250603038s) [noise=true , address=79.127.205.235:4685 , piece_id=4TYXR5RG5EHGNPQ2QGEX2ZB66AIVXLN3QUM552F3ZW3PO2OTNZHQ, seg_pos={0 0}, upload=57265]: Done uploading piece. err: <nil> Jul 26 23:18:24.604993482 (59h42m55.795292062s) [noise=true , address=45.91.116.11:4124 , piece_id=Z3TSCRITM7BDFXEKPXT57KSFGFDQK4GLBV4FDZQLNE55GTPVOPPA, seg_pos={0 0}, upload=57265]: Done uploading piece. err: <nil> Jul 26 23:18:24.670305175 (59h42m55.86060375s) [noise=true , address=79.127.205.246:4284 , piece_id=2MDFJAN4STW2IWQ7LXLPLUEENS5EURP5HPZ46I6SCQ5F5EIAIFEQ, seg_pos={0 0}, upload=57265]: Done uploading piece. err: <nil> Jul 26 23:18:26.395104683 (59h42m57.585403278s) [noise=true , address=45.91.116.14:4194 , piece_id=4F2RNWI4EXD2WAUK2KGOOKATBPW73L6ZMGAAPCOQN332CRC4BH6Q, seg_pos={0 0}, upload=57265]: Done uploading piece. err: ecclient: upload failed (node:19mdk9cXDR79ZXC4n6rMXFUhpwxMPXFpkPA14ymY24RK1jKkmd, address:45.91.116.14:4194): protocol: storage node overloaded, request limit: 2000 Jul 26 23:18:29.203901266 (59h43m0.394199847s) [noise=true , address=64.49.13.140:4508 , piece_id=JAOPGACQT6XOP33FBY47QVD4YN3AFPGP6WZQEW2JMSFVDUFTWELA, seg_pos={0 0}, upload=57265]: Done uploading piece. err: <nil> Jul 26 23:18:43.998605160 (59h43m15.188903732s) [noise=true , address=79.127.201.215:4347 , piece_id=OL46BT6WG3AVUCDTGNBZWOH2LIU6IF73DL3YQE23COTFXUEGR5VA, seg_pos={0 0}, upload=57265]: Done uploading piece. err: <nil> Jul 26 23:18:52.663055694 (59h43m23.85335428s) [noise=true , address=104.36.82.52:4102 , piece_id=NF7HNQRYH6I4YIFVRQEYMQYMEELHHAXMVDFLPLIBYXZ2WZCBM4TA, seg_pos={0 0}, upload=57265]: Done uploading piece. err: <nil> Jul 26 23:18:56.673687719 (59h43m27.863986305s) [noise=true , address=104.36.82.53:4125 , piece_id=7L5HHC7SVXVMOJO4WCUGRMHTMJWEL6MEJ2G4KSR3OOEVIGV2CPQA, seg_pos={0 0}, upload=57265]: Done uploading piece. err: <nil> Jul 26 23:19:11.141853102 (59h43m42.332151671s) [noise=true , address=104.36.82.55:4193 , piece_id=DTG5DK5BY6SJK35SUAJDZQISBEGO6OD6DRJ7NEW4T6V2QV75HM6A, seg_pos={0 0}, upload=57265]: Done uploading piece. err: <nil> Jul 26 23:19:22.169795332 (59h43m53.360093911s) [noise=true , address=77.90.31.16:4766 , piece_id=GX3CSCOQSKIX2I5RSHVVD3Q4CFFK6OJ7WZXQ32WEU4CJSIGYVLOA, seg_pos={0 0}, upload=57265]: Done uploading piece. err: ecclient: upload failed (node:1V8iMvJLcTddWZihWQBQLLCyMTwbD2h7ruDUjMEtHD7EHtNVEK, address:77.90.31.16:4766): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 23:19:22.169827224 (59h43m53.360125792s) [noise=true , address=104.36.82.46:4497 , piece_id=XZ37P6QATKZ5LOFHL74FVODPHKCN7PVI7KZB26KT6PFYCVVG7VNA, seg_pos={0 0}, upload=57265]: Done uploading piece. err: ecclient: upload failed (node:12Zg4PSYBuLxFvF9oYiuvn65JNbjT5RdagcVAQLp2izvjPe1NEc, address:104.36.82.46:4497): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 23:19:22.169856986 (59h43m53.360155565s) [noise=true , address=104.36.82.47:4540 , piece_id=5JFVP3E5U6IXTX7VVLYVOKRDFUJR5FWNP3GUVTV3GVB4ELWBUZ5A, seg_pos={0 0}, upload=57265]: Done uploading piece. err: ecclient: upload failed (node:1P92CRZqnx1avdDo18tMY2X7FujkZugb4gU98AaErD13gX4cLA, address:104.36.82.47:4540): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 23:19:22.169883296 (59h43m53.360181881s) [seg_pos={0 0}, upload=57265]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 26 23:19:22.280227166 (59h43m53.47052574s) [upload=57265]: Done uploading segments... Jul 26 23:19:22.280253655 (59h43m53.470552234s) [upload=57265]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1m0.112248456s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 15 out of 60 is failed (threshold=3) [.XX...X..X.XX......X.X.......X.......X........XX....X....X.X] duration: 1m0.017641477s FAIL (2025-07-26 23:19:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.017664175s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [.....................................X......XXXXXXXXXXXXXXXX] duration: 30.110834648s FAIL (2025-07-26 23:19:11 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== _pos={0 0}, upload=57270]: Uploading piece... Jul 26 23:18:42.002597239 (59h43m13.192895818s) [noise=true , address=104.36.82.2:4730 , piece_id=XXL2YGQUAKMEIYZFHJHBDQ5JV4W5ZUOPARLRXW2AXQ5AESV4DB3Q, seg_pos={0 0}, upload=57270]: Uploading piece... Jul 26 23:18:42.002975900 (59h43m13.19327448s) [noise=true , address=104.36.82.7:4912 , piece_id=DTG27IQ4QUWU7K5RGQ34BUFZKF7QHG25BXSUNUBIWO3XTYO6BTAQ, seg_pos={0 0}, upload=57270]: Uploading piece... Jul 26 23:18:42.003271060 (59h43m13.19356963s) [noise=true , address=104.36.82.10:4078 , piece_id=VB6VLRAM3NL57TK3VKD33PYPHK6XGJM6ZYNNCMYVJBGLXGQBD7GA, seg_pos={0 0}, upload=57270]: Uploading piece... Jul 26 23:18:42.003571069 (59h43m13.193869648s) [noise=true , address=104.36.82.5:4838 , piece_id=MXK6VB6LTHZQFH4XZG7NFFPK55DWE55VMMR23VWYISTWCOPRV2YA, seg_pos={0 0}, upload=57270]: Uploading piece... Jul 26 23:18:42.003850777 (59h43m13.194149356s) [noise=true , address=104.36.82.2:4729 , piece_id=3ZCPVF4DOZYCOYBDYC52UFRSXO7R25DP4INAX6PMYWBOSEGBOVXA, seg_pos={0 0}, upload=57270]: Uploading piece... Jul 26 23:18:42.004180690 (59h43m13.194479291s) [noise=true , address=104.36.82.6:4874 , piece_id=OAG6DOQJIHRREZUKCJA6XOWT5CWCWT4SHOJYZENE5A5PQ7ICYGXQ, seg_pos={0 0}, upload=57270]: Uploading piece... Jul 26 23:18:42.004730093 (59h43m13.195028693s) [noise=true , address=77.90.31.17:4771 , piece_id=DDLWSTBL2DAZCL6IFBNLQEMTIT6H2XGLPLLOWX4JKXYYUEKLABCA, seg_pos={0 0}, upload=57270]: Uploading piece... Jul 26 23:18:42.655785456 (59h43m13.846084027s) [upload=57270]: Next returned nil segment Jul 26 23:18:42.655804538 (59h43m13.846103106s) [upload=57270]: Waiting for error group managing segments... Jul 26 23:18:42.766418068 (59h43m13.956716654s) [noise=true , address=45.91.116.6:4974 , piece_id=JG7OBBLP54JVYM2ZM654IVKIZAWR2NXX7QSD4ZJUPUUJQMOY7AAQ, seg_pos={0 0}, upload=57270]: Done uploading piece. err: <nil> Jul 26 23:19:06.711043072 (59h43m37.901341662s) [noise=true , address=104.36.82.2:4729 , piece_id=3ZCPVF4DOZYCOYBDYC52UFRSXO7R25DP4INAX6PMYWBOSEGBOVXA, seg_pos={0 0}, upload=57270]: Done uploading piece. err: <nil> Jul 26 23:19:07.718035375 (59h43m38.908333959s) [noise=true , address=104.36.82.2:4730 , piece_id=XXL2YGQUAKMEIYZFHJHBDQ5JV4W5ZUOPARLRXW2AXQ5AESV4DB3Q, seg_pos={0 0}, upload=57270]: Done uploading piece. err: <nil> Jul 26 23:19:11.613478619 (59h43m42.803777185s) [noise=true , address=104.36.82.7:4912 , piece_id=DTG27IQ4QUWU7K5RGQ34BUFZKF7QHG25BXSUNUBIWO3XTYO6BTAQ, seg_pos={0 0}, upload=57270]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:104.36.82.7:4912): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 23:19:11.613538144 (59h43m42.803836736s) [noise=true , address=104.36.82.8:4972 , piece_id=PMVNGQN6A6B4ITSCSNJKWVY35PZEBGRYBXAMZIKSQAFGUMFXXATQ, seg_pos={0 0}, upload=57270]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 23:19:11.613581663 (59h43m42.803880229s) [noise=true , address=104.36.82.10:4078 , piece_id=VB6VLRAM3NL57TK3VKD33PYPHK6XGJM6ZYNNCMYVJBGLXGQBD7GA, seg_pos={0 0}, upload=57270]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: expected piece hash; write tcp 139.59.144.22:55422->104.36.82.10:4078: use of closed network connection; write tcp 139.59.144.22:55422->104.36.82.10:4078: use of closed network connection; EOF Jul 26 23:19:11.614016291 (59h43m42.804314874s) [noise=true , address=104.36.82.14:4223 , piece_id=BY2TXIBGWDK6N2NBYNMUQ4Y5AE35YY7L7JNGCEWY765LGFKIVZFA, seg_pos={0 0}, upload=57270]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 139.59.144.22:35350->104.36.82.14:4223: use of closed network connection; write tcp 139.59.144.22:35350->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35350->104.36.82.14:4223: use of closed network connection Jul 26 23:19:11.614045030 (59h43m42.804343598s) [noise=true , address=104.36.82.13:4186 , piece_id=BBHBAOIGT64CHJKU5EJYNLHTLAXNPDHVUC4I4PJNG4LZMYIQ4LJQ, seg_pos={0 0}, upload=57270]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 139.59.144.22:50860->104.36.82.13:4186: use of closed network connection; write tcp 139.59.144.22:50860->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50860->104.36.82.13:4186: use of closed network connection Jul 26 23:19:11.614139536 (59h43m42.804438105s) [noise=true , address=104.36.82.5:4838 , piece_id=MXK6VB6LTHZQFH4XZG7NFFPK55DWE55VMMR23VWYISTWCOPRV2YA, seg_pos={0 0}, upload=57270]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: write tcp 139.59.144.22:56764->104.36.82.5:4838: use of closed network connection; write tcp 139.59.144.22:56764->104.36.82.5:4838: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:56764->104.36.82.5:4838: use of closed network connection Jul 26 23:19:11.614198533 (59h43m42.804497112s) [noise=true , address=77.90.31.17:4771 , piece_id=DDLWSTBL2DAZCL6IFBNLQEMTIT6H2XGLPLLOWX4JKXYYUEKLABCA, seg_pos={0 0}, upload=57270]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:47924->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 23:19:11.614221178 (59h43m42.804519751s) [noise=true , address=104.36.82.6:4876 , piece_id=5IAMO2I2ZMOV3JYY7RPCLVQTFUGTUGZNWLGK6ZD3U7BEMFSEPI5Q, seg_pos={0 0}, upload=57270]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: write tcp 139.59.144.22:36596->104.36.82.6:4876: use of closed network connection; write tcp 139.59.144.22:36596->104.36.82.6:4876: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36596->104.36.82.6:4876: use of closed network connection Jul 26 23:19:11.614263691 (59h43m42.804562271s) [noise=true , address=104.36.82.4:4801 , piece_id=LC6VEKVM4DET73YUUPXEA5HLCWMVPZF6DKKTHUOV33ZY7P3TK7PQ, seg_pos={0 0}, upload=57270]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: write tcp 139.59.144.22:56232->104.36.82.4:4801: use of closed network connection; write tcp 139.59.144.22:56232->104.36.82.4:4801: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:56232->104.36.82.4:4801: use of closed network connection Jul 26 23:19:11.614300623 (59h43m42.804599192s) [noise=true , address=104.36.82.14:4222 , piece_id=W2JKJBWTBKM52DZ7FROJ67MOPCOPQFJ7TIO3BE7D3KPDRNN7QAEQ, seg_pos={0 0}, upload=57270]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: write tcp 139.59.144.22:36534->104.36.82.14:4222: use of closed network connection; write tcp 139.59.144.22:36534->104.36.82.14:4222: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36534->104.36.82.14:4222: use of closed network connection Jul 26 23:19:11.614319145 (59h43m42.804617723s) [noise=true , address=104.36.82.7:4911 , piece_id=ILUI6HHRCFOBTRVTE27NZQQ7XJCOUIJX7GXUQ5BTLDFW3J7RCOZQ, seg_pos={0 0}, upload=57270]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:104.36.82.7:4911): protocol: write tcp 139.59.144.22:38956->104.36.82.7:4911: use of closed network connection; write tcp 139.59.144.22:38956->104.36.82.7:4911: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38956->104.36.82.7:4911: use of closed network connection Jul 26 23:19:11.614339633 (59h43m42.804638204s) [noise=true , address=104.36.82.6:4874 , piece_id=OAG6DOQJIHRREZUKCJA6XOWT5CWCWT4SHOJYZENE5A5PQ7ICYGXQ, seg_pos={0 0}, upload=57270]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: write tcp 139.59.144.22:56794->104.36.82.6:4874: use of closed network connection; write tcp 139.59.144.22:56794->104.36.82.6:4874: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:56794->104.36.82.6:4874: use of closed network connection Jul 26 23:19:11.614358040 (59h43m42.80465662s) [noise=true , address=104.36.82.15:4257 , piece_id=5RK4VN3QYO73N3TGNTIYJ3XYMPSWHE4NKEKTZQXA7TVPTQVGFWYA, seg_pos={0 0}, upload=57270]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: write tcp 139.59.144.22:44418->104.36.82.15:4257: use of closed network connection; write tcp 139.59.144.22:44418->104.36.82.15:4257: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44418->104.36.82.15:4257: use of closed network connection Jul 26 23:19:11.614402614 (59h43m42.804701196s) [seg_pos={0 0}, upload=57270]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Jul 26 23:19:11.722873834 (59h43m42.913172414s) [upload=57270]: Done uploading segments... Jul 26 23:19:11.722907878 (59h43m42.913206458s) [upload=57270]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 30.110891s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [............................................X.XX...XX..X..XX] duration: 30.001058063s FAIL (2025-07-26 23:19:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001080555s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 19 out of 60 is failed (threshold=3) [.................X..................X.....XXXXXX.XXXXXXXXXXX] duration: 30.213723282s FAIL (2025-07-26 23:18:40 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 8) Duration: 30.213752515s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [.............................................X.XX...XX..X..X] duration: 30.002165779s FAIL (2025-07-26 23:18:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.002194801s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 42 out of 60 is failed (threshold=3) [XXXX.XXXX.XXXXX..XXX.....XXX.X.X.XX.X.XXXXXXXXX.XX..X.XXXXXX] duration: 1m0.105999262s FAIL (2025-07-26 23:18:22 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 376507445s) [upload=57251]: Starting upload Jul 26 23:17:22.186301588 (59h41m53.376600156s) [upload=57251]: Done starting upload Jul 26 23:17:22.186400906 (59h41m53.376699492s) [upload=57251]: Uploading segments... Jul 26 23:17:22.186660875 (59h41m53.376959452s) [upload=57251]: Got next segment. Inline: false Jul 26 23:17:22.186721660 (59h41m53.377020246s) [upload=57251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 23:17:22.186731838 (59h41m53.377030429s) [upload=57251]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 23:17:22.378365583 (59h41m53.568664162s) [seg_pos={0 0}, upload=57251]: Begin upload segment... Jul 26 23:17:22.378460370 (59h41m53.568758954s) [seg_pos={0 0}, upload=57251]: Done begin upload segment. Jul 26 23:17:22.378735682 (59h41m53.569034255s) [noise=true , address=104.36.82.47:4521 , piece_id=XEZRIJIJJF4OYXABOXIHQLKFY6SZHUFUEM5BQUSDOXQ2EQH2JNGQ, seg_pos={0 0}, upload=57251]: Uploading piece... Jul 26 23:17:22.378853733 (59h41m53.569152305s) [noise=true , address=104.36.82.46:4497 , piece_id=Q5MXFKGZPBHAHP7OG7ERSB7ZXF74CJE2ZWODRAK5ZSIR6EEHSTYA, seg_pos={0 0}, upload=57251]: Uploading piece... Jul 26 23:17:22.379247188 (59h41m53.56954576s) [noise=true , address=79.127.205.235:4680 , piece_id=Y4PRQRWNVWDDCWG2PYEUVUEK3C7VPCTHRNKRSG3J6TLEILUUABWQ, seg_pos={0 0}, upload=57251]: Uploading piece... Jul 26 23:17:22.381553719 (59h41m53.571852292s) [noise=true , address=45.91.116.5:4310 , piece_id=JK5TFR6H3APUKZ4KJEHRJAAD4POHAAMPWOHCH673TJNUDDOGYA5Q, seg_pos={0 0}, upload=57251]: Uploading piece... Jul 26 23:17:22.382741096 (59h41m53.573039667s) [noise=true , address=79.127.205.234:4619 , piece_id=KXVGJYMGQO2UYLXN6DJZDRF4BYH3OFQCBQFP2FTBG45GLAVMLMLQ, seg_pos={0 0}, upload=57251]: Uploading piece... Jul 26 23:17:22.384490877 (59h41m53.574789451s) [noise=true , address=64.49.13.170:4718 , piece_id=N2MVM5QHMPRPSU3YYRZOO3B35MSA2CXRDMFO3NGWH7FUF6THFWBQ, seg_pos={0 0}, upload=57251]: Uploading piece... Jul 26 23:17:22.385418452 (59h41m53.575717024s) [noise=true , address=79.127.201.215:4353 , piece_id=5RBVYYUHXF37X5F2RLKJNVLFCII3NYZ3PTH62OKWDDW6MILRMWSA, seg_pos={0 0}, upload=57251]: Uploading piece... Jul 26 23:17:22.387413869 (59h41m53.577712436s) [noise=true , address=104.36.82.54:4180 , piece_id=D4JXW4NF3DZYR3NTW6YEC2CI5VXRZQB5RBNDNMLMRUXRVYQTINUA, seg_pos={0 0}, upload=57251]: Uploading piece... Jul 26 23:17:22.389385157 (59h41m53.579683722s) [noise=true , address=104.36.82.4:4826 , piece_id=ZUAHYFX5WC7F6QSOWV6LVDFAHYSHA2RGIVXJEF37BT2HBPHLEV2A, seg_pos={0 0}, upload=57251]: Uploading piece... Jul 26 23:17:22.391405526 (59h41m53.581704099s) [noise=true , address=104.36.82.10:4101 , piece_id=OK6UDWMDKZO7XSTXF2RXLAMZWZK5CPQMQB2UB7P6BTO3YHL2HTJA, seg_pos={0 0}, upload=57251]: Uploading piece... Jul 26 23:17:22.393291993 (59h41m53.583590559s) [noise=true , address=104.36.82.52:4116 , piece_id=GOHUBS4T26OIABBJO6KA5HZPR4QO7T6LC5L3B7SCWZ6MCENP53TQ, seg_pos={0 0}, upload=57251]: Uploading piece... Jul 26 23:17:22.395136133 (59h41m53.585434707s) [noise=true , address=79.127.219.38:4004 , piece_id=YP4NG6V5P3BV3VCGTMO4G7GINKFX6LSV2UFVSQSLJ2WEDFUEU5MA, seg_pos={0 0}, upload=57251]: Uploading piece... Jul 26 23:17:22.395465661 (59h41m53.585764231s) [noise=true , address=104.36.82.11:4128 , piece_id=MBKA5APSQOTUIQMTRGUQ6LC3OHCZSIADHGP5QBUHKNY6F3GHKWUA, seg_pos={0 0}, upload=57251]: Uploading piece... Jul 26 23:17:22.396322491 (59h41m53.58662107s) [noise=true , address=104.36.82.53:4130 , piece_id=OGSZ3NCC4VJYQTW3F7OMRLZKZQLADR5EDGY4ESBMKXP46F6BABXQ, seg_pos={0 0}, upload=57251]: Uploading piece... Jul 26 23:17:22.396902651 (59h41m53.587201222s) [noise=true , address=45.91.116.11:4121 , piece_id=27JAR34Q2OA7NQHOXT6IKY4RU45FYZDEACIC4W5XCSQXTRPQSOJA, seg_pos={0 0}, upload=57251]: Uploading piece... Jul 26 23:17:22.398842363 (59h41m53.589140934s) [noise=true , address=104.36.82.55:4207 , piece_id=UEHKBYN6AXOSKZOC5DRAQ4VJSBYHLPTB7SC7SOCXRVVCLJRWTMIQ, seg_pos={0 0}, upload=57251]: Uploading piece... Jul 26 23:17:22.399175750 (59h41m53.589474329s) [noise=true , address=104.36.82.56:4262 , piece_id=5LFAB3TP3ANZNV3HB4DUD75MW54BAUCOLVUB4T4XXV7BTDBQPAVA, seg_pos={0 0}, upload=57251]: Uploading piece... Jul 26 23:17:22.399707075 (59h41m53.59000564s) [noise=true , address=77.90.31.16:4748 , piece_id=76RRNCPUFZY4MBVD25FO4JUBDNLOPT4RGQJ7IS5JJZUR26A27BEQ, seg_pos={0 0}, upload=57251]: Uploading piece... Jul 26 23:17:22.974112244 (59h41m54.164410816s) [upload=57251]: Next returned nil segment Jul 26 23:17:22.974144723 (59h41m54.164443305s) [upload=57251]: Waiting for error group managing segments... Jul 26 23:17:23.192559813 (59h41m54.38285843s) [noise=true , address=79.127.219.38:4004 , piece_id=YP4NG6V5P3BV3VCGTMO4G7GINKFX6LSV2UFVSQSLJ2WEDFUEU5MA, seg_pos={0 0}, upload=57251]: Done uploading piece. err: <nil> Jul 26 23:17:23.232723880 (59h41m54.423022449s) [noise=true , address=77.90.31.16:4748 , piece_id=76RRNCPUFZY4MBVD25FO4JUBDNLOPT4RGQJ7IS5JJZUR26A27BEQ, seg_pos={0 0}, upload=57251]: Done uploading piece. err: <nil> Jul 26 23:17:23.642400697 (59h41m54.832699283s) [noise=true , address=79.127.205.234:4619 , piece_id=KXVGJYMGQO2UYLXN6DJZDRF4BYH3OFQCBQFP2FTBG45GLAVMLMLQ, seg_pos={0 0}, upload=57251]: Done uploading piece. err: <nil> Jul 26 23:17:23.697671602 (59h41m54.88797017s) [noise=true , address=79.127.205.235:4680 , piece_id=Y4PRQRWNVWDDCWG2PYEUVUEK3C7VPCTHRNKRSG3J6TLEILUUABWQ, seg_pos={0 0}, upload=57251]: Done uploading piece. err: <nil> Jul 26 23:17:23.925816794 (59h41m55.116115454s) [noise=true , address=45.91.116.11:4121 , piece_id=27JAR34Q2OA7NQHOXT6IKY4RU45FYZDEACIC4W5XCSQXTRPQSOJA, seg_pos={0 0}, upload=57251]: Done uploading piece. err: ecclient: upload failed (node:1wkujwGr7G6yGYFLiYcxfBnN1pmKY6mHXoAzZWAjGCCiWWZNdh, address:45.91.116.11:4121): protocol: storage node overloaded, request limit: 2000 Jul 26 23:17:24.194742843 (59h41m55.385041423s) [noise=true , address=45.91.116.5:4310 , piece_id=JK5TFR6H3APUKZ4KJEHRJAAD4POHAAMPWOHCH673TJNUDDOGYA5Q, seg_pos={0 0}, upload=57251]: Done uploading piece. err: ecclient: upload failed (node:127RMUc62xRq9RU5z3XfkZsq5sehmv3kP8o53tr9BxVcPE3GQCY, address:45.91.116.5:4310): protocol: storage node overloaded, request limit: 2000 Jul 26 23:17:31.485172279 (59h42m2.675470859s) [noise=true , address=64.49.13.170:4718 , piece_id=N2MVM5QHMPRPSU3YYRZOO3B35MSA2CXRDMFO3NGWH7FUF6THFWBQ, seg_pos={0 0}, upload=57251]: Done uploading piece. err: <nil> Jul 26 23:17:39.719997999 (59h42m10.910296576s) [noise=true , address=104.36.82.54:4180 , piece_id=D4JXW4NF3DZYR3NTW6YEC2CI5VXRZQB5RBNDNMLMRUXRVYQTINUA, seg_pos={0 0}, upload=57251]: Done uploading piece. err: <nil> Jul 26 23:17:40.538069255 (59h42m11.728367837s) [noise=true , address=104.36.82.52:4116 , piece_id=GOHUBS4T26OIABBJO6KA5HZPR4QO7T6LC5L3B7SCWZ6MCENP53TQ, seg_pos={0 0}, upload=57251]: Done uploading piece. err: <nil> Jul 26 23:17:45.115662528 (59h42m16.305961094s) [noise=true , address=104.36.82.56:4262 , piece_id=5LFAB3TP3ANZNV3HB4DUD75MW54BAUCOLVUB4T4XXV7BTDBQPAVA, seg_pos={0 0}, upload=57251]: Done uploading piece. err: <nil> Jul 26 23:17:51.776254371 (59h42m22.966552942s) [noise=true , address=104.36.82.53:4130 , piece_id=OGSZ3NCC4VJYQTW3F7OMRLZKZQLADR5EDGY4ESBMKXP46F6BABXQ, seg_pos={0 0}, upload=57251]: Done uploading piece. err: <nil> Jul 26 23:17:58.728041060 (59h42m29.918339638s) [noise=true , address=104.36.82.46:4497 , piece_id=Q5MXFKGZPBHAHP7OG7ERSB7ZXF74CJE2ZWODRAK5ZSIR6EEHSTYA, seg_pos={0 0}, upload=57251]: Done uploading piece. err: <nil> Jul 26 23:18:01.661267704 (59h42m32.851566282s) [noise=true , address=79.127.201.215:4353 , piece_id=5RBVYYUHXF37X5F2RLKJNVLFCII3NYZ3PTH62OKWDDW6MILRMWSA, seg_pos={0 0}, upload=57251]: Done uploading piece. err: <nil> Jul 26 23:18:04.078428359 (59h42m35.268726937s) [noise=true , address=104.36.82.10:4101 , piece_id=OK6UDWMDKZO7XSTXF2RXLAMZWZK5CPQMQB2UB7P6BTO3YHL2HTJA, seg_pos={0 0}, upload=57251]: Done uploading piece. err: <nil> Jul 26 23:18:11.955098636 (59h42m43.145397209s) [noise=true , address=104.36.82.55:4207 , piece_id=UEHKBYN6AXOSKZOC5DRAQ4VJSBYHLPTB7SC7SOCXRVVCLJRWTMIQ, seg_pos={0 0}, upload=57251]: Done uploading piece. err: <nil> Jul 26 23:18:21.920737919 (59h42m53.11103649s) [noise=true , address=104.36.82.47:4521 , piece_id=XEZRIJIJJF4OYXABOXIHQLKFY6SZHUFUEM5BQUSDOXQ2EQH2JNGQ, seg_pos={0 0}, upload=57251]: Done uploading piece. err: ecclient: upload failed (node:12GQHySYUuqERoQyP4fCib2nRMNJhgpP2NkDzVxz6nurLkeJ5VD, address:104.36.82.47:4521): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 23:18:21.920770620 (59h42m53.111069189s) [noise=true , address=104.36.82.11:4128 , piece_id=MBKA5APSQOTUIQMTRGUQ6LC3OHCZSIADHGP5QBUHKNY6F3GHKWUA, seg_pos={0 0}, upload=57251]: Done uploading piece. err: ecclient: upload failed (node:1wABkAtAjoYkKyNoAiaVCxSvY6WgvHvAYFPCjpNojtpiYiGtDS, address:104.36.82.11:4128): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 23:18:21.920819954 (59h42m53.111118532s) [noise=true , address=104.36.82.4:4826 , piece_id=ZUAHYFX5WC7F6QSOWV6LVDFAHYSHA2RGIVXJEF37BT2HBPHLEV2A, seg_pos={0 0}, upload=57251]: Done uploading piece. err: ecclient: upload failed (node:14hpQXA7AA3CQQ4z8Y81ZTxjY9Bhd1eNCjqEKyandGUHoyi548, address:104.36.82.4:4826): protocol: write tcp 139.59.144.22:35976->104.36.82.4:4826: use of closed network connection; write tcp 139.59.144.22:35976->104.36.82.4:4826: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35976->104.36.82.4:4826: use of closed network connection Jul 26 23:18:21.920838244 (59h42m53.111136826s) [seg_pos={0 0}, upload=57251]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 26 23:18:22.024622345 (59h42m53.214920933s) [upload=57251]: Done uploading segments... Jul 26 23:18:22.024661646 (59h42m53.21496023s) [upload=57251]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1m0.106015354s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 15 out of 60 is failed (threshold=3) [X.XX...X..X.XX......X.X.......X.......X........XX....X....X.] duration: 51.39935729s FAIL (2025-07-26 23:18:20 UTC) orbiter warning: context deadline exceeded Duration: 51.399387929s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [......................................X......XXXXXXXXXXXXXXX] duration: 30.108456111s FAIL (2025-07-26 23:17:59 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== item: pb.BatchRequestItem_ObjectBegin Jul 26 23:17:29.517561177 (59h42m0.707859756s) [upload=57255]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 23:17:29.623193717 (59h42m0.813492306s) [seg_pos={0 0}, upload=57255]: Begin upload segment... Jul 26 23:17:29.623264491 (59h42m0.813563061s) [seg_pos={0 0}, upload=57255]: Done begin upload segment. Jul 26 23:17:29.623291283 (59h42m0.813589873s) [noise=true , address=104.36.82.14:4222 , piece_id=DXNNVTI5M2KMH4KRAJ3SDIA2V27RIL5AIOW2P4Q6XBH6T4HEOAXQ, seg_pos={0 0}, upload=57255]: Uploading piece... Jul 26 23:17:29.623307151 (59h42m0.81360574s) [noise=true , address=104.36.82.5:4839 , piece_id=MIALNKXBMRLLB45IBJFPPVSWZNVRUGRKJJWDDXWB5OIZTRB76EUA, seg_pos={0 0}, upload=57255]: Uploading piece... Jul 26 23:17:29.623757575 (59h42m0.814056168s) [noise=true , address=104.36.82.15:4259 , piece_id=JDOQZAVFJZ2B4BD7M5C4CWIFPQSDECUEX3KYC5PCOPTJTQSNRXAA, seg_pos={0 0}, upload=57255]: Uploading piece... Jul 26 23:17:29.623894907 (59h42m0.814193525s) [noise=true , address=104.36.82.4:4801 , piece_id=YIADM633C3P63YMNFQDVYNWDKUFPD6KKGAJNPP7UEYJQUYOJTX6A, seg_pos={0 0}, upload=57255]: Uploading piece... Jul 26 23:17:29.624133335 (59h42m0.814431922s) [noise=true , address=104.36.82.7:4909 , piece_id=AE7UKGF5AHCEFUQUV2SUKO2VNMGQJWQERGFZVUGEVWO2MNNMOHDQ, seg_pos={0 0}, upload=57255]: Uploading piece... Jul 26 23:17:29.624572948 (59h42m0.814871538s) [noise=true , address=77.90.31.21:4877 , piece_id=DYP36MM2O6MHKOCKZK6SPYFFOLWUK5I6NRE57EX5D4WKHCHG6VEA, seg_pos={0 0}, upload=57255]: Uploading piece... Jul 26 23:17:29.625871120 (59h42m0.816169709s) [noise=true , address=104.36.82.10:4078 , piece_id=MGEXVXHJLNX7KS42JJ6AEH7WNNOZ4QATCZHNMK7YNZWABCZQ7EAQ, seg_pos={0 0}, upload=57255]: Uploading piece... Jul 26 23:17:29.626245676 (59h42m0.816544264s) [noise=true , address=77.90.31.14:4661 , piece_id=25N3NXMIXWBHQ2LQCZW4K6Q3ZF7ZL4GIH43FEALZZ5TPQ3IUEMVQ, seg_pos={0 0}, upload=57255]: Uploading piece... Jul 26 23:17:29.626608059 (59h42m0.816906643s) [noise=true , address=45.91.116.13:4155 , piece_id=N3ZFQZQDMDIOYHQKNMYNJAPPUIFL6SLBJDFFJMLLR76RJT5MLQBA, seg_pos={0 0}, upload=57255]: Uploading piece... Jul 26 23:17:29.626963572 (59h42m0.817262162s) [noise=true , address=45.91.116.5:4299 , piece_id=7SLJXZSTJJ3C37FMKOMOCF63C7BWP3Y7ONTCJVH4XNOE7YFCTXMQ, seg_pos={0 0}, upload=57255]: Uploading piece... Jul 26 23:17:29.627307898 (59h42m0.817606489s) [noise=true , address=104.36.82.4:4803 , piece_id=FI4TBAYQG6F44LOXIBY2PIGNRUI4DU775QFRLVZRBJRQYKKICISA, seg_pos={0 0}, upload=57255]: Uploading piece... Jul 26 23:17:29.627648633 (59h42m0.817947222s) [noise=true , address=104.36.82.16:4293 , piece_id=S32VULEEW7OXL7XS6L5YNQNDF5MTNBKD2GMAVI46PYMRIBKOPN3A, seg_pos={0 0}, upload=57255]: Uploading piece... Jul 26 23:17:29.628002584 (59h42m0.818301173s) [noise=true , address=104.36.82.6:4873 , piece_id=L3CKPTBLNHX67XBFNZPOJHKTNOXQZLT37CYMJCNFDO4MXCA5QEXA, seg_pos={0 0}, upload=57255]: Uploading piece... Jul 26 23:17:29.628398428 (59h42m0.818697017s) [noise=true , address=45.91.116.13:4154 , piece_id=QVPBP7DMW26L4CYV5RB7E7FQIHLUPL4CPL4OAIRXNRPPZNJGUHEA, seg_pos={0 0}, upload=57255]: Uploading piece... Jul 26 23:17:29.628769916 (59h42m0.819068502s) [noise=true , address=45.91.116.7:4010 , piece_id=P5ZU4XJU3IV2LUCHIAXHYJFQGAAFAUYMD6TUO2JKPFI2OIE3C7CQ, seg_pos={0 0}, upload=57255]: Uploading piece... Jul 26 23:17:29.629158246 (59h42m0.819456835s) [noise=true , address=104.36.82.15:4257 , piece_id=3KCYOZWCHEMQW4LX2PXUQW4TV4MMKWXXGCDUWEL5N2STKCCAOIPQ, seg_pos={0 0}, upload=57255]: Uploading piece... Jul 26 23:17:30.248520966 (59h42m1.438819558s) [upload=57255]: Next returned nil segment Jul 26 23:17:30.248545404 (59h42m1.438843994s) [upload=57255]: Waiting for error group managing segments... Jul 26 23:17:30.389154796 (59h42m1.579453366s) [noise=true , address=45.91.116.13:4154 , piece_id=QVPBP7DMW26L4CYV5RB7E7FQIHLUPL4CPL4OAIRXNRPPZNJGUHEA, seg_pos={0 0}, upload=57255]: Done uploading piece. err: <nil> Jul 26 23:17:30.394594948 (59h42m1.584893522s) [noise=true , address=45.91.116.5:4299 , piece_id=7SLJXZSTJJ3C37FMKOMOCF63C7BWP3Y7ONTCJVH4XNOE7YFCTXMQ, seg_pos={0 0}, upload=57255]: Done uploading piece. err: <nil> Jul 26 23:17:30.394859570 (59h42m1.585158142s) [noise=true , address=77.90.31.14:4661 , piece_id=25N3NXMIXWBHQ2LQCZW4K6Q3ZF7ZL4GIH43FEALZZ5TPQ3IUEMVQ, seg_pos={0 0}, upload=57255]: Done uploading piece. err: <nil> Jul 26 23:17:30.396704635 (59h42m1.587003208s) [noise=true , address=45.91.116.7:4010 , piece_id=P5ZU4XJU3IV2LUCHIAXHYJFQGAAFAUYMD6TUO2JKPFI2OIE3C7CQ, seg_pos={0 0}, upload=57255]: Done uploading piece. err: <nil> Jul 26 23:17:30.414641695 (59h42m1.604940278s) [noise=true , address=45.91.116.13:4155 , piece_id=N3ZFQZQDMDIOYHQKNMYNJAPPUIFL6SLBJDFFJMLLR76RJT5MLQBA, seg_pos={0 0}, upload=57255]: Done uploading piece. err: <nil> Jul 26 23:17:30.496066710 (59h42m1.686365291s) [noise=true , address=77.90.31.21:4877 , piece_id=DYP36MM2O6MHKOCKZK6SPYFFOLWUK5I6NRE57EX5D4WKHCHG6VEA, seg_pos={0 0}, upload=57255]: Done uploading piece. err: <nil> Jul 26 23:17:40.844903505 (59h42m12.035202074s) [noise=true , address=104.36.82.6:4873 , piece_id=L3CKPTBLNHX67XBFNZPOJHKTNOXQZLT37CYMJCNFDO4MXCA5QEXA, seg_pos={0 0}, upload=57255]: Done uploading piece. err: <nil> Jul 26 23:17:41.460766035 (59h42m12.651064615s) [noise=true , address=104.36.82.5:4839 , piece_id=MIALNKXBMRLLB45IBJFPPVSWZNVRUGRKJJWDDXWB5OIZTRB76EUA, seg_pos={0 0}, upload=57255]: Done uploading piece. err: <nil> Jul 26 23:17:52.233207373 (59h42m23.423505939s) [noise=true , address=104.36.82.14:4222 , piece_id=DXNNVTI5M2KMH4KRAJ3SDIA2V27RIL5AIOW2P4Q6XBH6T4HEOAXQ, seg_pos={0 0}, upload=57255]: Done uploading piece. err: <nil> Jul 26 23:17:59.250160872 (59h42m30.440459442s) [noise=true , address=104.36.82.15:4257 , piece_id=3KCYOZWCHEMQW4LX2PXUQW4TV4MMKWXXGCDUWEL5N2STKCCAOIPQ, seg_pos={0 0}, upload=57255]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 23:17:59.250448320 (59h42m30.440746914s) [noise=true , address=104.36.82.15:4259 , piece_id=JDOQZAVFJZ2B4BD7M5C4CWIFPQSDECUEX3KYC5PCOPTJTQSNRXAA, seg_pos={0 0}, upload=57255]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:104.36.82.15:4259): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 23:17:59.250517254 (59h42m30.440815833s) [noise=true , address=104.36.82.4:4803 , piece_id=FI4TBAYQG6F44LOXIBY2PIGNRUI4DU775QFRLVZRBJRQYKKICISA, seg_pos={0 0}, upload=57255]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: write tcp 139.59.144.22:33284->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:33284->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33284->104.36.82.4:4803: use of closed network connection Jul 26 23:17:59.250567085 (59h42m30.440865657s) [noise=true , address=104.36.82.10:4078 , piece_id=MGEXVXHJLNX7KS42JJ6AEH7WNNOZ4QATCZHNMK7YNZWABCZQ7EAQ, seg_pos={0 0}, upload=57255]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: write tcp 139.59.144.22:47100->104.36.82.10:4078: use of closed network connection; write tcp 139.59.144.22:47100->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47100->104.36.82.10:4078: use of closed network connection Jul 26 23:17:59.250668157 (59h42m30.440966736s) [noise=true , address=104.36.82.16:4293 , piece_id=S32VULEEW7OXL7XS6L5YNQNDF5MTNBKD2GMAVI46PYMRIBKOPN3A, seg_pos={0 0}, upload=57255]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:104.36.82.16:4293): protocol: write tcp 139.59.144.22:41760->104.36.82.16:4293: use of closed network connection; write tcp 139.59.144.22:41760->104.36.82.16:4293: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41760->104.36.82.16:4293: use of closed network connection Jul 26 23:17:59.250723447 (59h42m30.441022026s) [noise=true , address=104.36.82.7:4909 , piece_id=AE7UKGF5AHCEFUQUV2SUKO2VNMGQJWQERGFZVUGEVWO2MNNMOHDQ, seg_pos={0 0}, upload=57255]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: write tcp 139.59.144.22:52788->104.36.82.7:4909: use of closed network connection; write tcp 139.59.144.22:52788->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52788->104.36.82.7:4909: use of closed network connection Jul 26 23:17:59.250756303 (59h42m30.441055099s) [noise=true , address=104.36.82.4:4801 , piece_id=YIADM633C3P63YMNFQDVYNWDKUFPD6KKGAJNPP7UEYJQUYOJTX6A, seg_pos={0 0}, upload=57255]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: write tcp 139.59.144.22:47114->104.36.82.4:4801: use of closed network connection; write tcp 139.59.144.22:47114->104.36.82.4:4801: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47114->104.36.82.4:4801: use of closed network connection Jul 26 23:17:59.250805640 (59h42m30.441104226s) [seg_pos={0 0}, upload=57255]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jul 26 23:17:59.356921733 (59h42m30.547220304s) [upload=57255]: Done uploading segments... Jul 26 23:17:59.356947489 (59h42m30.54724607s) [upload=57255]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 30.108487337s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 18 out of 60 is failed (threshold=3) [..................X..................X.....XXXXXX.XXXXXXXXXX] duration: 30.10512424s FAIL (2025-07-26 23:17:46 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.105284506s core uplink us1 vivint
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 15 out of 60 is failed (threshold=3) [.X.XX...X..X.XX......X.X.......X.......X........XX....X....X] duration: 1m0.001011861s FAIL (2025-07-26 23:17:29 UTC) orbiter warning: context deadline exceeded Duration: 1m0.001038742s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [.......................................X......XXXXXXXXXXXXXX] duration: 30.115735351s FAIL (2025-07-26 23:17:21 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== .32085597s) [seg_pos={0 0}, upload=57243]: Begin upload segment... Jul 26 23:16:52.130626317 (59h41m23.320924938s) [seg_pos={0 0}, upload=57243]: Done begin upload segment. Jul 26 23:16:52.130671894 (59h41m23.32097048s) [noise=true , address=104.36.82.9:4006 , piece_id=AJ4WEDV5PKCA2JWZAWCXIP54Q6AJISYJ63YMVYELQHQTNATEX2PA, seg_pos={0 0}, upload=57243]: Uploading piece... Jul 26 23:16:52.130710742 (59h41m23.321009327s) [noise=true , address=45.91.116.5:4298 , piece_id=H4BZS5YN3EAPXNSQU5BG4PQQLGRZG4BCOWNC6M345KOENCDIVS7A, seg_pos={0 0}, upload=57243]: Uploading piece... Jul 26 23:16:52.131183132 (59h41m23.321481722s) [noise=true , address=45.91.116.4:4938 , piece_id=T5FKMVO6CM3JF6UOVNBYGMYL27HJFMK2LUIDWS3OX3SDZGZHT43A, seg_pos={0 0}, upload=57243]: Uploading piece... Jul 26 23:16:52.131289198 (59h41m23.321587787s) [noise=true , address=104.36.82.8:4969 , piece_id=55APJTMZ35ZIMJST5JRTDZPOCBZFDKGY5CMI6LU3DEMOVGOVNPEQ, seg_pos={0 0}, upload=57243]: Uploading piece... Jul 26 23:16:52.131633441 (59h41m23.32193203s) [noise=true , address=77.90.31.14:4663 , piece_id=3DNF3O4NQIOJTEKLQW5OS3DO5U5NMBLHXXLDKVJCX6L5CFSHCC5Q, seg_pos={0 0}, upload=57243]: Uploading piece... Jul 26 23:16:52.132037022 (59h41m23.322335613s) [noise=true , address=104.36.82.12:4150 , piece_id=G7XRNN6WF7OG4MAQSJIOPUVWI4IKDREAFQO7XCGE64IDFYZOA36A, seg_pos={0 0}, upload=57243]: Uploading piece... Jul 26 23:16:52.133226555 (59h41m23.323525243s) [noise=true , address=45.91.116.7:4011 , piece_id=7BGZVZSGJHQTKZDHHQMGBOGYUYN62WPOBRR6ZANKLYXLUU2JEGTQ, seg_pos={0 0}, upload=57243]: Uploading piece... Jul 26 23:16:52.133726669 (59h41m23.324025258s) [noise=true , address=104.36.82.10:4078 , piece_id=JXHATJV34WQDJRMGN3PBZQ4G4T5HQCJFWO5L5755RTEJUAKTXFOA, seg_pos={0 0}, upload=57243]: Uploading piece... Jul 26 23:16:52.134235997 (59h41m23.324534586s) [noise=true , address=104.36.82.5:4838 , piece_id=YIHEPBBV7HGSQEZ4JUV26LSLTSQUWGE5HEDHMUNY2W5RCDPN73KQ, seg_pos={0 0}, upload=57243]: Uploading piece... Jul 26 23:16:52.134712086 (59h41m23.32501068s) [noise=true , address=45.91.116.7:4010 , piece_id=L4QIXOUACFPYN5252PSQYCL7XBYJDWG543MMFCG2BRLINJSBTMQA, seg_pos={0 0}, upload=57243]: Uploading piece... Jul 26 23:16:52.135184676 (59h41m23.325483264s) [noise=true , address=104.36.82.15:4258 , piece_id=3ZUPZI63SWZRQ5W6CA7PWQD3HRRS6T6FOSR4MCUVE63XSHCERUFA, seg_pos={0 0}, upload=57243]: Uploading piece... Jul 26 23:16:52.135638577 (59h41m23.325937166s) [noise=true , address=104.36.82.6:4875 , piece_id=XM7WV4P3JCFW764XZSX7TJZL567T4VL2ZQWLJHBTORP4IXQMLUCA, seg_pos={0 0}, upload=57243]: Uploading piece... Jul 26 23:16:52.136284221 (59h41m23.326582813s) [noise=true , address=104.36.82.14:4222 , piece_id=ODOPNJQDZQI5MKUK6B6U6MSMAIEQ4KQM4GNLZDF4ZVDVXGFN6KQA, seg_pos={0 0}, upload=57243]: Uploading piece... Jul 26 23:16:52.136753645 (59h41m23.327052236s) [noise=true , address=104.36.82.11:4113 , piece_id=JDVWKWAAVE3Y3PQ3PBQEVS45K4UAU42XVLPGC55UDYZXVNAAQBYQ, seg_pos={0 0}, upload=57243]: Uploading piece... Jul 26 23:16:52.137183736 (59h41m23.327482326s) [noise=true , address=104.36.82.2:4732 , piece_id=7GXMP53PFDNKUI5QJ3O5ROHITPL6UJN6FDZWA3R2IGPSYHMSWI5A, seg_pos={0 0}, upload=57243]: Uploading piece... Jul 26 23:16:52.137272500 (59h41m23.32757109s) [noise=true , address=77.90.31.12:4589 , piece_id=3NNPJYHRYE6VODBUUFD4JBBCR4PSXEBCGYKDP2XXF2E63CHVLZ4Q, seg_pos={0 0}, upload=57243]: Uploading piece... Jul 26 23:16:52.745053566 (59h41m23.935352146s) [upload=57243]: Next returned nil segment Jul 26 23:16:52.745071785 (59h41m23.935370367s) [upload=57243]: Waiting for error group managing segments... Jul 26 23:16:52.879977044 (59h41m24.070275614s) [noise=true , address=77.90.31.14:4663 , piece_id=3DNF3O4NQIOJTEKLQW5OS3DO5U5NMBLHXXLDKVJCX6L5CFSHCC5Q, seg_pos={0 0}, upload=57243]: Done uploading piece. err: <nil> Jul 26 23:16:52.884041205 (59h41m24.074339773s) [noise=true , address=45.91.116.7:4010 , piece_id=L4QIXOUACFPYN5252PSQYCL7XBYJDWG543MMFCG2BRLINJSBTMQA, seg_pos={0 0}, upload=57243]: Done uploading piece. err: <nil> Jul 26 23:16:52.889716077 (59h41m24.080014645s) [noise=true , address=45.91.116.4:4938 , piece_id=T5FKMVO6CM3JF6UOVNBYGMYL27HJFMK2LUIDWS3OX3SDZGZHT43A, seg_pos={0 0}, upload=57243]: Done uploading piece. err: <nil> Jul 26 23:16:52.893690801 (59h41m24.08398938s) [noise=true , address=45.91.116.7:4011 , piece_id=7BGZVZSGJHQTKZDHHQMGBOGYUYN62WPOBRR6ZANKLYXLUU2JEGTQ, seg_pos={0 0}, upload=57243]: Done uploading piece. err: <nil> Jul 26 23:16:52.895668940 (59h41m24.085967514s) [noise=true , address=45.91.116.5:4298 , piece_id=H4BZS5YN3EAPXNSQU5BG4PQQLGRZG4BCOWNC6M345KOENCDIVS7A, seg_pos={0 0}, upload=57243]: Done uploading piece. err: <nil> Jul 26 23:16:55.591032829 (59h41m26.781331401s) [noise=true , address=77.90.31.12:4589 , piece_id=3NNPJYHRYE6VODBUUFD4JBBCR4PSXEBCGYKDP2XXF2E63CHVLZ4Q, seg_pos={0 0}, upload=57243]: Done uploading piece. err: <nil> Jul 26 23:17:19.248869908 (59h41m50.43916848s) [noise=true , address=104.36.82.5:4838 , piece_id=YIHEPBBV7HGSQEZ4JUV26LSLTSQUWGE5HEDHMUNY2W5RCDPN73KQ, seg_pos={0 0}, upload=57243]: Done uploading piece. err: <nil> Jul 26 23:17:21.018609186 (59h41m52.208907755s) [noise=true , address=104.36.82.14:4222 , piece_id=ODOPNJQDZQI5MKUK6B6U6MSMAIEQ4KQM4GNLZDF4ZVDVXGFN6KQA, seg_pos={0 0}, upload=57243]: Done uploading piece. err: <nil> Jul 26 23:17:21.723815248 (59h41m52.914113818s) [noise=true , address=104.36.82.9:4006 , piece_id=AJ4WEDV5PKCA2JWZAWCXIP54Q6AJISYJ63YMVYELQHQTNATEX2PA, seg_pos={0 0}, upload=57243]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:104.36.82.9:4006): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 23:17:21.724298302 (59h41m52.914596872s) [noise=true , address=104.36.82.2:4732 , piece_id=7GXMP53PFDNKUI5QJ3O5ROHITPL6UJN6FDZWA3R2IGPSYHMSWI5A, seg_pos={0 0}, upload=57243]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 23:17:21.724615954 (59h41m52.914914533s) [noise=true , address=104.36.82.8:4969 , piece_id=55APJTMZ35ZIMJST5JRTDZPOCBZFDKGY5CMI6LU3DEMOVGOVNPEQ, seg_pos={0 0}, upload=57243]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 23:17:21.724770868 (59h41m52.915069446s) [noise=true , address=104.36.82.12:4150 , piece_id=G7XRNN6WF7OG4MAQSJIOPUVWI4IKDREAFQO7XCGE64IDFYZOA36A, seg_pos={0 0}, upload=57243]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:104.36.82.12:4150): protocol: write tcp 139.59.144.22:33394->104.36.82.12:4150: use of closed network connection; write tcp 139.59.144.22:33394->104.36.82.12:4150: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33394->104.36.82.12:4150: use of closed network connection Jul 26 23:17:21.724806637 (59h41m52.915105203s) [noise=true , address=104.36.82.6:4875 , piece_id=XM7WV4P3JCFW764XZSX7TJZL567T4VL2ZQWLJHBTORP4IXQMLUCA, seg_pos={0 0}, upload=57243]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:104.36.82.6:4875): protocol: write tcp 139.59.144.22:33782->104.36.82.6:4875: use of closed network connection; write tcp 139.59.144.22:33782->104.36.82.6:4875: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33782->104.36.82.6:4875: use of closed network connection Jul 26 23:17:21.724854488 (59h41m52.915153067s) [noise=true , address=104.36.82.15:4258 , piece_id=3ZUPZI63SWZRQ5W6CA7PWQD3HRRS6T6FOSR4MCUVE63XSHCERUFA, seg_pos={0 0}, upload=57243]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:104.36.82.15:4258): protocol: write tcp 139.59.144.22:55700->104.36.82.15:4258: use of closed network connection; write tcp 139.59.144.22:55700->104.36.82.15:4258: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55700->104.36.82.15:4258: use of closed network connection Jul 26 23:17:21.724931600 (59h41m52.915230165s) [noise=true , address=104.36.82.11:4113 , piece_id=JDVWKWAAVE3Y3PQ3PBQEVS45K4UAU42XVLPGC55UDYZXVNAAQBYQ, seg_pos={0 0}, upload=57243]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 139.59.144.22:36296->104.36.82.11:4113: use of closed network connection; write tcp 139.59.144.22:36296->104.36.82.11:4113: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36296->104.36.82.11:4113: use of closed network connection Jul 26 23:17:21.725004413 (59h41m52.915302997s) [noise=true , address=104.36.82.10:4078 , piece_id=JXHATJV34WQDJRMGN3PBZQ4G4T5HQCJFWO5L5755RTEJUAKTXFOA, seg_pos={0 0}, upload=57243]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: write tcp 139.59.144.22:54654->104.36.82.10:4078: use of closed network connection; write tcp 139.59.144.22:54654->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54654->104.36.82.10:4078: use of closed network connection Jul 26 23:17:21.725045401 (59h41m52.915343985s) [seg_pos={0 0}, upload=57243]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 26 23:17:21.837400358 (59h41m53.02769893s) [upload=57243]: Done uploading segments... Jul 26 23:17:21.837430491 (59h41m53.027729072s) [upload=57243]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 30.115764483s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 42 out of 60 is failed (threshold=3) [XXXXX.XXXX.XXXXX..XXX.....XXX.X.X.XX.X.XXXXXXXXX.XX..X.XXXXX] duration: 1m0.003652574s FAIL (2025-07-26 23:17:21 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== tarting upload Jul 26 23:16:22.057756402 (59h40m53.248054972s) [upload=57238]: Uploading segments... Jul 26 23:16:22.057844354 (59h40m53.248142935s) [upload=57238]: Got next segment. Inline: false Jul 26 23:16:22.057973577 (59h40m53.248272157s) [upload=57238]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 23:16:22.057980284 (59h40m53.248278865s) [upload=57238]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 23:16:22.258303419 (59h40m53.448602022s) [seg_pos={0 0}, upload=57238]: Begin upload segment... Jul 26 23:16:22.258391470 (59h40m53.448690053s) [seg_pos={0 0}, upload=57238]: Done begin upload segment. Jul 26 23:16:22.258426815 (59h40m53.448725404s) [noise=true , address=77.90.31.16:4766 , piece_id=4WQ7NJGPTEM72HPPZZ6MAJH6B3ZTSZFQKOUKR5OSODEE3AOFOCTQ, seg_pos={0 0}, upload=57238]: Uploading piece... Jul 26 23:16:22.258906518 (59h40m53.449205089s) [noise=true , address=77.90.31.21:4912 , piece_id=MYFSQ7GE7NBRCJ43NSM3YGXX7DTRHG4JE5326QLZDPEMOZD4WTEA, seg_pos={0 0}, upload=57238]: Uploading piece... Jul 26 23:16:22.259352690 (59h40m53.449651269s) [noise=true , address=77.90.31.19:4847 , piece_id=6UKSECSRISIHRN5BLGJFGKIVDXP3RRPRF3Z47IHBDLXMMRZL6AZA, seg_pos={0 0}, upload=57238]: Uploading piece... Jul 26 23:16:22.259802277 (59h40m53.450100859s) [noise=true , address=79.127.201.214:4321 , piece_id=AUHTTY6RWHWKKDEPLXRV6QGS22HD4TLPFQ4KAYYSOTY62MG4RFUQ, seg_pos={0 0}, upload=57238]: Uploading piece... Jul 26 23:16:22.260184447 (59h40m53.45048303s) [noise=true , address=104.36.82.52:4107 , piece_id=5L6NLBMI5CB4NFIRZSU2YKJPT4QRQ2SGXYH4DL3TJJAFJBFIAWJQ, seg_pos={0 0}, upload=57238]: Uploading piece... Jul 26 23:16:22.260648717 (59h40m53.450947308s) [noise=true , address=104.36.82.38:4708 , piece_id=Y2IOPPR3YBIANX3FLHRZIPINN4XKQ5GPIU72KRHKAOMHIVNPZGAQ, seg_pos={0 0}, upload=57238]: Uploading piece... Jul 26 23:16:22.261072604 (59h40m53.451371191s) [noise=true , address=104.36.82.56:4229 , piece_id=OHKBQYOGYZ34MDZGPRJRPI5JUR4IYPHXMHFMNINBBYL6TX4CTRUQ, seg_pos={0 0}, upload=57238]: Uploading piece... Jul 26 23:16:22.261452268 (59h40m53.451750855s) [noise=true , address=104.36.82.55:4212 , piece_id=IPOAFRHGTVNVWH7LNKRSDX6NDWJYT74IHPDEDSKFZGRTX5IDVG5Q, seg_pos={0 0}, upload=57238]: Uploading piece... Jul 26 23:16:22.261847126 (59h40m53.452145707s) [noise=true , address=79.127.205.233:4624 , piece_id=LD6WKXXUFDWSNIK2GITK3Y5PV77WQQYY3DDV3SKQHCOFDLTQKGJA, seg_pos={0 0}, upload=57238]: Uploading piece... Jul 26 23:16:22.262410945 (59h40m53.452709532s) [noise=true , address=104.36.82.46:4486 , piece_id=VMAGH6YN7IIS64SCTPDHWNV5KPING4QVZYBHVIKERHBHIKVSVLWA, seg_pos={0 0}, upload=57238]: Uploading piece... Jul 26 23:16:22.262802937 (59h40m53.45310152s) [noise=true , address=79.127.205.235:4690 , piece_id=GRSH2IBV433BVVBXU5GJKWDRVAH5WACXS62QFMU2XUTZQHJKV2MQ, seg_pos={0 0}, upload=57238]: Uploading piece... Jul 26 23:16:22.263198283 (59h40m53.453496873s) [noise=true , address=104.36.82.53:4150 , piece_id=DOXH47X4EO4JA34X23SNTGCIQK4K5ZLA6CHDAJDE4335PSPAOMNQ, seg_pos={0 0}, upload=57238]: Uploading piece... Jul 26 23:16:22.263576144 (59h40m53.453874727s) [noise=true , address=79.127.201.215:4353 , piece_id=W63VGC64GFW3LLZYHQXSGPYMZLB6GLIHLW2CFHKTUFAAEGNHAJ7Q, seg_pos={0 0}, upload=57238]: Uploading piece... Jul 26 23:16:22.263973337 (59h40m53.454271927s) [noise=true , address=77.90.31.10:4531 , piece_id=4MD6VJTMUUHWTPV2P3ZLBVHCACNSDGQWEBODIUGY23QIXDIK4LBQ, seg_pos={0 0}, upload=57238]: Uploading piece... Jul 26 23:16:22.264384007 (59h40m53.454682587s) [noise=true , address=104.36.82.47:4547 , piece_id=RI3IXADEHT4WNJTNWIL3RPFEPYQ63RDJKZDTO6EMCGGE3U54EFEQ, seg_pos={0 0}, upload=57238]: Uploading piece... Jul 26 23:16:22.264803121 (59h40m53.455101701s) [noise=true , address=104.36.82.21:4495 , piece_id=NTJCUH6FFWE7NYQVVDGCNGNCIIAPURMF57FVH3QKXYK6H2RW4MZA, seg_pos={0 0}, upload=57238]: Uploading piece... Jul 26 23:16:22.265295060 (59h40m53.455593639s) [noise=true , address=77.90.31.18:4828 , piece_id=P7CHP57J7Q4DQBEVPYOXLN4Q4H6WRDQUWRZ4DESDQRF6OPHTDUVQ, seg_pos={0 0}, upload=57238]: Uploading piece... Jul 26 23:16:22.265693055 (59h40m53.455991642s) [noise=true , address=104.36.82.54:4182 , piece_id=6S763NY7D2FVLKLVVWL4ADBT4F4HRC2XWO5DBTGCAXWHQR23GMTQ, seg_pos={0 0}, upload=57238]: Uploading piece... Jul 26 23:16:22.841548384 (59h40m54.031846949s) [upload=57238]: Next returned nil segment Jul 26 23:16:22.841570415 (59h40m54.031869001s) [upload=57238]: Waiting for error group managing segments... Jul 26 23:16:23.011305417 (59h40m54.201603998s) [noise=true , address=77.90.31.18:4828 , piece_id=P7CHP57J7Q4DQBEVPYOXLN4Q4H6WRDQUWRZ4DESDQRF6OPHTDUVQ, seg_pos={0 0}, upload=57238]: Done uploading piece. err: <nil> Jul 26 23:16:23.068153571 (59h40m54.25845215s) [noise=true , address=77.90.31.16:4766 , piece_id=4WQ7NJGPTEM72HPPZZ6MAJH6B3ZTSZFQKOUKR5OSODEE3AOFOCTQ, seg_pos={0 0}, upload=57238]: Done uploading piece. err: <nil> Jul 26 23:16:23.081997472 (59h40m54.272296045s) [noise=true , address=77.90.31.21:4912 , piece_id=MYFSQ7GE7NBRCJ43NSM3YGXX7DTRHG4JE5326QLZDPEMOZD4WTEA, seg_pos={0 0}, upload=57238]: Done uploading piece. err: <nil> Jul 26 23:16:23.527748255 (59h40m54.718046827s) [noise=true , address=79.127.205.233:4624 , piece_id=LD6WKXXUFDWSNIK2GITK3Y5PV77WQQYY3DDV3SKQHCOFDLTQKGJA, seg_pos={0 0}, upload=57238]: Done uploading piece. err: <nil> Jul 26 23:16:23.539320336 (59h40m54.729618916s) [noise=true , address=79.127.205.235:4690 , piece_id=GRSH2IBV433BVVBXU5GJKWDRVAH5WACXS62QFMU2XUTZQHJKV2MQ, seg_pos={0 0}, upload=57238]: Done uploading piece. err: <nil> Jul 26 23:16:26.307826037 (59h40m57.498124608s) [noise=true , address=79.127.201.215:4353 , piece_id=W63VGC64GFW3LLZYHQXSGPYMZLB6GLIHLW2CFHKTUFAAEGNHAJ7Q, seg_pos={0 0}, upload=57238]: Done uploading piece. err: <nil> Jul 26 23:16:26.558632563 (59h40m57.748931143s) [noise=true , address=79.127.201.214:4321 , piece_id=AUHTTY6RWHWKKDEPLXRV6QGS22HD4TLPFQ4KAYYSOTY62MG4RFUQ, seg_pos={0 0}, upload=57238]: Done uploading piece. err: <nil> Jul 26 23:16:35.767711309 (59h41m6.958009889s) [noise=true , address=104.36.82.55:4212 , piece_id=IPOAFRHGTVNVWH7LNKRSDX6NDWJYT74IHPDEDSKFZGRTX5IDVG5Q, seg_pos={0 0}, upload=57238]: Done uploading piece. err: <nil> Jul 26 23:16:38.281657184 (59h41m9.471955753s) [noise=true , address=104.36.82.38:4708 , piece_id=Y2IOPPR3YBIANX3FLHRZIPINN4XKQ5GPIU72KRHKAOMHIVNPZGAQ, seg_pos={0 0}, upload=57238]: Done uploading piece. err: <nil> Jul 26 23:16:39.978665446 (59h41m11.168964018s) [noise=true , address=104.36.82.56:4229 , piece_id=OHKBQYOGYZ34MDZGPRJRPI5JUR4IYPHXMHFMNINBBYL6TX4CTRUQ, seg_pos={0 0}, upload=57238]: Done uploading piece. err: <nil> Jul 26 23:16:46.612766718 (59h41m17.803065291s) [noise=true , address=104.36.82.21:4495 , piece_id=NTJCUH6FFWE7NYQVVDGCNGNCIIAPURMF57FVH3QKXYK6H2RW4MZA, seg_pos={0 0}, upload=57238]: Done uploading piece. err: <nil> Jul 26 23:17:05.567639414 (59h41m36.757937994s) [noise=true , address=77.90.31.19:4847 , piece_id=6UKSECSRISIHRN5BLGJFGKIVDXP3RRPRF3Z47IHBDLXMMRZL6AZA, seg_pos={0 0}, upload=57238]: Done uploading piece. err: <nil> Jul 26 23:17:08.473939419 (59h41m39.664238s) [noise=true , address=104.36.82.53:4150 , piece_id=DOXH47X4EO4JA34X23SNTGCIQK4K5ZLA6CHDAJDE4335PSPAOMNQ, seg_pos={0 0}, upload=57238]: Done uploading piece. err: <nil> Jul 26 23:17:15.039941481 (59h41m46.23024006s) [noise=true , address=104.36.82.47:4547 , piece_id=RI3IXADEHT4WNJTNWIL3RPFEPYQ63RDJKZDTO6EMCGGE3U54EFEQ, seg_pos={0 0}, upload=57238]: Done uploading piece. err: <nil> Jul 26 23:17:15.039984290 (59h41m46.230282874s) [seg_pos={0 0}, upload=57238]: Segment reached optimal threshold of 14 pieces. Jul 26 23:17:15.040137008 (59h41m46.230435591s) [noise=true , address=104.36.82.52:4107 , piece_id=5L6NLBMI5CB4NFIRZSU2YKJPT4QRQ2SGXYH4DL3TJJAFJBFIAWJQ, seg_pos={0 0}, upload=57238]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YVUtzGaKXr1Leb1URHZuaQXZv4zTtYwnv59j5GaAfd9zoK2Kj): protocol: expected piece hash; context canceled; EOF Jul 26 23:17:15.040228321 (59h41m46.230526903s) [noise=true , address=104.36.82.46:4486 , piece_id=VMAGH6YN7IIS64SCTPDHWNV5KPING4QVZYBHVIKERHBHIKVSVLWA, seg_pos={0 0}, upload=57238]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fbs2btcbiu2yYGrvxNZe8sNSqDnNPG1Bmfyhcew86FCFLKAic): protocol: expected piece hash; context canceled; EOF Jul 26 23:17:15.040605847 (59h41m46.230904433s) [noise=true , address=77.90.31.10:4531 , piece_id=4MD6VJTMUUHWTPV2P3ZLBVHCACNSDGQWEBODIUGY23QIXDIK4LBQ, seg_pos={0 0}, upload=57238]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1AvSrhj1nHWLBJrw1SKacnjPTi9g4AzjqKvVFoxtzAdJFYAfQt): protocol: write tcp 139.59.144.22:38420->77.90.31.10:4531: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 23:17:15.040701503 (59h41m46.231000099s) [noise=true , address=104.36.82.54:4182 , piece_id=6S763NY7D2FVLKLVVWL4ADBT4F4HRC2XWO5DBTGCAXWHQR23GMTQ, seg_pos={0 0}, upload=57238]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HbkH31sfDUnZLXAbmzQphcFYmzB8MQo98Q6Ufbo59et42zxMy): protocol: rpc: dial tcp 104.36.82.54:4182: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 23:17:15.040742278 (59h41m46.231040862s) [seg_pos={0 0}, upload=57238]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 26 23:17:15.040831640 (59h41m46.231130234s) [upload=57238]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 23:17:15.040837698 (59h41m46.231136276s) [upload=57238]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 23:17:15.190866784 (59h41m46.381165364s) [upload=57238]: Done uploading segments... Jul 26 23:17:15.190886443 (59h41m46.381185016s) [upload=57238]: Upload finished. err: <nil> Duration: 1m0.003670405s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 17 out of 60 is failed (threshold=3) [...................X..................X.....XXXXXX.XXXXXXXXX] duration: 30.109190834s FAIL (2025-07-26 23:17:09 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 5) Duration: 30.10921688s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [........................................X......XXXXXXXXXXXXX] duration: 30.120466348s FAIL (2025-07-26 23:16:51 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== SRLEUMJPAM6L75OJAIXG3XBMNSJTIGD36RYCA, seg_pos={0 0}, upload=57237]: Uploading piece... Jul 26 23:16:21.847715016 (59h40m53.038013605s) [noise=true , address=104.36.82.14:4222 , piece_id=2FMCFIG2T45B4EFFY2EZMPYIZXB3VSTC67ROKZ3VNGPRPTSBF4NA, seg_pos={0 0}, upload=57237]: Uploading piece... Jul 26 23:16:21.848139925 (59h40m53.038438508s) [noise=true , address=45.91.116.13:4153 , piece_id=HOR2MFMBKTM6OH67NGRYINK3ESDBVWBMXFQXQF4FA2FAQWRFCKJA, seg_pos={0 0}, upload=57237]: Uploading piece... Jul 26 23:16:21.848164494 (59h40m53.038463083s) [noise=true , address=104.36.82.2:4731 , piece_id=NUFIBSMLPEV2DC6MSRHS3TBQK3JLACAXTFXELOG4G5SY4NMTZ5LA, seg_pos={0 0}, upload=57237]: Uploading piece... Jul 26 23:16:21.848698105 (59h40m53.038996725s) [noise=true , address=45.91.116.5:4298 , piece_id=QNT2V6OOINOWWMQQIFHQWIZMIQLP677CEIWIPEA35NCEWUJVU76Q, seg_pos={0 0}, upload=57237]: Uploading piece... Jul 26 23:16:21.848729937 (59h40m53.039028518s) [noise=true , address=77.90.31.15:4698 , piece_id=2HM246DM5F3HQW3KHXGVIJNF746XOZRZILSL27DXH5EF2ZZ727KA, seg_pos={0 0}, upload=57237]: Uploading piece... Jul 26 23:16:21.849051111 (59h40m53.03934969s) [noise=true , address=104.36.82.14:4221 , piece_id=QH5NR2JKHVJTVXYSCAQNY7EVLSWQ6BKIKOMEMH26O4W33BLPXIHA, seg_pos={0 0}, upload=57237]: Uploading piece... Jul 26 23:16:21.849105840 (59h40m53.03940443s) [noise=true , address=45.91.116.4:4940 , piece_id=FSSCSXJT3O3QPKMBEZ52DD54F2GPCOWXANGNUKKWKY7XPX3GGWNA, seg_pos={0 0}, upload=57237]: Uploading piece... Jul 26 23:16:21.849321210 (59h40m53.039619788s) [noise=true , address=45.91.116.8:4048 , piece_id=CIKLJN2IBZ54GEHIBU3UTQYVEWIEP6MKQN72KS5BJWPOSI7RHUGA, seg_pos={0 0}, upload=57237]: Uploading piece... Jul 26 23:16:21.849467958 (59h40m53.039766554s) [noise=true , address=104.36.82.7:4909 , piece_id=7ZGCB5HMSABXPBCE3S2GYTZBNZDOEPQDITZCMHIHCBUSUAG6BWWA, seg_pos={0 0}, upload=57237]: Uploading piece... Jul 26 23:16:21.849882360 (59h40m53.040180949s) [noise=true , address=104.36.82.4:4804 , piece_id=O5AUMXBCAQIB2WIE3XN6H7SJWA7P4GSMLLKPOLIDKAVH3RJUNBSQ, seg_pos={0 0}, upload=57237]: Uploading piece... Jul 26 23:16:21.850399578 (59h40m53.040698166s) [noise=true , address=77.90.31.21:4879 , piece_id=ZSMTAWQUHYFDGUQG64PSTUPUB3BUK6K374SMAJPROTVJNLWA3MCA, seg_pos={0 0}, upload=57237]: Uploading piece... Jul 26 23:16:21.850917567 (59h40m53.041216155s) [noise=true , address=77.90.31.14:4661 , piece_id=J3VACJUHOXHZN27LGG4CNPY3VNOBIFQBUNYUQHXXHPXC6PYAZJTQ, seg_pos={0 0}, upload=57237]: Uploading piece... Jul 26 23:16:21.851566804 (59h40m53.041865394s) [noise=true , address=45.91.116.12:4227 , piece_id=WTYJBU44NU5LEEUVN7PYYE7EIVNMGJDQCJTM3ECTSWY7RFWQPFWA, seg_pos={0 0}, upload=57237]: Uploading piece... Jul 26 23:16:21.851951692 (59h40m53.042250285s) [noise=true , address=104.36.82.12:4150 , piece_id=RNQN37KAOXIYFWVTWKLT7BKPDOHOFYOQWDD72DMU6VJ6YZ5RLPTA, seg_pos={0 0}, upload=57237]: Uploading piece... Jul 26 23:16:21.853203375 (59h40m53.04350194s) [noise=true , address=104.36.82.6:4873 , piece_id=M23CZKKH4XOZNPMRE5SGOQ6ONNRHINP4JM3JXEUKWG2I5NV4TOTQ, seg_pos={0 0}, upload=57237]: Uploading piece... Jul 26 23:16:22.469245343 (59h40m53.659543912s) [upload=57237]: Next returned nil segment Jul 26 23:16:22.469261083 (59h40m53.659559654s) [upload=57237]: Waiting for error group managing segments... Jul 26 23:16:22.603238915 (59h40m53.793537501s) [noise=true , address=45.91.116.13:4153 , piece_id=HOR2MFMBKTM6OH67NGRYINK3ESDBVWBMXFQXQF4FA2FAQWRFCKJA, seg_pos={0 0}, upload=57237]: Done uploading piece. err: <nil> Jul 26 23:16:22.606238951 (59h40m53.796537563s) [noise=true , address=77.90.31.14:4661 , piece_id=J3VACJUHOXHZN27LGG4CNPY3VNOBIFQBUNYUQHXXHPXC6PYAZJTQ, seg_pos={0 0}, upload=57237]: Done uploading piece. err: <nil> Jul 26 23:16:22.608964615 (59h40m53.799263197s) [noise=true , address=45.91.116.8:4048 , piece_id=CIKLJN2IBZ54GEHIBU3UTQYVEWIEP6MKQN72KS5BJWPOSI7RHUGA, seg_pos={0 0}, upload=57237]: Done uploading piece. err: <nil> Jul 26 23:16:22.610418046 (59h40m53.800716629s) [noise=true , address=45.91.116.4:4940 , piece_id=FSSCSXJT3O3QPKMBEZ52DD54F2GPCOWXANGNUKKWKY7XPX3GGWNA, seg_pos={0 0}, upload=57237]: Done uploading piece. err: <nil> Jul 26 23:16:22.620249694 (59h40m53.810548276s) [noise=true , address=45.91.116.5:4298 , piece_id=QNT2V6OOINOWWMQQIFHQWIZMIQLP677CEIWIPEA35NCEWUJVU76Q, seg_pos={0 0}, upload=57237]: Done uploading piece. err: <nil> Jul 26 23:16:22.622458621 (59h40m53.812757195s) [noise=true , address=45.91.116.12:4227 , piece_id=WTYJBU44NU5LEEUVN7PYYE7EIVNMGJDQCJTM3ECTSWY7RFWQPFWA, seg_pos={0 0}, upload=57237]: Done uploading piece. err: <nil> Jul 26 23:16:22.626347111 (59h40m53.816645684s) [noise=true , address=77.90.31.21:4879 , piece_id=ZSMTAWQUHYFDGUQG64PSTUPUB3BUK6K374SMAJPROTVJNLWA3MCA, seg_pos={0 0}, upload=57237]: Done uploading piece. err: <nil> Jul 26 23:16:23.338379175 (59h40m54.528677743s) [noise=true , address=77.90.31.15:4698 , piece_id=2HM246DM5F3HQW3KHXGVIJNF746XOZRZILSL27DXH5EF2ZZ727KA, seg_pos={0 0}, upload=57237]: Done uploading piece. err: <nil> Jul 26 23:16:49.910100303 (59h41m21.10039888s) [noise=true , address=104.36.82.7:4909 , piece_id=7ZGCB5HMSABXPBCE3S2GYTZBNZDOEPQDITZCMHIHCBUSUAG6BWWA, seg_pos={0 0}, upload=57237]: Done uploading piece. err: <nil> Jul 26 23:16:51.469014800 (59h41m22.659313386s) [noise=true , address=104.36.82.6:4873 , piece_id=M23CZKKH4XOZNPMRE5SGOQ6ONNRHINP4JM3JXEUKWG2I5NV4TOTQ, seg_pos={0 0}, upload=57237]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:104.36.82.6:4873): protocol: write tcp 139.59.144.22:47336->104.36.82.6:4873: use of closed network connection; write tcp 139.59.144.22:47336->104.36.82.6:4873: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47336->104.36.82.6:4873: use of closed network connection Jul 26 23:16:51.469240713 (59h41m22.659539297s) [noise=true , address=104.36.82.14:4222 , piece_id=2FMCFIG2T45B4EFFY2EZMPYIZXB3VSTC67ROKZ3VNGPRPTSBF4NA, seg_pos={0 0}, upload=57237]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: write tcp 139.59.144.22:55298->104.36.82.14:4222: use of closed network connection; write tcp 139.59.144.22:55298->104.36.82.14:4222: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55298->104.36.82.14:4222: use of closed network connection Jul 26 23:16:51.469317710 (59h41m22.659616294s) [noise=true , address=104.36.82.15:4258 , piece_id=BXRWN3ZQUZR6GQNSRLEUMJPAM6L75OJAIXG3XBMNSJTIGD36RYCA, seg_pos={0 0}, upload=57237]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:104.36.82.15:4258): protocol: write tcp 139.59.144.22:41540->104.36.82.15:4258: use of closed network connection; write tcp 139.59.144.22:41540->104.36.82.15:4258: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41540->104.36.82.15:4258: use of closed network connection Jul 26 23:16:51.469385053 (59h41m22.659683634s) [noise=true , address=104.36.82.14:4221 , piece_id=QH5NR2JKHVJTVXYSCAQNY7EVLSWQ6BKIKOMEMH26O4W33BLPXIHA, seg_pos={0 0}, upload=57237]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:104.36.82.14:4221): protocol: write tcp 139.59.144.22:55392->104.36.82.14:4221: use of closed network connection; write tcp 139.59.144.22:55392->104.36.82.14:4221: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55392->104.36.82.14:4221: use of closed network connection Jul 26 23:16:51.469445653 (59h41m22.659744233s) [noise=true , address=104.36.82.4:4804 , piece_id=O5AUMXBCAQIB2WIE3XN6H7SJWA7P4GSMLLKPOLIDKAVH3RJUNBSQ, seg_pos={0 0}, upload=57237]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 139.59.144.22:50118->104.36.82.4:4804: use of closed network connection; write tcp 139.59.144.22:50118->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50118->104.36.82.4:4804: use of closed network connection Jul 26 23:16:51.469515830 (59h41m22.65981441s) [noise=true , address=104.36.82.12:4150 , piece_id=RNQN37KAOXIYFWVTWKLT7BKPDOHOFYOQWDD72DMU6VJ6YZ5RLPTA, seg_pos={0 0}, upload=57237]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:104.36.82.12:4150): protocol: write tcp 139.59.144.22:45910->104.36.82.12:4150: use of closed network connection; write tcp 139.59.144.22:45910->104.36.82.12:4150: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:45910->104.36.82.12:4150: use of closed network connection Jul 26 23:16:51.469574908 (59h41m22.659873489s) [noise=true , address=104.36.82.2:4731 , piece_id=NUFIBSMLPEV2DC6MSRHS3TBQK3JLACAXTFXELOG4G5SY4NMTZ5LA, seg_pos={0 0}, upload=57237]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:104.36.82.2:4731): protocol: write tcp 139.59.144.22:45678->104.36.82.2:4731: use of closed network connection; write tcp 139.59.144.22:45678->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:45678->104.36.82.2:4731: use of closed network connection Jul 26 23:16:51.469595064 (59h41m22.65989365s) [seg_pos={0 0}, upload=57237]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jul 26 23:16:51.578153596 (59h41m22.768452176s) [upload=57237]: Done uploading segments... Jul 26 23:16:51.578181655 (59h41m22.768480238s) [upload=57237]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 30.120491958s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..............................................X.XX...XX..X..] duration: 2.64456955s FAIL (2025-07-26 23:16:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.644605511s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 16 out of 60 is failed (threshold=3) [....................X..................X.....XXXXXX.XXXXXXXX] duration: 30.116784221s FAIL (2025-07-26 23:16:37 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 7) Duration: 30.116810951s core uplink us1 vivint
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 14 out of 60 is failed (threshold=3) [..X.XX...X..X.XX......X.X.......X.......X........XX....X....] duration: 19.372647436s FAIL (2025-07-26 23:16:23 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 19.372674516s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 41 out of 60 is failed (threshold=3) [.XXXXX.XXXX.XXXXX..XXX.....XXX.X.X.XX.X.XXXXXXXXX.XX..X.XXXX] duration: 1m0.107622718s FAIL (2025-07-26 23:16:21 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 38997s) [upload=57219]: Got next segment. Inline: false Jul 26 23:15:21.793182729 (59h39m52.983481299s) [upload=57219]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 23:15:21.793188681 (59h39m52.98348725s) [upload=57219]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 23:15:21.990740801 (59h39m53.181039391s) [seg_pos={0 0}, upload=57219]: Begin upload segment... Jul 26 23:15:21.990830555 (59h39m53.181129145s) [seg_pos={0 0}, upload=57219]: Done begin upload segment. Jul 26 23:15:21.990887610 (59h39m53.181186202s) [noise=true , address=79.127.219.38:4004 , piece_id=FJ723G5TR7IU44L4GP3LTOTVUEPWEFFOT35ELTLH3BZK3BXT5SVQ, seg_pos={0 0}, upload=57219]: Uploading piece... Jul 26 23:15:21.991401013 (59h39m53.181699602s) [noise=true , address=156.146.43.225:4448 , piece_id=OIK5HVT5EQ4OH6HT47YGU5RH23E5DURFER7NTTUWJTBXXE6BOHPQ, seg_pos={0 0}, upload=57219]: Uploading piece... Jul 26 23:15:21.991783912 (59h39m53.182082501s) [noise=true , address=79.127.219.47:4069 , piece_id=4FY6NZMGIQJED5G2D4LWMCPRTB6CJ6YX65CRC7UD3BIZBMP7ETPA, seg_pos={0 0}, upload=57219]: Uploading piece... Jul 26 23:15:21.992139707 (59h39m53.1824383s) [noise=true , address=64.49.13.138:4482 , piece_id=U6KFSZOCBRVVPXRPM7HF4UXE6NKXI6OQYVEKBUL7KZ6VUULEF2ZA, seg_pos={0 0}, upload=57219]: Uploading piece... Jul 26 23:15:21.992714100 (59h39m53.18301269s) [noise=true , address=79.127.220.101:4048 , piece_id=GMLNHORFATXIIEVELYMRV5VLJ2AZRYTNPNWAJNFVGA74XPT2JGMA, seg_pos={0 0}, upload=57219]: Uploading piece... Jul 26 23:15:21.993111323 (59h39m53.183409915s) [noise=true , address=104.36.82.54:4183 , piece_id=X2ZG5KRJJICPAVJWYCYQHNNNG7FMRMFSTSPOWGBY5ZHRUGHS654A, seg_pos={0 0}, upload=57219]: Uploading piece... Jul 26 23:15:21.993485623 (59h39m53.183784212s) [noise=true , address=79.127.205.235:4677 , piece_id=6TTWNLPLKD4JEZGUYU7FW22EIQZU3DKYSDN3TJP4BP7OZXBE7M3A, seg_pos={0 0}, upload=57219]: Uploading piece... Jul 26 23:15:21.993860561 (59h39m53.184159151s) [noise=true , address=104.36.82.46:4483 , piece_id=TKVJX23XHI456IA6RBRNHADFVAXS4CYKLDYVN7UMV3BHK3E4EJTQ, seg_pos={0 0}, upload=57219]: Uploading piece... Jul 26 23:15:21.994063863 (59h39m53.184362457s) [noise=true , address=77.90.31.10:4528 , piece_id=JQO4LOFNM6SN55CWIIYSUIUIGOV6SXTD3NNLK4LCDWSTYPWKTHUQ, seg_pos={0 0}, upload=57219]: Uploading piece... Jul 26 23:15:21.994375157 (59h39m53.184673748s) [noise=true , address=104.36.82.52:4101 , piece_id=YNDNW54VGQW373RSACLAHUQGJ4EGIJ37GMJSURLSTM4GEFKGZTKA, seg_pos={0 0}, upload=57219]: Uploading piece... Jul 26 23:15:21.994537953 (59h39m53.18483656s) [noise=true , address=77.90.31.18:4822 , piece_id=X6YJURR7HOIFKCBJDN7ECJ4JZN2AFSAXTEJPIVJQMOOMEHCCU74A, seg_pos={0 0}, upload=57219]: Uploading piece... Jul 26 23:15:21.994959601 (59h39m53.185258193s) [noise=true , address=104.36.82.47:4535 , piece_id=47DRRVYUM6MZOIVCLHFYDLMPFYQ5XDRHSATCT2E2244TJFHXGWSA, seg_pos={0 0}, upload=57219]: Uploading piece... Jul 26 23:15:21.995336031 (59h39m53.18563462s) [noise=true , address=104.36.82.55:4195 , piece_id=FFS2CABSG7FGDM5FYW6BR54C2KKD5JPVLYN73SUAKUT4Z6HWMX5A, seg_pos={0 0}, upload=57219]: Uploading piece... Jul 26 23:15:21.995714369 (59h39m53.186012959s) [noise=true , address=79.127.201.215:4347 , piece_id=5CSF3M2RDDUJTPAGQZ6PTM2NKS7IX4XOICO6X5DTVJBXR3CBUTWQ, seg_pos={0 0}, upload=57219]: Uploading piece... Jul 26 23:15:21.996063611 (59h39m53.186362205s) [noise=true , address=77.90.31.21:4894 , piece_id=QBJ3KXTJCPB766CWF5YR2FIFEFTI623VMICI6J3W563M6VPOC3HQ, seg_pos={0 0}, upload=57219]: Uploading piece... Jul 26 23:15:21.996517389 (59h39m53.186815979s) [noise=true , address=104.36.82.53:4149 , piece_id=BGPLJOOZSUQIZSXYVNFUGBQVZ3JEWU6FMN44R5MLOZFTG2NELWQQ, seg_pos={0 0}, upload=57219]: Uploading piece... Jul 26 23:15:21.996874928 (59h39m53.187173517s) [noise=true , address=64.49.13.170:4718 , piece_id=WKDJOQWLCPK5OARNMBYB5SAUXD3G7XVDK3DKKAINYBHLZHPV5V3Q, seg_pos={0 0}, upload=57219]: Uploading piece... Jul 26 23:15:21.997215861 (59h39m53.187514454s) [noise=true , address=79.127.205.245:4246 , piece_id=4BFC25SX5ZSYLJLCGAMAI2XZ6LGQP65OVEUHHGAQPW4QIMXXVADA, seg_pos={0 0}, upload=57219]: Uploading piece... Jul 26 23:15:22.510570698 (59h39m53.70086927s) [noise=true , address=79.127.220.101:4048 , piece_id=GMLNHORFATXIIEVELYMRV5VLJ2AZRYTNPNWAJNFVGA74XPT2JGMA, seg_pos={0 0}, upload=57219]: Done uploading piece. err: ecclient: upload failed (node:1csdxFUHCWKuGfrZnVfpRJ3R6DYCzxx3voY1UR5kWwSXaxZaPg, address:79.127.220.101:4048): protocol: storage node overloaded, request limit: 2000 Jul 26 23:15:22.570426336 (59h39m53.760724927s) [upload=57219]: Next returned nil segment Jul 26 23:15:22.570450561 (59h39m53.760749151s) [upload=57219]: Waiting for error group managing segments... Jul 26 23:15:22.742294741 (59h39m53.932593329s) [noise=true , address=77.90.31.18:4822 , piece_id=X6YJURR7HOIFKCBJDN7ECJ4JZN2AFSAXTEJPIVJQMOOMEHCCU74A, seg_pos={0 0}, upload=57219]: Done uploading piece. err: <nil> Jul 26 23:15:22.771674479 (59h39m53.961973063s) [noise=true , address=79.127.219.47:4069 , piece_id=4FY6NZMGIQJED5G2D4LWMCPRTB6CJ6YX65CRC7UD3BIZBMP7ETPA, seg_pos={0 0}, upload=57219]: Done uploading piece. err: <nil> Jul 26 23:15:22.781458040 (59h39m53.971756631s) [noise=true , address=79.127.219.38:4004 , piece_id=FJ723G5TR7IU44L4GP3LTOTVUEPWEFFOT35ELTLH3BZK3BXT5SVQ, seg_pos={0 0}, upload=57219]: Done uploading piece. err: <nil> Jul 26 23:15:22.815628802 (59h39m54.005927368s) [noise=true , address=77.90.31.21:4894 , piece_id=QBJ3KXTJCPB766CWF5YR2FIFEFTI623VMICI6J3W563M6VPOC3HQ, seg_pos={0 0}, upload=57219]: Done uploading piece. err: <nil> Jul 26 23:15:23.402930563 (59h39m54.593229143s) [noise=true , address=79.127.205.235:4677 , piece_id=6TTWNLPLKD4JEZGUYU7FW22EIQZU3DKYSDN3TJP4BP7OZXBE7M3A, seg_pos={0 0}, upload=57219]: Done uploading piece. err: <nil> Jul 26 23:15:23.631757129 (59h39m54.8220557s) [noise=true , address=79.127.205.245:4246 , piece_id=4BFC25SX5ZSYLJLCGAMAI2XZ6LGQP65OVEUHHGAQPW4QIMXXVADA, seg_pos={0 0}, upload=57219]: Done uploading piece. err: <nil> Jul 26 23:15:24.663624205 (59h39m55.853922779s) [noise=true , address=64.49.13.138:4482 , piece_id=U6KFSZOCBRVVPXRPM7HF4UXE6NKXI6OQYVEKBUL7KZ6VUULEF2ZA, seg_pos={0 0}, upload=57219]: Done uploading piece. err: <nil> Jul 26 23:15:25.749894894 (59h39m56.940193472s) [noise=true , address=104.36.82.47:4535 , piece_id=47DRRVYUM6MZOIVCLHFYDLMPFYQ5XDRHSATCT2E2244TJFHXGWSA, seg_pos={0 0}, upload=57219]: Done uploading piece. err: ecclient: upload failed (node:12KFwCcqBCTyeBBQYzg6SFG4YDzBm2H9s4D2JRvnC31VAGvpLhu, address:104.36.82.47:4535): protocol: storage node overloaded, request limit: 2000 Jul 26 23:15:26.255575052 (59h39m57.445873622s) [noise=true , address=156.146.43.225:4448 , piece_id=OIK5HVT5EQ4OH6HT47YGU5RH23E5DURFER7NTTUWJTBXXE6BOHPQ, seg_pos={0 0}, upload=57219]: Done uploading piece. err: <nil> Jul 26 23:15:27.532178910 (59h39m58.722477476s) [noise=true , address=79.127.201.215:4347 , piece_id=5CSF3M2RDDUJTPAGQZ6PTM2NKS7IX4XOICO6X5DTVJBXR3CBUTWQ, seg_pos={0 0}, upload=57219]: Done uploading piece. err: <nil> Jul 26 23:15:29.173201358 (59h40m0.363499927s) [noise=true , address=64.49.13.170:4718 , piece_id=WKDJOQWLCPK5OARNMBYB5SAUXD3G7XVDK3DKKAINYBHLZHPV5V3Q, seg_pos={0 0}, upload=57219]: Done uploading piece. err: <nil> Jul 26 23:15:31.549636450 (59h40m2.739935035s) [noise=true , address=104.36.82.55:4195 , piece_id=FFS2CABSG7FGDM5FYW6BR54C2KKD5JPVLYN73SUAKUT4Z6HWMX5A, seg_pos={0 0}, upload=57219]: Done uploading piece. err: ecclient: upload failed (node:12cizR87omcPERJsRZt64xHfeQyw6BUpBsJPB2txS4LjZR9StMa, address:104.36.82.55:4195): protocol: storage node overloaded, request limit: 2000 Jul 26 23:15:47.266947612 (59h40m18.457246185s) [noise=true , address=104.36.82.54:4183 , piece_id=X2ZG5KRJJICPAVJWYCYQHNNNG7FMRMFSTSPOWGBY5ZHRUGHS654A, seg_pos={0 0}, upload=57219]: Done uploading piece. err: <nil> Jul 26 23:15:57.857684266 (59h40m29.047982846s) [noise=true , address=104.36.82.52:4101 , piece_id=YNDNW54VGQW373RSACLAHUQGJ4EGIJ37GMJSURLSTM4GEFKGZTKA, seg_pos={0 0}, upload=57219]: Done uploading piece. err: <nil> Jul 26 23:16:21.519652767 (59h40m52.709951348s) [noise=true , address=104.36.82.46:4483 , piece_id=TKVJX23XHI456IA6RBRNHADFVAXS4CYKLDYVN7UMV3BHK3E4EJTQ, seg_pos={0 0}, upload=57219]: Done uploading piece. err: ecclient: upload failed (node:1igSs2ZGX3zWVsSdmm1pnwYd7JckcyBubTuNU1gJDxQsZ9nKaY, address:104.36.82.46:4483): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 23:16:21.519770115 (59h40m52.710068695s) [noise=true , address=104.36.82.53:4149 , piece_id=BGPLJOOZSUQIZSXYVNFUGBQVZ3JEWU6FMN44R5MLOZFTG2NELWQQ, seg_pos={0 0}, upload=57219]: Done uploading piece. err: ecclient: upload failed (node:1nDCYjyLUqt1zkfotDNVcor73Cdpq3fnyMiygujdc9V4tGNHLY, address:104.36.82.53:4149): protocol: write tcp 139.59.144.22:47500->104.36.82.53:4149: use of closed network connection; write tcp 139.59.144.22:47500->104.36.82.53:4149: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47500->104.36.82.53:4149: use of closed network connection Jul 26 23:16:21.519814594 (59h40m52.710113173s) [noise=true , address=77.90.31.10:4528 , piece_id=JQO4LOFNM6SN55CWIIYSUIUIGOV6SXTD3NNLK4LCDWSTYPWKTHUQ, seg_pos={0 0}, upload=57219]: Done uploading piece. err: ecclient: upload failed (node:15fkvAWei1E5Tn2BYkCavyZ8se6XwPJcmpr7aEsv8sbMtk1n3Q, address:77.90.31.10:4528): protocol: read tcp 139.59.144.22:39498->77.90.31.10:4528: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 23:16:21.519843947 (59h40m52.710142529s) [seg_pos={0 0}, upload=57219]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Jul 26 23:16:21.625144084 (59h40m52.815442664s) [upload=57219]: Done uploading segments... Jul 26 23:16:21.625176284 (59h40m52.815474866s) [upload=57219]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 1m0.107640288s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [...............................................X.XX...XX..X.] duration: 23.221313322s FAIL (2025-07-26 23:16:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 23.221347863s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.........................................X......XXXXXXXXXXXX] duration: 30.117519917s FAIL (2025-07-26 23:15:58 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ed network connection; piecestore: piecestore close: write tcp 139.59.144.22:56984->104.36.82.9:4005: use of closed network connection Jul 26 23:14:54.441903682 (59h39m25.632202264s) [seg_pos={0 0}, upload=57205]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jul 26 23:14:54.553012911 (59h39m25.743311478s) [upload=57205]: Done uploading segments... Jul 26 23:14:54.553049336 (59h39m25.743347914s) [upload=57205]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Jul 26 23:15:28.308310758 (59h39m59.498609329s) [upload=57221]: Starting upload Jul 26 23:15:28.308632288 (59h39m59.498930873s) [upload=57221]: Done starting upload Jul 26 23:15:28.308706969 (59h39m59.499005552s) [upload=57221]: Uploading segments... Jul 26 23:15:28.309049803 (59h39m59.499348374s) [upload=57221]: Got next segment. Inline: false Jul 26 23:15:28.309099273 (59h39m59.499397843s) [upload=57221]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 23:15:28.309105581 (59h39m59.499404159s) [upload=57221]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 23:15:28.418027521 (59h39m59.608326087s) [seg_pos={0 0}, upload=57221]: Begin upload segment... Jul 26 23:15:28.418076832 (59h39m59.608375421s) [seg_pos={0 0}, upload=57221]: Done begin upload segment. Jul 26 23:15:28.418102783 (59h39m59.608401365s) [noise=true , address=77.90.31.19:4843 , piece_id=AKICSAT5XKCBX3OXH7X332IQYZFR3TJL53DCAAT3PY3UCVW5QONQ, seg_pos={0 0}, upload=57221]: Uploading piece... Jul 26 23:15:28.418121945 (59h39m59.608420535s) [noise=true , address=104.36.82.4:4801 , piece_id=DMWSPZFBTD7CJEL2DSLGQI2FS6NSRSIK5JRJXYMM5N5RAM3M2DAA, seg_pos={0 0}, upload=57221]: Uploading piece... Jul 26 23:15:28.418430852 (59h39m59.608729431s) [noise=true , address=45.91.116.5:4298 , piece_id=HRN4J6OQLT3XAJWZPAMJA6R4SECEY5ETWGH76NFTK7AWE3P3TLJQ, seg_pos={0 0}, upload=57221]: Uploading piece... Jul 26 23:15:28.418675419 (59h39m59.608973998s) [noise=true , address=104.36.82.6:4873 , piece_id=UKDXLX3Q65J7LLIQPBYD527LVBREKKFEI4W2ST5IHWGVICF3S6HQ, seg_pos={0 0}, upload=57221]: Uploading piece... Jul 26 23:15:28.418746639 (59h39m59.609045228s) [noise=true , address=77.90.31.16:4734 , piece_id=RUVNUCXBG5TMJN3EASG5DQJN7B4ZCY43GZ4CB7OZXA3U33Q4FRAQ, seg_pos={0 0}, upload=57221]: Uploading piece... Jul 26 23:15:28.418929256 (59h39m59.609227837s) [noise=true , address=104.36.82.7:4909 , piece_id=3OIPKWAQQNVOB5TCSDDWTCOSEM6LV2LTXQSLDBCFGEGHGAKW22UQ, seg_pos={0 0}, upload=57221]: Uploading piece... Jul 26 23:15:28.419113787 (59h39m59.60941237s) [noise=true , address=77.90.31.21:4879 , piece_id=WOD5QIWZW7OGDLN7J5TTOHFW47FYL426TYVGCXNV47ZYDYMPNITQ, seg_pos={0 0}, upload=57221]: Uploading piece... Jul 26 23:15:28.419164266 (59h39m59.609462845s) [noise=true , address=104.36.82.11:4115 , piece_id=NHLMQRFVBT3OI5GO72ERLZM6UWWUWZLK3M3KY7EZVF2AO6ZGPKAQ, seg_pos={0 0}, upload=57221]: Uploading piece... Jul 26 23:15:28.419417006 (59h39m59.609715584s) [noise=true , address=45.91.116.8:4045 , piece_id=WDPPKZNGPP2WO6M6HNJ3GC6QFZOHA3ZPVHKJ6ZZKD7F477NIO7EA, seg_pos={0 0}, upload=57221]: Uploading piece... Jul 26 23:15:28.419665292 (59h39m59.609963871s) [noise=true , address=45.91.116.7:4010 , piece_id=VOIDAAYM3PMH55JLX4MSHSL6N4HBRBSQNKKF4LJPFP742V3VJ6RA, seg_pos={0 0}, upload=57221]: Uploading piece... Jul 26 23:15:28.420658238 (59h39m59.610956827s) [noise=true , address=45.91.116.5:4299 , piece_id=JSSL4XJEEVTVEWZCHJ2TUTQHPUUDOU2FHXOXUBIW52QA7N26W3KA, seg_pos={0 0}, upload=57221]: Uploading piece... Jul 26 23:15:28.421089729 (59h39m59.611388313s) [noise=true , address=77.90.31.14:4661 , piece_id=JH3CYETWLYPE4RI6ESWV7J5Q3GSVRQRMSCZKODQ3A2FKXAGXDJJA, seg_pos={0 0}, upload=57221]: Uploading piece... Jul 26 23:15:28.422822307 (59h39m59.613120887s) [noise=true , address=45.91.116.10:4082 , piece_id=W4JCO3UOBNHIZXQC7W2UR66CKKFDLPL5ZCUSF2S4TCEYEPRVNUQQ, seg_pos={0 0}, upload=57221]: Uploading piece... Jul 26 23:15:28.423302310 (59h39m59.613600896s) [noise=true , address=45.91.116.6:4975 , piece_id=ZCPHZ66ITNLYI6HRJ553SKMJHDYGRJPTIFHQEQUW4MPFPSRR2KQQ, seg_pos={0 0}, upload=57221]: Uploading piece... Jul 26 23:15:28.423703986 (59h39m59.614002569s) [noise=true , address=45.91.116.4:4937 , piece_id=RXUYE3NN6KYGTFDXH34RGIWN26T62VFPRLSZTW5YEGSN44GVVBTQ, seg_pos={0 0}, upload=57221]: Uploading piece... Jul 26 23:15:28.424100013 (59h39m59.614398598s) [noise=true , address=45.91.116.7:4009 , piece_id=SYL4SOJC7BKXAHVOGVGUUYQTYUAIL4T4T7Z6CZ6URT4T3ZBHLRCQ, seg_pos={0 0}, upload=57221]: Uploading piece... Jul 26 23:15:29.027443437 (59h40m0.217742027s) [upload=57221]: Next returned nil segment Jul 26 23:15:29.027467754 (59h40m0.217766344s) [upload=57221]: Waiting for error group managing segments... Jul 26 23:15:29.166036455 (59h40m0.356335034s) [noise=true , address=45.91.116.7:4009 , piece_id=SYL4SOJC7BKXAHVOGVGUUYQTYUAIL4T4T7Z6CZ6URT4T3ZBHLRCQ, seg_pos={0 0}, upload=57221]: Done uploading piece. err: <nil> Jul 26 23:15:29.167641458 (59h40m0.35794003s) [noise=true , address=45.91.116.7:4010 , piece_id=VOIDAAYM3PMH55JLX4MSHSL6N4HBRBSQNKKF4LJPFP742V3VJ6RA, seg_pos={0 0}, upload=57221]: Done uploading piece. err: <nil> Jul 26 23:15:29.167739101 (59h40m0.358037673s) [noise=true , address=45.91.116.5:4298 , piece_id=HRN4J6OQLT3XAJWZPAMJA6R4SECEY5ETWGH76NFTK7AWE3P3TLJQ, seg_pos={0 0}, upload=57221]: Done uploading piece. err: <nil> Jul 26 23:15:29.169075254 (59h40m0.359373827s) [noise=true , address=45.91.116.10:4082 , piece_id=W4JCO3UOBNHIZXQC7W2UR66CKKFDLPL5ZCUSF2S4TCEYEPRVNUQQ, seg_pos={0 0}, upload=57221]: Done uploading piece. err: <nil> Jul 26 23:15:29.178880897 (59h40m0.369179478s) [noise=true , address=77.90.31.14:4661 , piece_id=JH3CYETWLYPE4RI6ESWV7J5Q3GSVRQRMSCZKODQ3A2FKXAGXDJJA, seg_pos={0 0}, upload=57221]: Done uploading piece. err: <nil> Jul 26 23:15:29.179029106 (59h40m0.369327835s) [noise=true , address=45.91.116.5:4299 , piece_id=JSSL4XJEEVTVEWZCHJ2TUTQHPUUDOU2FHXOXUBIW52QA7N26W3KA, seg_pos={0 0}, upload=57221]: Done uploading piece. err: <nil> Jul 26 23:15:29.181560864 (59h40m0.371859435s) [noise=true , address=45.91.116.4:4937 , piece_id=RXUYE3NN6KYGTFDXH34RGIWN26T62VFPRLSZTW5YEGSN44GVVBTQ, seg_pos={0 0}, upload=57221]: Done uploading piece. err: <nil> Jul 26 23:15:29.185157584 (59h40m0.375456154s) [noise=true , address=45.91.116.8:4045 , piece_id=WDPPKZNGPP2WO6M6HNJ3GC6QFZOHA3ZPVHKJ6ZZKD7F477NIO7EA, seg_pos={0 0}, upload=57221]: Done uploading piece. err: <nil> Jul 26 23:15:29.196829854 (59h40m0.387128442s) [noise=true , address=45.91.116.6:4975 , piece_id=ZCPHZ66ITNLYI6HRJ553SKMJHDYGRJPTIFHQEQUW4MPFPSRR2KQQ, seg_pos={0 0}, upload=57221]: Done uploading piece. err: <nil> Jul 26 23:15:29.216099701 (59h40m0.406398273s) [noise=true , address=77.90.31.21:4879 , piece_id=WOD5QIWZW7OGDLN7J5TTOHFW47FYL426TYVGCXNV47ZYDYMPNITQ, seg_pos={0 0}, upload=57221]: Done uploading piece. err: <nil> Jul 26 23:15:29.308909540 (59h40m0.499208115s) [noise=true , address=77.90.31.16:4734 , piece_id=RUVNUCXBG5TMJN3EASG5DQJN7B4ZCY43GZ4CB7OZXA3U33Q4FRAQ, seg_pos={0 0}, upload=57221]: Done uploading piece. err: <nil> Jul 26 23:15:30.561760720 (59h40m1.752059301s) [noise=true , address=77.90.31.19:4843 , piece_id=AKICSAT5XKCBX3OXH7X332IQYZFR3TJL53DCAAT3PY3UCVW5QONQ, seg_pos={0 0}, upload=57221]: Done uploading piece. err: <nil> Jul 26 23:15:58.040913272 (59h40m29.23121185s) [noise=true , address=104.36.82.4:4801 , piece_id=DMWSPZFBTD7CJEL2DSLGQI2FS6NSRSIK5JRJXYMM5N5RAM3M2DAA, seg_pos={0 0}, upload=57221]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 23:15:58.041068137 (59h40m29.231366705s) [noise=true , address=104.36.82.11:4115 , piece_id=NHLMQRFVBT3OI5GO72ERLZM6UWWUWZLK3M3KY7EZVF2AO6ZGPKAQ, seg_pos={0 0}, upload=57221]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:104.36.82.11:4115): protocol: write tcp 139.59.144.22:45454->104.36.82.11:4115: use of closed network connection; write tcp 139.59.144.22:45454->104.36.82.11:4115: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:45454->104.36.82.11:4115: use of closed network connection Jul 26 23:15:58.041121228 (59h40m29.231419811s) [noise=true , address=104.36.82.6:4873 , piece_id=UKDXLX3Q65J7LLIQPBYD527LVBREKKFEI4W2ST5IHWGVICF3S6HQ, seg_pos={0 0}, upload=57221]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:104.36.82.6:4873): protocol: write tcp 139.59.144.22:57594->104.36.82.6:4873: use of closed network connection; write tcp 139.59.144.22:57594->104.36.82.6:4873: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57594->104.36.82.6:4873: use of closed network connection Jul 26 23:15:58.041170678 (59h40m29.231469247s) [noise=true , address=104.36.82.7:4909 , piece_id=3OIPKWAQQNVOB5TCSDDWTCOSEM6LV2LTXQSLDBCFGEGHGAKW22UQ, seg_pos={0 0}, upload=57221]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: write tcp 139.59.144.22:33408->104.36.82.7:4909: use of closed network connection; write tcp 139.59.144.22:33408->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33408->104.36.82.7:4909: use of closed network connection Jul 26 23:15:58.041186446 (59h40m29.231485029s) [seg_pos={0 0}, upload=57221]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 23:15:58.156508440 (59h40m29.346807042s) [upload=57221]: Done uploading segments... Jul 26 23:15:58.156546508 (59h40m29.346845088s) [upload=57221]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.1175518s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 14 out of 60 is failed (threshold=3) [...X.XX...X..X.XX......X.X.......X.......X........XX....X...] duration: 52.701875888s FAIL (2025-07-26 23:15:57 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 52.701903662s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [................................................X.XX...XX..X] duration: 30.000964972s FAIL (2025-07-26 23:15:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001010479s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 15 out of 60 is failed (threshold=3) [.....................X..................X.....XXXXXX.XXXXXXX] duration: 30.12730387s FAIL (2025-07-26 23:15:29 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 8) Duration: 30.127375209s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 40 out of 60 is failed (threshold=3) [..XXXXX.XXXX.XXXXX..XXX.....XXX.X.X.XX.X.XXXXXXXXX.XX..X.XXX] duration: 1m0.106967598s FAIL (2025-07-26 23:15:21 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ad=57203]: Done starting upload Jul 26 23:14:21.539516146 (59h38m52.72981472s) [upload=57203]: Uploading segments... Jul 26 23:14:21.539670059 (59h38m52.729968638s) [upload=57203]: Got next segment. Inline: false Jul 26 23:14:21.539715901 (59h38m52.730014472s) [upload=57203]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 23:14:21.539721791 (59h38m52.730020361s) [upload=57203]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 23:14:21.735855609 (59h38m52.926154188s) [seg_pos={0 0}, upload=57203]: Begin upload segment... Jul 26 23:14:21.735922568 (59h38m52.926221137s) [seg_pos={0 0}, upload=57203]: Done begin upload segment. Jul 26 23:14:21.735943265 (59h38m52.926241853s) [noise=true , address=79.127.205.234:4617 , piece_id=VVHD2E7ZJDOZRT3LXKLFCVJWUK7QOF7G4PB4FAPORJL477CHNXOQ, seg_pos={0 0}, upload=57203]: Uploading piece... Jul 26 23:14:21.735956331 (59h38m52.926254911s) [noise=true , address=77.90.31.12:4611 , piece_id=OSM4YJ2IY6SKRWVSTNEFD2EEWGV463SM7COP262LNCCGG3CW2TMA, seg_pos={0 0}, upload=57203]: Uploading piece... Jul 26 23:14:21.736351852 (59h38m52.926650431s) [noise=true , address=77.90.31.10:4537 , piece_id=LIMMBCSPR3Y6RIAD7B7XW6RI62SKTFDPJXC3YBWF74I2OY6VMTWA, seg_pos={0 0}, upload=57203]: Uploading piece... Jul 26 23:14:21.736633051 (59h38m52.926931641s) [noise=true , address=77.90.31.21:4905 , piece_id=7XFYGT35LLMUPAQ23M2RAIKNOL3PMK2WSHGJM5J2IBSVPE7TVKQA, seg_pos={0 0}, upload=57203]: Uploading piece... Jul 26 23:14:21.736850639 (59h38m52.927149227s) [noise=true , address=77.90.31.15:4731 , piece_id=ICUAWRYOTXPP2N4URYQZRTM3XG26BGF4RD3LI22ZU25K3ZNMIJJQ, seg_pos={0 0}, upload=57203]: Uploading piece... Jul 26 23:14:21.737079402 (59h38m52.927377991s) [noise=true , address=79.127.205.246:4253 , piece_id=L6EY3M6TXLLXU4HCS3KK77FWGSGBAPH5ZOT7HYYKQO3AY4CSXIBQ, seg_pos={0 0}, upload=57203]: Uploading piece... Jul 26 23:14:21.737151479 (59h38m52.927450061s) [noise=true , address=104.36.82.46:4499 , piece_id=YFZNL5NTX37LBU52MPDMRKQNNYEM7YO5IZCHOLFSYOZGLYYW5USQ, seg_pos={0 0}, upload=57203]: Uploading piece... Jul 26 23:14:21.737464207 (59h38m52.927762795s) [noise=true , address=104.36.82.47:4531 , piece_id=XA6POGWTT2ODVTX6CGIYLQ3YVBO6YY7V37FADOMANUHNPWCHAT7Q, seg_pos={0 0}, upload=57203]: Uploading piece... Jul 26 23:14:21.737799829 (59h38m52.928098408s) [noise=true , address=79.127.219.51:4210 , piece_id=Y6E2NMMDH77RHEXCBBQXANJGQ44AXT36D466VWWT33YUQSU7N5GQ, seg_pos={0 0}, upload=57203]: Uploading piece... Jul 26 23:14:21.742366432 (59h38m52.93266502s) [noise=true , address=104.36.82.55:4205 , piece_id=SSXSXX4LTXXVM5H4NNXQUZCZM4SWZMNI4PJ33JL7FC5ZFRXENSYQ, seg_pos={0 0}, upload=57203]: Uploading piece... Jul 26 23:14:21.742704887 (59h38m52.933003525s) [noise=true , address=104.36.82.56:4231 , piece_id=5K36ZSIRUNGLQFJEK7UFOENQ76WIX7XDYO6TRZGUMNDNTMHMMHOQ, seg_pos={0 0}, upload=57203]: Uploading piece... Jul 26 23:14:21.743042361 (59h38m52.933340933s) [noise=true , address=79.127.219.47:4061 , piece_id=RIYCNHJ4AD7UYEQOL5E3OR3ICQOPU2M6ZBX2UHAQEUSJDIXWLHZQ, seg_pos={0 0}, upload=57203]: Uploading piece... Jul 26 23:14:21.743167308 (59h38m52.933465888s) [noise=true , address=79.127.205.235:4688 , piece_id=HXH2MHZCSABY43FIG4DX7JT44NZVQFYSVOKNIOFUX3K6ODUH3VVQ, seg_pos={0 0}, upload=57203]: Uploading piece... Jul 26 23:14:21.743521312 (59h38m52.933819897s) [noise=true , address=79.127.219.38:4004 , piece_id=WGKKECSZSPZ7Q6DYH3JVPUBBBKIIY7424BQPKXKP4DVAZBYCOV7Q, seg_pos={0 0}, upload=57203]: Uploading piece... Jul 26 23:14:21.743964524 (59h38m52.934263113s) [noise=true , address=104.36.82.53:4153 , piece_id=QFLQM6FZ32TC4NWIRVKBJVUEIDIUOGLZQLVSAZBXEDGKAF2KRXEA, seg_pos={0 0}, upload=57203]: Uploading piece... Jul 26 23:14:21.744429791 (59h38m52.93472838s) [noise=true , address=45.91.116.11:4151 , piece_id=BMGPRP6VPHGNWMYPVTYP3CI32ELJLXQUDNIDGZ6AYITLLJTSMJDA, seg_pos={0 0}, upload=57203]: Uploading piece... Jul 26 23:14:21.744488157 (59h38m52.934786736s) [noise=true , address=104.36.82.14:4256 , piece_id=BAXTEHXTOCSJO4TEEZBOKZ7KA6WKM3NDKZ752DNPHDJP5AWHWPFQ, seg_pos={0 0}, upload=57203]: Uploading piece... Jul 26 23:14:21.746334350 (59h38m52.936632932s) [noise=true , address=104.36.82.54:4159 , piece_id=ULPEPD75YHJHD3J6GCDYQYIVEY7F3DZLYM2DAZN2ENUZDK36WFAQ, seg_pos={0 0}, upload=57203]: Uploading piece... Jul 26 23:14:22.153741036 (59h38m53.344039615s) [noise=true , address=79.127.219.51:4210 , piece_id=Y6E2NMMDH77RHEXCBBQXANJGQ44AXT36D466VWWT33YUQSU7N5GQ, seg_pos={0 0}, upload=57203]: Done uploading piece. err: ecclient: upload failed (node:1yCKELQfksAs1CLfacoYZgpHGzRM3UTGak3eWPu9ds3qn1a9G2, address:79.127.219.51:4210): protocol: storage node overloaded, request limit: 2000 Jul 26 23:14:22.263966941 (59h38m53.454265519s) [noise=true , address=79.127.219.47:4061 , piece_id=RIYCNHJ4AD7UYEQOL5E3OR3ICQOPU2M6ZBX2UHAQEUSJDIXWLHZQ, seg_pos={0 0}, upload=57203]: Done uploading piece. err: ecclient: upload failed (node:12f4AfVs2T4a8Yhd456WKdse7TZSjHBDULMR8myvo1ua9jiUrNu, address:79.127.219.47:4061): protocol: storage node overloaded, request limit: 2000 Jul 26 23:14:22.325532531 (59h38m53.515831116s) [upload=57203]: Next returned nil segment Jul 26 23:14:22.325558218 (59h38m53.515856808s) [upload=57203]: Waiting for error group managing segments... Jul 26 23:14:22.491494509 (59h38m53.681793096s) [noise=true , address=79.127.219.38:4004 , piece_id=WGKKECSZSPZ7Q6DYH3JVPUBBBKIIY7424BQPKXKP4DVAZBYCOV7Q, seg_pos={0 0}, upload=57203]: Done uploading piece. err: <nil> Jul 26 23:14:22.566967594 (59h38m53.757266176s) [noise=true , address=77.90.31.21:4905 , piece_id=7XFYGT35LLMUPAQ23M2RAIKNOL3PMK2WSHGJM5J2IBSVPE7TVKQA, seg_pos={0 0}, upload=57203]: Done uploading piece. err: <nil> Jul 26 23:14:22.950062316 (59h38m54.140360886s) [noise=true , address=79.127.205.234:4617 , piece_id=VVHD2E7ZJDOZRT3LXKLFCVJWUK7QOF7G4PB4FAPORJL477CHNXOQ, seg_pos={0 0}, upload=57203]: Done uploading piece. err: <nil> Jul 26 23:14:23.011763725 (59h38m54.202062312s) [noise=true , address=79.127.205.235:4688 , piece_id=HXH2MHZCSABY43FIG4DX7JT44NZVQFYSVOKNIOFUX3K6ODUH3VVQ, seg_pos={0 0}, upload=57203]: Done uploading piece. err: <nil> Jul 26 23:14:23.045618177 (59h38m54.235916747s) [noise=true , address=79.127.205.246:4253 , piece_id=L6EY3M6TXLLXU4HCS3KK77FWGSGBAPH5ZOT7HYYKQO3AY4CSXIBQ, seg_pos={0 0}, upload=57203]: Done uploading piece. err: <nil> Jul 26 23:14:23.834742606 (59h38m55.025041179s) [noise=true , address=45.91.116.11:4151 , piece_id=BMGPRP6VPHGNWMYPVTYP3CI32ELJLXQUDNIDGZ6AYITLLJTSMJDA, seg_pos={0 0}, upload=57203]: Done uploading piece. err: <nil> Jul 26 23:14:25.721378603 (59h38m56.911677183s) [noise=true , address=77.90.31.15:4731 , piece_id=ICUAWRYOTXPP2N4URYQZRTM3XG26BGF4RD3LI22ZU25K3ZNMIJJQ, seg_pos={0 0}, upload=57203]: Done uploading piece. err: <nil> Jul 26 23:14:48.261804768 (59h39m19.452103341s) [noise=true , address=104.36.82.53:4153 , piece_id=QFLQM6FZ32TC4NWIRVKBJVUEIDIUOGLZQLVSAZBXEDGKAF2KRXEA, seg_pos={0 0}, upload=57203]: Done uploading piece. err: <nil> Jul 26 23:14:52.841361018 (59h39m24.031659587s) [noise=true , address=104.36.82.54:4159 , piece_id=ULPEPD75YHJHD3J6GCDYQYIVEY7F3DZLYM2DAZN2ENUZDK36WFAQ, seg_pos={0 0}, upload=57203]: Done uploading piece. err: <nil> Jul 26 23:14:53.733775087 (59h39m24.92407366s) [noise=true , address=104.36.82.14:4256 , piece_id=BAXTEHXTOCSJO4TEEZBOKZ7KA6WKM3NDKZ752DNPHDJP5AWHWPFQ, seg_pos={0 0}, upload=57203]: Done uploading piece. err: <nil> Jul 26 23:15:11.626161927 (59h39m42.81646051s) [noise=true , address=104.36.82.47:4531 , piece_id=XA6POGWTT2ODVTX6CGIYLQ3YVBO6YY7V37FADOMANUHNPWCHAT7Q, seg_pos={0 0}, upload=57203]: Done uploading piece. err: <nil> Jul 26 23:15:18.450863629 (59h39m49.641162199s) [noise=true , address=104.36.82.46:4499 , piece_id=YFZNL5NTX37LBU52MPDMRKQNNYEM7YO5IZCHOLFSYOZGLYYW5USQ, seg_pos={0 0}, upload=57203]: Done uploading piece. err: <nil> Jul 26 23:15:21.255148513 (59h39m52.445447108s) [noise=true , address=104.36.82.55:4205 , piece_id=SSXSXX4LTXXVM5H4NNXQUZCZM4SWZMNI4PJ33JL7FC5ZFRXENSYQ, seg_pos={0 0}, upload=57203]: Done uploading piece. err: ecclient: upload failed (node:12UafxBVo9nuRVBcQmGE4gBP2pnXQ5YUrwHZdwPCeSBSBWCZKpQ, address:104.36.82.55:4205): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 23:15:21.255310999 (59h39m52.445609582s) [noise=true , address=104.36.82.56:4231 , piece_id=5K36ZSIRUNGLQFJEK7UFOENQ76WIX7XDYO6TRZGUMNDNTMHMMHOQ, seg_pos={0 0}, upload=57203]: Done uploading piece. err: ecclient: upload failed (node:129WWyyn7SrJFbFJ2kjPVhwDNRoYM7Kj449TZiZLzUbqpwK7qJ4, address:104.36.82.56:4231): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 23:15:21.255421137 (59h39m52.445719719s) [noise=true , address=77.90.31.12:4611 , piece_id=OSM4YJ2IY6SKRWVSTNEFD2EEWGV463SM7COP262LNCCGG3CW2TMA, seg_pos={0 0}, upload=57203]: Done uploading piece. err: ecclient: upload failed (node:1r7qdqiUuUqsKnTXdC3S1Ek84rsCPtjPoXbPugVHWBzJezv4W2, address:77.90.31.12:4611): protocol: write tcp 139.59.144.22:51298->77.90.31.12:4611: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 23:15:21.255498458 (59h39m52.445797042s) [noise=true , address=77.90.31.10:4537 , piece_id=LIMMBCSPR3Y6RIAD7B7XW6RI62SKTFDPJXC3YBWF74I2OY6VMTWA, seg_pos={0 0}, upload=57203]: Done uploading piece. err: ecclient: upload failed (node:12M5ger8kngGvMCD9c2qBxpFNCpLxka2fh8iWrf2XerxRiQEUja, address:77.90.31.10:4537): protocol: rpc: dial tcp 77.90.31.10:4537: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 23:15:21.255532812 (59h39m52.445831417s) [seg_pos={0 0}, upload=57203]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Jul 26 23:15:21.359331473 (59h39m52.549630042s) [upload=57203]: Done uploading segments... Jul 26 23:15:21.359361971 (59h39m52.549660561s) [upload=57203]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 1m0.106996774s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [..........................................X......XXXXXXXXXXX] duration: 30.113170531s FAIL (2025-07-26 23:14:54 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== SegmentBegin Jul 26 23:14:24.830790725 (59h38m56.021089308s) [seg_pos={0 0}, upload=57205]: Begin upload segment... Jul 26 23:14:24.830844678 (59h38m56.021143258s) [seg_pos={0 0}, upload=57205]: Done begin upload segment. Jul 26 23:14:24.830871717 (59h38m56.021170296s) [noise=true , address=45.91.116.13:4153 , piece_id=CFSZIBKGER7WKALHMFBTVLDBM3I4LABDIBZMOBBSWOAVKGBYCRWQ, seg_pos={0 0}, upload=57205]: Uploading piece... Jul 26 23:14:24.830901984 (59h38m56.021200565s) [noise=true , address=45.91.116.7:4010 , piece_id=2ZCSTC5XFE2RMAWTTIKJZQDI7ZNEMZ3YMDKWFWVDEFP5NEHBCGAA, seg_pos={0 0}, upload=57205]: Uploading piece... Jul 26 23:14:24.831248011 (59h38m56.021546591s) [noise=true , address=104.36.82.4:4804 , piece_id=UN5MK75ZZTOMEX6F6ZVVU2VYWFUI7BX2Q55UZQNR2BBKHT7AMHWA, seg_pos={0 0}, upload=57205]: Uploading piece... Jul 26 23:14:24.831640591 (59h38m56.021939176s) [noise=true , address=45.91.116.6:4973 , piece_id=ZSMLGOMN4UWOXRYQXU4AUZ7WD2Q3JEZLVC5A7CEC67PCUJ2PPMGQ, seg_pos={0 0}, upload=57205]: Uploading piece... Jul 26 23:14:24.831951504 (59h38m56.022250085s) [noise=true , address=77.90.31.19:4843 , piece_id=RPLGDS3OUD6XCTXLNLEQHIKI6GNU5OZMESUH2J3ODFRAASGM6WYA, seg_pos={0 0}, upload=57205]: Uploading piece... Jul 26 23:14:24.832297224 (59h38m56.022595806s) [noise=true , address=45.91.116.14:4189 , piece_id=G2LU4EQO4ST5EP6UKONG4LHNANMAJEUYGCO3CMYQDYNK4LCLIWPA, seg_pos={0 0}, upload=57205]: Uploading piece... Jul 26 23:14:24.832693687 (59h38m56.022992269s) [noise=true , address=104.36.82.6:4875 , piece_id=ZQGHLIFYF6DMLZJZTMCOED5PYN6PYOOZNGQEXIRU5YR5WFASSP3A, seg_pos={0 0}, upload=57205]: Uploading piece... Jul 26 23:14:24.832980902 (59h38m56.023279485s) [noise=true , address=77.90.31.18:4807 , piece_id=F6M5OBUIELNTRC767AITZQOG5L45IA37RTP3ZJWDIJZNJA7GIDIA, seg_pos={0 0}, upload=57205]: Uploading piece... Jul 26 23:14:24.833346757 (59h38m56.023645342s) [noise=true , address=104.36.82.9:4005 , piece_id=QHH6WS33OIUAFC7QBKGKMGKXH4ZBM3655ZYU45SGZR6POPPCIANA, seg_pos={0 0}, upload=57205]: Uploading piece... Jul 26 23:14:24.833653834 (59h38m56.023952413s) [noise=true , address=104.36.82.16:4294 , piece_id=NC36RCVDUMJ5OHL43QDLYLXD5QFXSIJBVJ7ACVZB7OXZPXM6WOUQ, seg_pos={0 0}, upload=57205]: Uploading piece... Jul 26 23:14:24.833961454 (59h38m56.024260034s) [noise=true , address=45.91.116.4:4937 , piece_id=WPQA6DMCV745QWBT7GXNQWDBNYKMZE7RCHX7SZCWEHJLS6STDXAA, seg_pos={0 0}, upload=57205]: Uploading piece... Jul 26 23:14:24.834253449 (59h38m56.024552034s) [noise=true , address=45.91.116.5:4299 , piece_id=U4H42HNOFULZR6IPQHBVLUF2DLJBJTCYP5M7AOUAODQPITJVQ2OQ, seg_pos={0 0}, upload=57205]: Uploading piece... Jul 26 23:14:24.834595024 (59h38m56.024893605s) [noise=true , address=104.36.82.14:4223 , piece_id=MVAFGTN4UD32THUOYW24OFJC6X6IDV2ZAHETNVKMEZ3ZVJ6CHTNA, seg_pos={0 0}, upload=57205]: Uploading piece... Jul 26 23:14:24.834905030 (59h38m56.025203618s) [noise=true , address=104.36.82.15:4258 , piece_id=J7ATMI62QG3JFY2IJKLYCTSLNVTMN6TE7CPA6BY6VRGKKNNBASFQ, seg_pos={0 0}, upload=57205]: Uploading piece... Jul 26 23:14:24.835144140 (59h38m56.025442722s) [noise=true , address=104.36.82.2:4732 , piece_id=IQDZKD5YUACTFSAM4KGLPITOLNSCC3AJB5NFLIRTZD73AGSXOZZQ, seg_pos={0 0}, upload=57205]: Uploading piece... Jul 26 23:14:24.835418241 (59h38m56.025716823s) [noise=true , address=104.36.82.12:4149 , piece_id=X2QJE5RAGH4M5KWZZLJUFOYR73WP5B3J5J56RWWGG7WCVW4UIPMQ, seg_pos={0 0}, upload=57205]: Uploading piece... Jul 26 23:14:25.454490865 (59h38m56.644789456s) [upload=57205]: Next returned nil segment Jul 26 23:14:25.454512085 (59h38m56.644810675s) [upload=57205]: Waiting for error group managing segments... Jul 26 23:14:25.589241603 (59h38m56.779540203s) [noise=true , address=45.91.116.5:4299 , piece_id=U4H42HNOFULZR6IPQHBVLUF2DLJBJTCYP5M7AOUAODQPITJVQ2OQ, seg_pos={0 0}, upload=57205]: Done uploading piece. err: <nil> Jul 26 23:14:25.590539497 (59h38m56.78083808s) [noise=true , address=45.91.116.4:4937 , piece_id=WPQA6DMCV745QWBT7GXNQWDBNYKMZE7RCHX7SZCWEHJLS6STDXAA, seg_pos={0 0}, upload=57205]: Done uploading piece. err: <nil> Jul 26 23:14:25.595558897 (59h38m56.785857463s) [noise=true , address=45.91.116.13:4153 , piece_id=CFSZIBKGER7WKALHMFBTVLDBM3I4LABDIBZMOBBSWOAVKGBYCRWQ, seg_pos={0 0}, upload=57205]: Done uploading piece. err: <nil> Jul 26 23:14:25.605715448 (59h38m56.796014051s) [noise=true , address=45.91.116.6:4973 , piece_id=ZSMLGOMN4UWOXRYQXU4AUZ7WD2Q3JEZLVC5A7CEC67PCUJ2PPMGQ, seg_pos={0 0}, upload=57205]: Done uploading piece. err: <nil> Jul 26 23:14:25.611136467 (59h38m56.80143504s) [noise=true , address=45.91.116.14:4189 , piece_id=G2LU4EQO4ST5EP6UKONG4LHNANMAJEUYGCO3CMYQDYNK4LCLIWPA, seg_pos={0 0}, upload=57205]: Done uploading piece. err: <nil> Jul 26 23:14:25.611180875 (59h38m56.801479447s) [noise=true , address=45.91.116.7:4010 , piece_id=2ZCSTC5XFE2RMAWTTIKJZQDI7ZNEMZ3YMDKWFWVDEFP5NEHBCGAA, seg_pos={0 0}, upload=57205]: Done uploading piece. err: <nil> Jul 26 23:14:25.620720563 (59h38m56.811019143s) [noise=true , address=77.90.31.18:4807 , piece_id=F6M5OBUIELNTRC767AITZQOG5L45IA37RTP3ZJWDIJZNJA7GIDIA, seg_pos={0 0}, upload=57205]: Done uploading piece. err: <nil> Jul 26 23:14:27.001767620 (59h38m58.192066206s) [noise=true , address=77.90.31.19:4843 , piece_id=RPLGDS3OUD6XCTXLNLEQHIKI6GNU5OZMESUH2J3ODFRAASGM6WYA, seg_pos={0 0}, upload=57205]: Done uploading piece. err: <nil> Jul 26 23:14:52.869460101 (59h39m24.059758687s) [noise=true , address=104.36.82.14:4223 , piece_id=MVAFGTN4UD32THUOYW24OFJC6X6IDV2ZAHETNVKMEZ3ZVJ6CHTNA, seg_pos={0 0}, upload=57205]: Done uploading piece. err: <nil> Jul 26 23:14:54.441409458 (59h39m25.631708037s) [noise=true , address=104.36.82.6:4875 , piece_id=ZQGHLIFYF6DMLZJZTMCOED5PYN6PYOOZNGQEXIRU5YR5WFASSP3A, seg_pos={0 0}, upload=57205]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:104.36.82.6:4875): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 23:14:54.441610847 (59h39m25.631909426s) [noise=true , address=104.36.82.2:4732 , piece_id=IQDZKD5YUACTFSAM4KGLPITOLNSCC3AJB5NFLIRTZD73AGSXOZZQ, seg_pos={0 0}, upload=57205]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: write tcp 139.59.144.22:34314->104.36.82.2:4732: use of closed network connection; write tcp 139.59.144.22:34314->104.36.82.2:4732: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34314->104.36.82.2:4732: use of closed network connection Jul 26 23:14:54.441694004 (59h39m25.631992583s) [noise=true , address=104.36.82.16:4294 , piece_id=NC36RCVDUMJ5OHL43QDLYLXD5QFXSIJBVJ7ACVZB7OXZPXM6WOUQ, seg_pos={0 0}, upload=57205]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:104.36.82.16:4294): protocol: write tcp 139.59.144.22:57340->104.36.82.16:4294: use of closed network connection; write tcp 139.59.144.22:57340->104.36.82.16:4294: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57340->104.36.82.16:4294: use of closed network connection Jul 26 23:14:54.441740694 (59h39m25.632039274s) [noise=true , address=104.36.82.12:4149 , piece_id=X2QJE5RAGH4M5KWZZLJUFOYR73WP5B3J5J56RWWGG7WCVW4UIPMQ, seg_pos={0 0}, upload=57205]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:104.36.82.12:4149): protocol: write tcp 139.59.144.22:60136->104.36.82.12:4149: use of closed network connection; write tcp 139.59.144.22:60136->104.36.82.12:4149: use of closed network connection Jul 26 23:14:54.441783075 (59h39m25.632081653s) [noise=true , address=104.36.82.4:4804 , piece_id=UN5MK75ZZTOMEX6F6ZVVU2VYWFUI7BX2Q55UZQNR2BBKHT7AMHWA, seg_pos={0 0}, upload=57205]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 139.59.144.22:56108->104.36.82.4:4804: use of closed network connection; write tcp 139.59.144.22:56108->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:56108->104.36.82.4:4804: use of closed network connection Jul 26 23:14:54.441820375 (59h39m25.632118956s) [noise=true , address=104.36.82.15:4258 , piece_id=J7ATMI62QG3JFY2IJKLYCTSLNVTMN6TE7CPA6BY6VRGKKNNBASFQ, seg_pos={0 0}, upload=57205]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:104.36.82.15:4258): protocol: write tcp 139.59.144.22:52584->104.36.82.15:4258: use of closed network connection; write tcp 139.59.144.22:52584->104.36.82.15:4258: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52584->104.36.82.15:4258: use of closed network connection Jul 26 23:14:54.441884632 (59h39m25.632183212s) [noise=true , address=104.36.82.9:4005 , piece_id=QHH6WS33OIUAFC7QBKGKMGKXH4ZBM3655ZYU45SGZR6POPPCIANA, seg_pos={0 0}, upload=57205]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: write tcp 139.59.144.22:56984->104.36.82.9:4005: use of closed network connection; write tcp 139.59.144.22:56984->104.36.82.9:4005: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:56984->104.36.82.9:4005: use of closed network connection Jul 26 23:14:54.441903682 (59h39m25.632202264s) [seg_pos={0 0}, upload=57205]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jul 26 23:14:54.553012911 (59h39m25.743311478s) [upload=57205]: Done uploading segments... Jul 26 23:14:54.553049336 (59h39m25.743347914s) [upload=57205]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 30.113200196s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 14 out of 60 is failed (threshold=3) [......................X..................X.....XXXXXX.XXXXXX] duration: 30.157824081s FAIL (2025-07-26 23:14:45 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 9) Duration: 30.157855043s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.................................................X.XX...XX..] duration: 15.77128649s FAIL (2025-07-26 23:14:45 UTC) orbiter warning: context deadline exceeded Duration: 15.77131101s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 15 out of 60 is failed (threshold=3) [X...X.XX...X..X.XX......X.X.......X.......X........XX....X..] duration: 42.854511389s FAIL (2025-07-26 23:14:36 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 42.854537079s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 39 out of 60 is failed (threshold=3) [...XXXXX.XXXX.XXXXX..XXX.....XXX.X.X.XX.X.XXXXXXXXX.XX..X.XX] duration: 1m0.106076152s FAIL (2025-07-26 23:14:21 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Done starting upload Jul 26 23:13:21.291672382 (59h37m52.481970954s) [upload=57188]: Uploading segments... Jul 26 23:13:21.291723103 (59h37m52.482021683s) [upload=57188]: Got next segment. Inline: false Jul 26 23:13:21.291762187 (59h37m52.482060756s) [upload=57188]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 23:13:21.291767808 (59h37m52.482066388s) [upload=57188]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 23:13:21.489144152 (59h37m52.679442731s) [seg_pos={0 0}, upload=57188]: Begin upload segment... Jul 26 23:13:21.489211083 (59h37m52.679509652s) [seg_pos={0 0}, upload=57188]: Done begin upload segment. Jul 26 23:13:21.489247541 (59h37m52.679546134s) [noise=true , address=79.127.219.51:4210 , piece_id=YXRR5RYJ45BAYSALBYZ67RXIY4GRN3S5XZICSHY7EJ56Q4Y446DQ, seg_pos={0 0}, upload=57188]: Uploading piece... Jul 26 23:13:21.489707813 (59h37m52.680006394s) [noise=true , address=77.90.31.16:4743 , piece_id=UZSQRPJGF43ODO6JSERIJEHOCQ7TH54M7MNX3VC2UL4ZFEZX5RUQ, seg_pos={0 0}, upload=57188]: Uploading piece... Jul 26 23:13:21.490249226 (59h37m52.680547806s) [noise=true , address=77.90.31.15:4720 , piece_id=ZYS5Y3SE7URUS63VOHW3WHHPY634BDJGTVNKFU4DUUINN63VJOBQ, seg_pos={0 0}, upload=57188]: Uploading piece... Jul 26 23:13:21.490939903 (59h37m52.681238482s) [noise=true , address=79.127.201.215:4358 , piece_id=65MOHI5DN6YFXQRXZUPJKM5EBWKI7G6DYTEH75QOYPM2K7BJUGFQ, seg_pos={0 0}, upload=57188]: Uploading piece... Jul 26 23:13:21.491194982 (59h37m52.681493561s) [noise=true , address=104.36.82.46:4486 , piece_id=RTJGZ3UXPBOHSXRDXXAER5II73K5QXV54QDCFGEOAUWQH5DZRNPA, seg_pos={0 0}, upload=57188]: Uploading piece... Jul 26 23:13:21.492014436 (59h37m52.682313026s) [noise=true , address=79.127.219.38:4017 , piece_id=VND3MTPKVO276T6QKO22VKAMCXP22E6DVI5TQCYVTK3YFNEPCXZQ, seg_pos={0 0}, upload=57188]: Uploading piece... Jul 26 23:13:21.492092033 (59h37m52.682390613s) [noise=true , address=79.127.219.48:4090 , piece_id=BNE6VVV4D2QHVH2ALFN7VGLJUD4DMP7HX74ZKXNYKCUOK4DGCSLQ, seg_pos={0 0}, upload=57188]: Uploading piece... Jul 26 23:13:21.492985657 (59h37m52.683284238s) [noise=true , address=104.36.82.56:4240 , piece_id=PCGSIKYTFHG3HKVVWHEXYJSRRXISZJC3RUFETJY4Z2MPEYETEIFQ, seg_pos={0 0}, upload=57188]: Uploading piece... Jul 26 23:13:21.493272141 (59h37m52.683570723s) [noise=true , address=79.127.205.235:4687 , piece_id=ZAZZ2S2KBIZAOXVV7ASBM5ZFOIQ5ECMJYUTCB4M2LYLXMGK64WKQ, seg_pos={0 0}, upload=57188]: Uploading piece... Jul 26 23:13:21.493641532 (59h37m52.683940123s) [noise=true , address=77.90.31.11:4559 , piece_id=WIUQKFFYDCNGLV4JODLXC4NGOMLPTPNWC4IKOCMQK4AA5HLCI6BA, seg_pos={0 0}, upload=57188]: Uploading piece... Jul 26 23:13:21.494105967 (59h37m52.684404547s) [noise=true , address=156.146.43.225:4448 , piece_id=IXVM5RQCI3UG5SZ2NYVONP7PZVXBAAEGNY6CITDXMWYIHDLYAL6Q, seg_pos={0 0}, upload=57188]: Uploading piece... Jul 26 23:13:21.494455250 (59h37m52.68475383s) [noise=true , address=79.127.205.233:4634 , piece_id=2SJBYKTRMFTX77SSO5AK4JXV6CRNPWER7LC6U3IMSWLIZGK2VSZA, seg_pos={0 0}, upload=57188]: Uploading piece... Jul 26 23:13:21.494841309 (59h37m52.68513989s) [noise=true , address=77.90.31.10:4538 , piece_id=HF2HXZD3CTHBTCQBQRLXE6JHNETM4CZAWFG44F2MWLTPWC7TNM6Q, seg_pos={0 0}, upload=57188]: Uploading piece... Jul 26 23:13:21.495235648 (59h37m52.685534226s) [noise=true , address=104.36.82.54:4166 , piece_id=FTFNTBNCRZOM42PQNMIPMXDNEDU2WF2RVEIKJA7ZOD2ECW4TNTRQ, seg_pos={0 0}, upload=57188]: Uploading piece... Jul 26 23:13:21.495428745 (59h37m52.685727324s) [noise=true , address=104.36.82.7:4924 , piece_id=LGMBLZMMOWV3QK2NWQWVSVDNA3IVOID77VFTLGTM4TTLXILCA3QQ, seg_pos={0 0}, upload=57188]: Uploading piece... Jul 26 23:13:21.495820291 (59h37m52.686118873s) [noise=true , address=104.36.82.47:4518 , piece_id=SOW5GPNBK66P63OR3WRSDMARLWTQVM4UN6MBUTNH7TATQS6BI4MA, seg_pos={0 0}, upload=57188]: Uploading piece... Jul 26 23:13:21.498287708 (59h37m52.68858629s) [noise=true , address=79.127.219.51:4212 , piece_id=RQKMY4WWDGGTA5L7TAKH7X45QWBPND2PMRUVOJDEXXEUOPUVRNPA, seg_pos={0 0}, upload=57188]: Uploading piece... Jul 26 23:13:21.499244185 (59h37m52.689542765s) [noise=true , address=104.36.82.52:4114 , piece_id=WKVTIFQHL2HDOHD35O2RJYIN77PZTZ6QO6SILB5EU2TFPXU53WUQ, seg_pos={0 0}, upload=57188]: Uploading piece... Jul 26 23:13:22.066733168 (59h37m53.257031737s) [upload=57188]: Next returned nil segment Jul 26 23:13:22.066761426 (59h37m53.257060012s) [upload=57188]: Waiting for error group managing segments... Jul 26 23:13:22.249219315 (59h37m53.439517886s) [noise=true , address=79.127.219.48:4090 , piece_id=BNE6VVV4D2QHVH2ALFN7VGLJUD4DMP7HX74ZKXNYKCUOK4DGCSLQ, seg_pos={0 0}, upload=57188]: Done uploading piece. err: <nil> Jul 26 23:13:22.273133388 (59h37m53.463431974s) [noise=true , address=77.90.31.11:4559 , piece_id=WIUQKFFYDCNGLV4JODLXC4NGOMLPTPNWC4IKOCMQK4AA5HLCI6BA, seg_pos={0 0}, upload=57188]: Done uploading piece. err: <nil> Jul 26 23:13:22.284354560 (59h37m53.47465314s) [noise=true , address=77.90.31.16:4743 , piece_id=UZSQRPJGF43ODO6JSERIJEHOCQ7TH54M7MNX3VC2UL4ZFEZX5RUQ, seg_pos={0 0}, upload=57188]: Done uploading piece. err: <nil> Jul 26 23:13:22.440110685 (59h37m53.630409258s) [noise=true , address=79.127.219.38:4017 , piece_id=VND3MTPKVO276T6QKO22VKAMCXP22E6DVI5TQCYVTK3YFNEPCXZQ, seg_pos={0 0}, upload=57188]: Done uploading piece. err: <nil> Jul 26 23:13:22.791376922 (59h37m53.981675506s) [noise=true , address=104.36.82.54:4166 , piece_id=FTFNTBNCRZOM42PQNMIPMXDNEDU2WF2RVEIKJA7ZOD2ECW4TNTRQ, seg_pos={0 0}, upload=57188]: Done uploading piece. err: ecclient: upload failed (node:1BV7mWGAwKMXzmZBc3qAQFUXYyWtgQC7QEfG54YJgyJ4U3tFiv, address:104.36.82.54:4166): protocol: write tcp 139.59.144.22:37068->104.36.82.54:4166: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 23:13:22.837382452 (59h37m54.027681021s) [noise=true , address=79.127.205.233:4634 , piece_id=2SJBYKTRMFTX77SSO5AK4JXV6CRNPWER7LC6U3IMSWLIZGK2VSZA, seg_pos={0 0}, upload=57188]: Done uploading piece. err: <nil> Jul 26 23:13:23.460821670 (59h37m54.651120243s) [noise=true , address=79.127.205.235:4687 , piece_id=ZAZZ2S2KBIZAOXVV7ASBM5ZFOIQ5ECMJYUTCB4M2LYLXMGK64WKQ, seg_pos={0 0}, upload=57188]: Done uploading piece. err: <nil> Jul 26 23:13:23.811081977 (59h37m55.00138055s) [noise=true , address=79.127.219.51:4210 , piece_id=YXRR5RYJ45BAYSALBYZ67RXIY4GRN3S5XZICSHY7EJ56Q4Y446DQ, seg_pos={0 0}, upload=57188]: Done uploading piece. err: <nil> Jul 26 23:13:25.950196556 (59h37m57.140495145s) [noise=true , address=156.146.43.225:4448 , piece_id=IXVM5RQCI3UG5SZ2NYVONP7PZVXBAAEGNY6CITDXMWYIHDLYAL6Q, seg_pos={0 0}, upload=57188]: Done uploading piece. err: <nil> Jul 26 23:13:26.424974311 (59h37m57.615272879s) [noise=true , address=79.127.219.51:4212 , piece_id=RQKMY4WWDGGTA5L7TAKH7X45QWBPND2PMRUVOJDEXXEUOPUVRNPA, seg_pos={0 0}, upload=57188]: Done uploading piece. err: <nil> Jul 26 23:13:28.877586747 (59h38m0.067885328s) [noise=true , address=77.90.31.15:4720 , piece_id=ZYS5Y3SE7URUS63VOHW3WHHPY634BDJGTVNKFU4DUUINN63VJOBQ, seg_pos={0 0}, upload=57188]: Done uploading piece. err: <nil> Jul 26 23:13:53.821395037 (59h38m25.011693618s) [noise=true , address=104.36.82.52:4114 , piece_id=WKVTIFQHL2HDOHD35O2RJYIN77PZTZ6QO6SILB5EU2TFPXU53WUQ, seg_pos={0 0}, upload=57188]: Done uploading piece. err: <nil> Jul 26 23:14:01.694681167 (59h38m32.884979754s) [noise=true , address=104.36.82.56:4240 , piece_id=PCGSIKYTFHG3HKVVWHEXYJSRRXISZJC3RUFETJY4Z2MPEYETEIFQ, seg_pos={0 0}, upload=57188]: Done uploading piece. err: <nil> Jul 26 23:14:01.747246397 (59h38m32.937544986s) [noise=true , address=79.127.201.215:4358 , piece_id=65MOHI5DN6YFXQRXZUPJKM5EBWKI7G6DYTEH75QOYPM2K7BJUGFQ, seg_pos={0 0}, upload=57188]: Done uploading piece. err: <nil> Jul 26 23:14:21.008244357 (59h38m52.198542937s) [noise=true , address=104.36.82.46:4486 , piece_id=RTJGZ3UXPBOHSXRDXXAER5II73K5QXV54QDCFGEOAUWQH5DZRNPA, seg_pos={0 0}, upload=57188]: Done uploading piece. err: ecclient: upload failed (node:1fbs2btcbiu2yYGrvxNZe8sNSqDnNPG1Bmfyhcew86FCFLKAic, address:104.36.82.46:4486): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 23:14:21.008280451 (59h38m52.19857903s) [noise=true , address=104.36.82.47:4518 , piece_id=SOW5GPNBK66P63OR3WRSDMARLWTQVM4UN6MBUTNH7TATQS6BI4MA, seg_pos={0 0}, upload=57188]: Done uploading piece. err: ecclient: upload failed (node:17TZ9JLdCivFF46guhSTW6hcoUmTz5uo5wvLkDqC4BGmpm9eLb, address:104.36.82.47:4518): protocol: write tcp 139.59.144.22:59748->104.36.82.47:4518: use of closed network connection; write tcp 139.59.144.22:59748->104.36.82.47:4518: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59748->104.36.82.47:4518: use of closed network connection Jul 26 23:14:21.008559764 (59h38m52.198858343s) [noise=true , address=104.36.82.7:4924 , piece_id=LGMBLZMMOWV3QK2NWQWVSVDNA3IVOID77VFTLGTM4TTLXILCA3QQ, seg_pos={0 0}, upload=57188]: Done uploading piece. err: ecclient: upload failed (node:12mXBUcQEzsNgQ1YQzU9ezpZigBEHHhPttBac2G1Xrjks3SwM6z, address:104.36.82.7:4924): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 23:14:21.008606715 (59h38m52.198905297s) [noise=true , address=77.90.31.10:4538 , piece_id=HF2HXZD3CTHBTCQBQRLXE6JHNETM4CZAWFG44F2MWLTPWC7TNM6Q, seg_pos={0 0}, upload=57188]: Done uploading piece. err: ecclient: upload failed (node:12kRwLj3eRSu1a8oRSnUF5NuG6sJ9R9qMmFT89CHXMWB11WQD1N, address:77.90.31.10:4538): protocol: context deadline exceeded; context deadline exceeded Jul 26 23:14:21.008639789 (59h38m52.198938378s) [seg_pos={0 0}, upload=57188]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 26 23:14:21.111472092 (59h38m52.301770675s) [upload=57188]: Done uploading segments... Jul 26 23:14:21.111548378 (59h38m52.301846963s) [upload=57188]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1m0.106118681s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 13 out of 60 is failed (threshold=3) [.......................X..................X.....XXXXXX.XXXXX] duration: 30.574986358s FAIL (2025-07-26 23:14:14 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 10) Duration: 30.575005562s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [...........................................X......XXXXXXXXXX] duration: 30.124660911s FAIL (2025-07-26 23:14:05 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== ading piece... Jul 26 23:13:35.817795607 (59h38m7.008094177s) [noise=true , address=77.90.31.13:4626 , piece_id=EKC2NS3CODI3TIRKQ523GBVHAKTX47FU5OL62M3JTVOOIFC7PLMQ, seg_pos={0 0}, upload=57192]: Uploading piece... Jul 26 23:13:35.818074338 (59h38m7.008372909s) [seg_pos={0 0}, upload=57192]: Done begin upload segment. Jul 26 23:13:35.818136161 (59h38m7.008434735s) [noise=true , address=45.91.116.4:4939 , piece_id=IDLUBAUNLMD4SYXHWFIY6X5P7DKFU4UQDFSQMPODAFF6IOXZGRAQ, seg_pos={0 0}, upload=57192]: Uploading piece... Jul 26 23:13:35.818234407 (59h38m7.008532993s) [noise=true , address=45.91.116.4:4940 , piece_id=C4ILTXYESDOMNNTBAOZ2I65OAUELBNNAPVNHPN4FCKB53IRJLZZQ, seg_pos={0 0}, upload=57192]: Uploading piece... Jul 26 23:13:35.818539741 (59h38m7.008838312s) [noise=true , address=45.91.116.5:4298 , piece_id=LMJWHGKOFB3KAQNU2HZLQZP5LQYMH4JN7BREHK7UXAQLZOGQR2ZA, seg_pos={0 0}, upload=57192]: Uploading piece... Jul 26 23:13:35.818611523 (59h38m7.008910107s) [noise=true , address=45.91.116.14:4191 , piece_id=QLX5KHYN4IQ6V4J6PZNRRTILO6N7BMNGUIXQQHI3FOKHNEUSXH4Q, seg_pos={0 0}, upload=57192]: Uploading piece... Jul 26 23:13:35.818845559 (59h38m7.009144128s) [noise=true , address=104.36.82.2:4731 , piece_id=76LSPVXB5NIY3IOSR4L4C4AAMYSOXGZBEQIRUCIWF3KEWOAG27ZQ, seg_pos={0 0}, upload=57192]: Uploading piece... Jul 26 23:13:35.819190976 (59h38m7.009489561s) [noise=true , address=104.36.82.16:4294 , piece_id=HWAN72SOM7ZTY7XT7FARHBPOOVFK6UGRSFYWRISIMMFIAU7KZ4WA, seg_pos={0 0}, upload=57192]: Uploading piece... Jul 26 23:13:35.820198036 (59h38m7.010496617s) [noise=true , address=104.36.82.6:4873 , piece_id=6VOQ2L7MIIH7GCDLOUR6OYUBJLP3QR2ETILOY7KFDL45FIUCPGTQ, seg_pos={0 0}, upload=57192]: Uploading piece... Jul 26 23:13:35.822155766 (59h38m7.012454353s) [noise=true , address=104.36.82.15:4258 , piece_id=4T2IHI4IXDYXENKF56PA3XHKFI5K2TXFRBMP27MGCPJRYIOVPMTA, seg_pos={0 0}, upload=57192]: Uploading piece... Jul 26 23:13:35.822541922 (59h38m7.012840502s) [noise=true , address=77.90.31.13:4628 , piece_id=RUGMPQHYEWC2L7DCULYVWYXIMASS23C2AC4EUOG544IQJXEQISTQ, seg_pos={0 0}, upload=57192]: Uploading piece... Jul 26 23:13:35.822859215 (59h38m7.013157795s) [noise=true , address=104.36.82.6:4876 , piece_id=ZJN6VVJPC6ILEU23DAGBDNKGRIJCFCX5UY7SQEOQ3LKQDNUQ5WBA, seg_pos={0 0}, upload=57192]: Uploading piece... Jul 26 23:13:35.823128776 (59h38m7.013427358s) [noise=true , address=104.36.82.2:4730 , piece_id=SAH7R6LBTPD6HMK74227ZDXIND7VL4L2XIC23KW4EWYINBNHL4TQ, seg_pos={0 0}, upload=57192]: Uploading piece... Jul 26 23:13:35.823403174 (59h38m7.013701753s) [noise=true , address=45.91.116.8:4048 , piece_id=LFAYMNRYAJ5AMK3KRAWB5JOX4FW337JHNDDWHKUJKBJYLGKFB7WQ, seg_pos={0 0}, upload=57192]: Uploading piece... Jul 26 23:13:35.823689821 (59h38m7.013988407s) [noise=true , address=104.36.82.4:4803 , piece_id=UKRWSFZGGVVKV4BBQAV3TBBDZHN7UHZDRRWDXXEF3LML343HW34A, seg_pos={0 0}, upload=57192]: Uploading piece... Jul 26 23:13:35.824052564 (59h38m7.014351153s) [noise=true , address=45.91.116.10:4083 , piece_id=RY7YKRUZJHWTPE5CN6FEIPK3XAAA7CGWKYTFYNIWKEEFVPYYJKRA, seg_pos={0 0}, upload=57192]: Uploading piece... Jul 26 23:13:36.424197028 (59h38m7.614495598s) [upload=57192]: Next returned nil segment Jul 26 23:13:36.424215287 (59h38m7.614513857s) [upload=57192]: Waiting for error group managing segments... Jul 26 23:13:36.562112092 (59h38m7.752410672s) [noise=true , address=45.91.116.4:4940 , piece_id=C4ILTXYESDOMNNTBAOZ2I65OAUELBNNAPVNHPN4FCKB53IRJLZZQ, seg_pos={0 0}, upload=57192]: Done uploading piece. err: <nil> Jul 26 23:13:36.565309274 (59h38m7.755607855s) [noise=true , address=45.91.116.5:4298 , piece_id=LMJWHGKOFB3KAQNU2HZLQZP5LQYMH4JN7BREHK7UXAQLZOGQR2ZA, seg_pos={0 0}, upload=57192]: Done uploading piece. err: <nil> Jul 26 23:13:36.565446388 (59h38m7.755744966s) [noise=true , address=45.91.116.14:4191 , piece_id=QLX5KHYN4IQ6V4J6PZNRRTILO6N7BMNGUIXQQHI3FOKHNEUSXH4Q, seg_pos={0 0}, upload=57192]: Done uploading piece. err: <nil> Jul 26 23:13:36.571746001 (59h38m7.762044574s) [noise=true , address=45.91.116.4:4939 , piece_id=IDLUBAUNLMD4SYXHWFIY6X5P7DKFU4UQDFSQMPODAFF6IOXZGRAQ, seg_pos={0 0}, upload=57192]: Done uploading piece. err: <nil> Jul 26 23:13:36.573079818 (59h38m7.763378387s) [noise=true , address=45.91.116.8:4048 , piece_id=LFAYMNRYAJ5AMK3KRAWB5JOX4FW337JHNDDWHKUJKBJYLGKFB7WQ, seg_pos={0 0}, upload=57192]: Done uploading piece. err: <nil> Jul 26 23:13:36.575031647 (59h38m7.765330221s) [noise=true , address=45.91.116.10:4083 , piece_id=RY7YKRUZJHWTPE5CN6FEIPK3XAAA7CGWKYTFYNIWKEEFVPYYJKRA, seg_pos={0 0}, upload=57192]: Done uploading piece. err: <nil> Jul 26 23:13:36.576880247 (59h38m7.767178819s) [noise=true , address=45.91.116.11:4119 , piece_id=VPMO4MHWYNYNDWULOW4NODUUXVVVVSXBICD6IXBY72OYLMZI3CMQ, seg_pos={0 0}, upload=57192]: Done uploading piece. err: <nil> Jul 26 23:13:36.671346303 (59h38m7.861644891s) [noise=true , address=77.90.31.13:4628 , piece_id=RUGMPQHYEWC2L7DCULYVWYXIMASS23C2AC4EUOG544IQJXEQISTQ, seg_pos={0 0}, upload=57192]: Done uploading piece. err: <nil> Jul 26 23:13:36.673944325 (59h38m7.864242919s) [noise=true , address=77.90.31.13:4626 , piece_id=EKC2NS3CODI3TIRKQ523GBVHAKTX47FU5OL62M3JTVOOIFC7PLMQ, seg_pos={0 0}, upload=57192]: Done uploading piece. err: <nil> Jul 26 23:14:05.334194217 (59h38m36.524492797s) [noise=true , address=104.36.82.16:4294 , piece_id=HWAN72SOM7ZTY7XT7FARHBPOOVFK6UGRSFYWRISIMMFIAU7KZ4WA, seg_pos={0 0}, upload=57192]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:104.36.82.16:4294): protocol: write tcp 139.59.144.22:42542->104.36.82.16:4294: use of closed network connection; write tcp 139.59.144.22:42542->104.36.82.16:4294: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42542->104.36.82.16:4294: use of closed network connection Jul 26 23:14:05.334515960 (59h38m36.524814549s) [noise=true , address=104.36.82.6:4873 , piece_id=6VOQ2L7MIIH7GCDLOUR6OYUBJLP3QR2ETILOY7KFDL45FIUCPGTQ, seg_pos={0 0}, upload=57192]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:104.36.82.6:4873): protocol: write tcp 139.59.144.22:47418->104.36.82.6:4873: use of closed network connection; write tcp 139.59.144.22:47418->104.36.82.6:4873: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47418->104.36.82.6:4873: use of closed network connection Jul 26 23:14:05.334550420 (59h38m36.524849s) [noise=true , address=104.36.82.15:4258 , piece_id=4T2IHI4IXDYXENKF56PA3XHKFI5K2TXFRBMP27MGCPJRYIOVPMTA, seg_pos={0 0}, upload=57192]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:104.36.82.15:4258): protocol: write tcp 139.59.144.22:36126->104.36.82.15:4258: use of closed network connection; write tcp 139.59.144.22:36126->104.36.82.15:4258: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36126->104.36.82.15:4258: use of closed network connection Jul 26 23:14:05.334637421 (59h38m36.524936006s) [noise=true , address=104.36.82.2:4730 , piece_id=SAH7R6LBTPD6HMK74227ZDXIND7VL4L2XIC23KW4EWYINBNHL4TQ, seg_pos={0 0}, upload=57192]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: write tcp 139.59.144.22:37120->104.36.82.2:4730: use of closed network connection; write tcp 139.59.144.22:37120->104.36.82.2:4730: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37120->104.36.82.2:4730: use of closed network connection Jul 26 23:14:05.334721249 (59h38m36.525019895s) [noise=true , address=104.36.82.2:4731 , piece_id=76LSPVXB5NIY3IOSR4L4C4AAMYSOXGZBEQIRUCIWF3KEWOAG27ZQ, seg_pos={0 0}, upload=57192]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:104.36.82.2:4731): protocol: write tcp 139.59.144.22:46320->104.36.82.2:4731: use of closed network connection; write tcp 139.59.144.22:46320->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46320->104.36.82.2:4731: use of closed network connection Jul 26 23:14:05.334730394 (59h38m36.525028978s) [noise=true , address=104.36.82.6:4876 , piece_id=ZJN6VVJPC6ILEU23DAGBDNKGRIJCFCX5UY7SQEOQ3LKQDNUQ5WBA, seg_pos={0 0}, upload=57192]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: write tcp 139.59.144.22:33788->104.36.82.6:4876: use of closed network connection; write tcp 139.59.144.22:33788->104.36.82.6:4876: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33788->104.36.82.6:4876: use of closed network connection Jul 26 23:14:05.334741246 (59h38m36.525039811s) [noise=true , address=104.36.82.4:4803 , piece_id=UKRWSFZGGVVKV4BBQAV3TBBDZHN7UHZDRRWDXXEF3LML343HW34A, seg_pos={0 0}, upload=57192]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: write tcp 139.59.144.22:37826->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:37826->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37826->104.36.82.4:4803: use of closed network connection Jul 26 23:14:05.334792588 (59h38m36.525091176s) [seg_pos={0 0}, upload=57192]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jul 26 23:14:05.457005526 (59h38m36.64730411s) [upload=57192]: Done uploading segments... Jul 26 23:14:05.457055767 (59h38m36.647354344s) [upload=57192]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 30.124680833s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [..................................................X.XX...XX.] duration: 6.050675694s FAIL (2025-07-26 23:14:01 UTC) orbiter warning: context deadline exceeded Duration: 6.050725281s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 12 out of 60 is failed (threshold=3) [........................X..................X.....XXXXXX.XXXX] duration: 30.344099521s FAIL (2025-07-26 23:13:44 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 7) Duration: 30.344125425s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [............................................X......XXXXXXXXX] duration: 30.112694947s FAIL (2025-07-26 23:13:35 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== KLWE3X6477F3UIA, seg_pos={0 0}, upload=57185]: Uploading piece... Jul 26 23:13:05.566045890 (59h37m36.75634448s) [seg_pos={0 0}, upload=57185]: Done begin upload segment. Jul 26 23:13:05.566112060 (59h37m36.75641065s) [noise=true , address=45.91.116.8:4045 , piece_id=B5BHDIWFBIYORU4M6TU4A5NUXFY4SIEGYETRNUXNTWUB7ARE4WFA, seg_pos={0 0}, upload=57185]: Uploading piece... Jul 26 23:13:05.566562383 (59h37m36.756860974s) [noise=true , address=104.36.82.6:4875 , piece_id=LXP6IXFZY32JNXD4SCXCG4B4GBWS4RNZY2JUJWZIIHRNBFOANR3A, seg_pos={0 0}, upload=57185]: Uploading piece... Jul 26 23:13:05.566953021 (59h37m36.75725161s) [noise=true , address=104.36.82.2:4730 , piece_id=WQEBB6UEZK6WPOHJPZKNUEHNDIJTGX2FPAKNM7FDPSDVQBOVSNWQ, seg_pos={0 0}, upload=57185]: Uploading piece... Jul 26 23:13:05.567259212 (59h37m36.757557796s) [noise=true , address=77.90.31.14:4661 , piece_id=5YPP25QR73R6BZMHXPQB2BM3OODJ5C43K673FVIOFH3QBQRXLBFA, seg_pos={0 0}, upload=57185]: Uploading piece... Jul 26 23:13:05.567544464 (59h37m36.757843044s) [noise=true , address=104.36.82.2:4731 , piece_id=2ACH7GM5L7PGPRPQQI5XOSZ3UO6K4QOFIFJ2QPOWOS3OGGJTHHLA, seg_pos={0 0}, upload=57185]: Uploading piece... Jul 26 23:13:05.569248093 (59h37m36.759546686s) [noise=true , address=104.36.82.6:4874 , piece_id=G72NY7CI67PQ2YREIVYX4NC5QHBU3IFMUIYOU4OA66GUZWQRXIYA, seg_pos={0 0}, upload=57185]: Uploading piece... Jul 26 23:13:05.569706736 (59h37m36.760005321s) [noise=true , address=45.91.116.7:4009 , piece_id=AIDEWOJYSMEPCYN7U5W5GOG2BLNOGHINS24K5ZXNCYLLC4MYM65Q, seg_pos={0 0}, upload=57185]: Uploading piece... Jul 26 23:13:05.570052108 (59h37m36.760350688s) [noise=true , address=104.36.82.8:4969 , piece_id=TPZQQA74ZQCOIFWXBESUUYPHK7ZD5W5KZBZ3XKCE6WUGW57SCUNQ, seg_pos={0 0}, upload=57185]: Uploading piece... Jul 26 23:13:05.570381843 (59h37m36.760680425s) [noise=true , address=45.91.116.6:4974 , piece_id=I5NELMANX3RDP3ZL2OEUDZZ7NQKTQPNNL5HPCBGVH7W7NZ6RBQSQ, seg_pos={0 0}, upload=57185]: Uploading piece... Jul 26 23:13:05.570749402 (59h37m36.761047987s) [noise=true , address=104.36.82.8:4972 , piece_id=AOQKYI6CM3CGGFDBKT2NTSYHG7S4GLFP7MNKOGDB5VTW6QR6S46A, seg_pos={0 0}, upload=57185]: Uploading piece... Jul 26 23:13:05.571143867 (59h37m36.761442449s) [noise=true , address=104.36.82.15:4259 , piece_id=F6M6GOYXQ2E6DEKKYBG2HTJJ6HZJX2SWZ7NOOSEFBBTOW2SPBOPQ, seg_pos={0 0}, upload=57185]: Uploading piece... Jul 26 23:13:05.571496079 (59h37m36.761794665s) [noise=true , address=45.91.116.7:4010 , piece_id=GIAO4I5VVG4XNXOPAVWN7UFINX2FVS2W2S24JGL7D77QBDZNQPJQ, seg_pos={0 0}, upload=57185]: Uploading piece... Jul 26 23:13:05.571875159 (59h37m36.762173748s) [noise=true , address=45.91.116.12:4227 , piece_id=HI4H6RYRLWU6KUPVES2A7YL4BTMRJUZHEF35YB77DMZNVIULRXEA, seg_pos={0 0}, upload=57185]: Uploading piece... Jul 26 23:13:05.572270855 (59h37m36.762569444s) [noise=true , address=104.36.82.11:4115 , piece_id=B5BSSASTZASMMQEAEN67S5GQIYSTEFVKVRI37WZDBJ6PKZTELNZQ, seg_pos={0 0}, upload=57185]: Uploading piece... Jul 26 23:13:05.572706830 (59h37m36.763005413s) [noise=true , address=104.36.82.9:4005 , piece_id=OICQFD5XMOT7F3NBW664RJ7JF2TIRUD5TK6Q3LIYEJ5R7ZLUGVNA, seg_pos={0 0}, upload=57185]: Uploading piece... Jul 26 23:13:06.180772736 (59h37m37.371071315s) [upload=57185]: Next returned nil segment Jul 26 23:13:06.180792749 (59h37m37.371091322s) [upload=57185]: Waiting for error group managing segments... Jul 26 23:13:06.316217592 (59h37m37.506516161s) [noise=true , address=45.91.116.12:4227 , piece_id=HI4H6RYRLWU6KUPVES2A7YL4BTMRJUZHEF35YB77DMZNVIULRXEA, seg_pos={0 0}, upload=57185]: Done uploading piece. err: <nil> Jul 26 23:13:06.316324578 (59h37m37.50662318s) [noise=true , address=45.91.116.8:4045 , piece_id=B5BHDIWFBIYORU4M6TU4A5NUXFY4SIEGYETRNUXNTWUB7ARE4WFA, seg_pos={0 0}, upload=57185]: Done uploading piece. err: <nil> Jul 26 23:13:06.317196657 (59h37m37.50749524s) [noise=true , address=77.90.31.14:4661 , piece_id=5YPP25QR73R6BZMHXPQB2BM3OODJ5C43K673FVIOFH3QBQRXLBFA, seg_pos={0 0}, upload=57185]: Done uploading piece. err: <nil> Jul 26 23:13:06.329426883 (59h37m37.519725458s) [noise=true , address=45.91.116.7:4009 , piece_id=AIDEWOJYSMEPCYN7U5W5GOG2BLNOGHINS24K5ZXNCYLLC4MYM65Q, seg_pos={0 0}, upload=57185]: Done uploading piece. err: <nil> Jul 26 23:13:06.333900974 (59h37m37.524199555s) [noise=true , address=45.91.116.6:4974 , piece_id=I5NELMANX3RDP3ZL2OEUDZZ7NQKTQPNNL5HPCBGVH7W7NZ6RBQSQ, seg_pos={0 0}, upload=57185]: Done uploading piece. err: <nil> Jul 26 23:13:06.337872056 (59h37m37.528170628s) [noise=true , address=45.91.116.7:4010 , piece_id=GIAO4I5VVG4XNXOPAVWN7UFINX2FVS2W2S24JGL7D77QBDZNQPJQ, seg_pos={0 0}, upload=57185]: Done uploading piece. err: <nil> Jul 26 23:13:06.349624765 (59h37m37.539923337s) [noise=true , address=104.36.82.8:4972 , piece_id=AOQKYI6CM3CGGFDBKT2NTSYHG7S4GLFP7MNKOGDB5VTW6QR6S46A, seg_pos={0 0}, upload=57185]: Done uploading piece. err: <nil> Jul 26 23:13:06.384668648 (59h37m37.574967212s) [noise=true , address=104.36.82.8:4969 , piece_id=TPZQQA74ZQCOIFWXBESUUYPHK7ZD5W5KZBZ3XKCE6WUGW57SCUNQ, seg_pos={0 0}, upload=57185]: Done uploading piece. err: <nil> Jul 26 23:13:35.060936404 (59h38m6.251234974s) [noise=true , address=104.36.82.2:4730 , piece_id=WQEBB6UEZK6WPOHJPZKNUEHNDIJTGX2FPAKNM7FDPSDVQBOVSNWQ, seg_pos={0 0}, upload=57185]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 23:13:35.061009971 (59h38m6.25130855s) [noise=true , address=104.36.82.15:4259 , piece_id=F6M6GOYXQ2E6DEKKYBG2HTJJ6HZJX2SWZ7NOOSEFBBTOW2SPBOPQ, seg_pos={0 0}, upload=57185]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:104.36.82.15:4259): protocol: write tcp 139.59.144.22:50932->104.36.82.15:4259: use of closed network connection; write tcp 139.59.144.22:50932->104.36.82.15:4259: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50932->104.36.82.15:4259: use of closed network connection Jul 26 23:13:35.061044521 (59h38m6.2513431s) [noise=true , address=104.36.82.2:4731 , piece_id=2ACH7GM5L7PGPRPQQI5XOSZ3UO6K4QOFIFJ2QPOWOS3OGGJTHHLA, seg_pos={0 0}, upload=57185]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:104.36.82.2:4731): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 23:13:35.061080292 (59h38m6.251378871s) [noise=true , address=104.36.82.6:4874 , piece_id=G72NY7CI67PQ2YREIVYX4NC5QHBU3IFMUIYOU4OA66GUZWQRXIYA, seg_pos={0 0}, upload=57185]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: write tcp 139.59.144.22:57508->104.36.82.6:4874: use of closed network connection; write tcp 139.59.144.22:57508->104.36.82.6:4874: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57508->104.36.82.6:4874: use of closed network connection Jul 26 23:13:35.061103590 (59h38m6.25140217s) [noise=true , address=104.36.82.11:4115 , piece_id=B5BSSASTZASMMQEAEN67S5GQIYSTEFVKVRI37WZDBJ6PKZTELNZQ, seg_pos={0 0}, upload=57185]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:104.36.82.11:4115): protocol: write tcp 139.59.144.22:55804->104.36.82.11:4115: use of closed network connection; write tcp 139.59.144.22:55804->104.36.82.11:4115: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55804->104.36.82.11:4115: use of closed network connection Jul 26 23:13:35.061153746 (59h38m6.251452325s) [noise=true , address=104.36.82.9:4005 , piece_id=OICQFD5XMOT7F3NBW664RJ7JF2TIRUD5TK6Q3LIYEJ5R7ZLUGVNA, seg_pos={0 0}, upload=57185]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: write tcp 139.59.144.22:34930->104.36.82.9:4005: use of closed network connection; write tcp 139.59.144.22:34930->104.36.82.9:4005: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34930->104.36.82.9:4005: use of closed network connection Jul 26 23:13:35.061163451 (59h38m6.251462023s) [noise=true , address=104.36.82.3:4765 , piece_id=JOXG2OPRTMYBNXSPGMDNNJDE3DQ7ER6D4TVBFKLWE3X6477F3UIA, seg_pos={0 0}, upload=57185]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:104.36.82.3:4765): protocol: write tcp 139.59.144.22:46244->104.36.82.3:4765: use of closed network connection; write tcp 139.59.144.22:46244->104.36.82.3:4765: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46244->104.36.82.3:4765: use of closed network connection Jul 26 23:13:35.061172126 (59h38m6.251470706s) [noise=true , address=104.36.82.6:4875 , piece_id=LXP6IXFZY32JNXD4SCXCG4B4GBWS4RNZY2JUJWZIIHRNBFOANR3A, seg_pos={0 0}, upload=57185]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:104.36.82.6:4875): protocol: write tcp 139.59.144.22:54992->104.36.82.6:4875: use of closed network connection; write tcp 139.59.144.22:54992->104.36.82.6:4875: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54992->104.36.82.6:4875: use of closed network connection Jul 26 23:13:35.061201889 (59h38m6.251500471s) [seg_pos={0 0}, upload=57185]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 26 23:13:35.171240260 (59h38m6.361538832s) [upload=57185]: Done uploading segments... Jul 26 23:13:35.171291941 (59h38m6.361590521s) [upload=57185]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 30.11277639s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [...................................................X.XX...XX] duration: 30.001323384s FAIL (2025-07-26 23:13:34 UTC) orbiter warning: context deadline exceeded Duration: 30.001344223s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 39 out of 60 is failed (threshold=3) [X...XXXXX.XXXX.XXXXX..XXX.....XXX.X.X.XX.X.XXXXXXXXX.XX..X.X] duration: 1m0.130003178s FAIL (2025-07-26 23:13:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 23s) [upload=57171]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 23:12:21.223683149 (59h36m52.413981731s) [seg_pos={0 0}, upload=57171]: Begin upload segment... Jul 26 23:12:21.223750382 (59h36m52.414048951s) [seg_pos={0 0}, upload=57171]: Done begin upload segment. Jul 26 23:12:21.223810455 (59h36m52.414109042s) [noise=true , address=104.36.82.3:4772 , piece_id=YX2F34OL6R75KA36IPGKHE77WBNOP5COKIV3ZRDXNLPOS4YASLGA, seg_pos={0 0}, upload=57171]: Uploading piece... Jul 26 23:12:21.224302627 (59h36m52.414601208s) [noise=true , address=77.90.31.21:4897 , piece_id=GEWGIT7Y32C67GNWA72J4APHEABYO6B762PJRELMG7UUDU5GMHNQ, seg_pos={0 0}, upload=57171]: Uploading piece... Jul 26 23:12:21.224523054 (59h36m52.414821635s) [noise=true , address=79.127.201.215:4347 , piece_id=27RYJGOMSKFDRRUAQZDIB5U7TNGK2EW6SUIMGVXTGZUCBI45VVEQ, seg_pos={0 0}, upload=57171]: Uploading piece... Jul 26 23:12:21.225233313 (59h36m52.415531896s) [noise=true , address=45.91.116.5:4310 , piece_id=PZMO45ANMNN73KWD5QUPB4C23MGVQB6CD2YKFZJ3PF53SBY2X6BA, seg_pos={0 0}, upload=57171]: Uploading piece... Jul 26 23:12:21.225977473 (59h36m52.416276057s) [noise=true , address=104.36.82.46:4496 , piece_id=MTSJK4Y5L4XUBPPW72FFLRE2FVOB4KOTDJ55JNJ7Y3S76IOI3T3A, seg_pos={0 0}, upload=57171]: Uploading piece... Jul 26 23:12:21.226041089 (59h36m52.416339677s) [noise=true , address=79.127.205.234:4620 , piece_id=MIEA5DWJSXJHVONRSU3KJJVI3S2TEAGAEMHEIAQ3NZRHA36DX5UQ, seg_pos={0 0}, upload=57171]: Uploading piece... Jul 26 23:12:21.226296373 (59h36m52.416594955s) [noise=true , address=77.90.31.10:4542 , piece_id=WRHU36OHNPTL7T7XQ4X2V7WKQYQOHUG7PMTW55UWOWOKRC2E4WTQ, seg_pos={0 0}, upload=57171]: Uploading piece... Jul 26 23:12:21.226361894 (59h36m52.416660476s) [noise=true , address=104.36.82.56:4240 , piece_id=6E6HAIF5YPUTN23V5SECBXMOOJBKVJB4ZTASX7344ZUGJBUR3DOQ, seg_pos={0 0}, upload=57171]: Uploading piece... Jul 26 23:12:21.226640461 (59h36m52.416939046s) [noise=true , address=64.49.13.140:4514 , piece_id=GHQP6M2V7T5SVD53UTEKKZ6F6R5YFR6D2JTMSNV5CBDPS6BMAC2A, seg_pos={0 0}, upload=57171]: Uploading piece... Jul 26 23:12:21.226671310 (59h36m52.416969895s) [noise=true , address=79.127.205.235:4657 , piece_id=64CJNWPNLTZELACEVTLRJY5D4E7OTIQRVUWGDEM63YCVB5C3JU7A, seg_pos={0 0}, upload=57171]: Uploading piece... Jul 26 23:12:21.226937010 (59h36m52.41723559s) [noise=true , address=104.36.82.53:4137 , piece_id=AQXBHSUCKKU7DHCO3XTELNI7EHAVMF4LEE6MEEBGKZ5OQUNAATKQ, seg_pos={0 0}, upload=57171]: Uploading piece... Jul 26 23:12:21.226966968 (59h36m52.417265612s) [noise=true , address=79.127.219.38:4018 , piece_id=QQBAQXQOQBGBM3L5UAUSSUZBV3DW2432N22MVK3TBZPAKSBFF56Q, seg_pos={0 0}, upload=57171]: Uploading piece... Jul 26 23:12:21.227211066 (59h36m52.417509648s) [noise=true , address=104.36.82.47:4534 , piece_id=6KC4R6M7ZFP4TL2DKGX33FOF2NBGUKRYZLJZ4OT2FR6UV4IF2Z7A, seg_pos={0 0}, upload=57171]: Uploading piece... Jul 26 23:12:21.227220879 (59h36m52.417519457s) [noise=true , address=77.90.31.17:4786 , piece_id=4ORKEXIRMAQGEFHMZ6OYHN5ZFVXW7GGYQSJF2PBUC7QLIRQ2MKTQ, seg_pos={0 0}, upload=57171]: Uploading piece... Jul 26 23:12:21.227454516 (59h36m52.417753098s) [noise=true , address=104.36.82.54:4189 , piece_id=SAJBNIPKDGSIIYO6AWVSKDRG3RACUFJCQRB2GANGNTZNLHTQT5BQ, seg_pos={0 0}, upload=57171]: Uploading piece... Jul 26 23:12:21.227511563 (59h36m52.417810146s) [noise=true , address=104.36.82.52:4099 , piece_id=X77KOOXQECYGS45HOXDFXN2U5PZXSGHNHN2FXQ46EMQAGWG26OPA, seg_pos={0 0}, upload=57171]: Uploading piece... Jul 26 23:12:21.227769792 (59h36m52.418068374s) [noise=true , address=104.36.82.55:4198 , piece_id=IAAOUE6KU6RKS2J5XGCNED4HJ7WQU5MT5DC4FR6KOZCSXZQ3VZ5Q, seg_pos={0 0}, upload=57171]: Uploading piece... Jul 26 23:12:21.227901350 (59h36m52.41819994s) [noise=true , address=79.127.205.245:4243 , piece_id=NM7YJPCNO3KUI2HSB3IN4EECIHBBIN6IONHZEUOQSXFCOFXXFSXQ, seg_pos={0 0}, upload=57171]: Uploading piece... Jul 26 23:12:21.831467172 (59h36m53.021765745s) [upload=57171]: Next returned nil segment Jul 26 23:12:21.831484551 (59h36m53.02178313s) [upload=57171]: Waiting for error group managing segments... Jul 26 23:12:21.992515039 (59h36m53.182813611s) [noise=true , address=79.127.219.38:4018 , piece_id=QQBAQXQOQBGBM3L5UAUSSUZBV3DW2432N22MVK3TBZPAKSBFF56Q, seg_pos={0 0}, upload=57171]: Done uploading piece. err: <nil> Jul 26 23:12:22.033591032 (59h36m53.22388962s) [noise=true , address=77.90.31.21:4897 , piece_id=GEWGIT7Y32C67GNWA72J4APHEABYO6B762PJRELMG7UUDU5GMHNQ, seg_pos={0 0}, upload=57171]: Done uploading piece. err: <nil> Jul 26 23:12:22.469793377 (59h36m53.660091949s) [noise=true , address=79.127.205.235:4657 , piece_id=64CJNWPNLTZELACEVTLRJY5D4E7OTIQRVUWGDEM63YCVB5C3JU7A, seg_pos={0 0}, upload=57171]: Done uploading piece. err: <nil> Jul 26 23:12:22.533809301 (59h36m53.724107957s) [noise=true , address=79.127.205.245:4243 , piece_id=NM7YJPCNO3KUI2HSB3IN4EECIHBBIN6IONHZEUOQSXFCOFXXFSXQ, seg_pos={0 0}, upload=57171]: Done uploading piece. err: <nil> Jul 26 23:12:22.854947113 (59h36m54.045245686s) [noise=true , address=79.127.205.234:4620 , piece_id=MIEA5DWJSXJHVONRSU3KJJVI3S2TEAGAEMHEIAQ3NZRHA36DX5UQ, seg_pos={0 0}, upload=57171]: Done uploading piece. err: <nil> Jul 26 23:12:26.505081005 (59h36m57.695379584s) [noise=true , address=104.36.82.55:4198 , piece_id=IAAOUE6KU6RKS2J5XGCNED4HJ7WQU5MT5DC4FR6KOZCSXZQ3VZ5Q, seg_pos={0 0}, upload=57171]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:104.36.82.55:4198): protocol: storage node overloaded, request limit: 2000 Jul 26 23:12:27.178734312 (59h36m58.369032891s) [noise=true , address=45.91.116.5:4310 , piece_id=PZMO45ANMNN73KWD5QUPB4C23MGVQB6CD2YKFZJ3PF53SBY2X6BA, seg_pos={0 0}, upload=57171]: Done uploading piece. err: ecclient: upload failed (node:127RMUc62xRq9RU5z3XfkZsq5sehmv3kP8o53tr9BxVcPE3GQCY, address:45.91.116.5:4310): protocol: storage node overloaded, request limit: 2000 Jul 26 23:12:30.063787060 (59h37m1.254085634s) [noise=true , address=64.49.13.140:4514 , piece_id=GHQP6M2V7T5SVD53UTEKKZ6F6R5YFR6D2JTMSNV5CBDPS6BMAC2A, seg_pos={0 0}, upload=57171]: Done uploading piece. err: <nil> Jul 26 23:12:32.874834825 (59h37m4.065133398s) [noise=true , address=79.127.201.215:4347 , piece_id=27RYJGOMSKFDRRUAQZDIB5U7TNGK2EW6SUIMGVXTGZUCBI45VVEQ, seg_pos={0 0}, upload=57171]: Done uploading piece. err: <nil> Jul 26 23:12:41.736054865 (59h37m12.92635343s) [noise=true , address=104.36.82.56:4240 , piece_id=6E6HAIF5YPUTN23V5SECBXMOOJBKVJB4ZTASX7344ZUGJBUR3DOQ, seg_pos={0 0}, upload=57171]: Done uploading piece. err: <nil> Jul 26 23:13:07.100037150 (59h37m38.290335735s) [noise=true , address=104.36.82.3:4772 , piece_id=YX2F34OL6R75KA36IPGKHE77WBNOP5COKIV3ZRDXNLPOS4YASLGA, seg_pos={0 0}, upload=57171]: Done uploading piece. err: <nil> Jul 26 23:13:09.822479856 (59h37m41.012778439s) [noise=true , address=104.36.82.46:4496 , piece_id=MTSJK4Y5L4XUBPPW72FFLRE2FVOB4KOTDJ55JNJ7Y3S76IOI3T3A, seg_pos={0 0}, upload=57171]: Done uploading piece. err: <nil> Jul 26 23:13:20.743990298 (59h37m51.934288888s) [noise=true , address=104.36.82.54:4189 , piece_id=SAJBNIPKDGSIIYO6AWVSKDRG3RACUFJCQRB2GANGNTZNLHTQT5BQ, seg_pos={0 0}, upload=57171]: Done uploading piece. err: ecclient: upload failed (node:13mB9AoQJubhEJAAVqzd4o4eq8AfNvLkdqHTDAPN3UyXXwE4B7, address:104.36.82.54:4189): protocol: expected piece hash; write tcp 139.59.144.22:47104->104.36.82.54:4189: use of closed network connection; write tcp 139.59.144.22:47104->104.36.82.54:4189: use of closed network connection; EOF Jul 26 23:13:20.744276384 (59h37m51.93457497s) [noise=true , address=77.90.31.10:4542 , piece_id=WRHU36OHNPTL7T7XQ4X2V7WKQYQOHUG7PMTW55UWOWOKRC2E4WTQ, seg_pos={0 0}, upload=57171]: Done uploading piece. err: ecclient: upload failed (node:1T8wyLvnYL2oRtQUhw8FhVFZ98Q5DhyRv1EnkCNAbW8TqcAjJc, address:77.90.31.10:4542): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 23:13:20.744351270 (59h37m51.934649856s) [noise=true , address=104.36.82.53:4137 , piece_id=AQXBHSUCKKU7DHCO3XTELNI7EHAVMF4LEE6MEEBGKZ5OQUNAATKQ, seg_pos={0 0}, upload=57171]: Done uploading piece. err: ecclient: upload failed (node:1dZZBaTDA1kbdJmNZzzYgdPkKE2rNvEqdWaMGUxuTD2ERbSanc, address:104.36.82.53:4137): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 23:13:20.744641375 (59h37m51.934939958s) [noise=true , address=104.36.82.47:4534 , piece_id=6KC4R6M7ZFP4TL2DKGX33FOF2NBGUKRYZLJZ4OT2FR6UV4IF2Z7A, seg_pos={0 0}, upload=57171]: Done uploading piece. err: ecclient: upload failed (node:1VDrNGcCnaKEmG5NN4nex1JoZAgdCEfX5CdsKbxytfLfKzPbr, address:104.36.82.47:4534): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 23:13:20.744741570 (59h37m51.935040153s) [noise=true , address=104.36.82.52:4099 , piece_id=X77KOOXQECYGS45HOXDFXN2U5PZXSGHNHN2FXQ46EMQAGWG26OPA, seg_pos={0 0}, upload=57171]: Done uploading piece. err: ecclient: upload failed (node:122dGFWNmsuheRmfEfNGWHftPpCcWv7HHXKymgJxsye6P7ZKhZT, address:104.36.82.52:4099): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 23:13:20.745175905 (59h37m51.935474491s) [noise=true , address=77.90.31.17:4786 , piece_id=4ORKEXIRMAQGEFHMZ6OYHN5ZFVXW7GGYQSJF2PBUC7QLIRQ2MKTQ, seg_pos={0 0}, upload=57171]: Done uploading piece. err: ecclient: upload failed (node:128QrekLqpXLAJs7yqNziNLmt2i13SuMx1ZhMPdUpDosw71dgQh, address:77.90.31.17:4786): protocol: rpc: dial tcp 77.90.31.17:4786: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 23:13:20.745203930 (59h37m51.935502518s) [seg_pos={0 0}, upload=57171]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 26 23:13:20.871679533 (59h37m52.061978107s) [upload=57171]: Done uploading segments... Jul 26 23:13:20.871708876 (59h37m52.062007454s) [upload=57171]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration:* 1m0.130026725s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)