Orbiter History

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 2 out of 60 is failed (threshold=3) [.................................X............X.............] duration: 235.116801ms remote: "tcp:185.244.226.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 22:36:05 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 235.144554ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 12 out of 60 is failed (threshold=3) [..........XXXX..............................XXXXXXXX........] duration: 3.125651881s FAIL (2025-08-07 22:36:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== s... Aug 7 22:21:14.636082751 (12h41m36.380139602s) [upload=13066]: Got next segment. Inline: false Aug 7 22:21:14.636136774 (12h41m36.380193634s) [upload=13066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:21:14.636142452 (12h41m36.380199312s) [upload=13066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:21:16.211357551 (12h41m37.955414411s) [seg_pos={0 0}, upload=13066]: Begin upload segment... Aug 7 22:21:16.211426917 (12h41m37.955483784s) [noise=true , address=45.91.116.5:4327 , piece_id=IGBMWNLDFIKQY7AJ3UJ6JILAFLRQ5DZAE6FFO3EJ4NDPVWZXCK5A, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.214054689 (12h41m37.95811156s) [seg_pos={0 0}, upload=13066]: Done begin upload segment. Aug 7 22:21:16.214143921 (12h41m37.958200791s) [noise=true , address=104.36.82.2:4754 , piece_id=PQJBMG6FQ3HUTE26IBB5R3WQCAJTFKMGECOXSJPV2VDHP33JOPUA, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.214660710 (12h41m37.95871758s) [noise=true , address=104.36.82.59:4336 , piece_id=5AVF5EEMC334PPHVOSYDAYZ5CKQ4F5LE6BRD5G446NRRJNYQBOYQ, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.215094607 (12h41m37.959151469s) [noise=true , address=104.36.82.44:4904 , piece_id=RWD3KVEEHT5BUVYBAHRHZFCBTCZUTNHRD6PU7OWOUIF2QVBZXSSQ, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.215554381 (12h41m37.959611251s) [noise=true , address=104.36.82.58:4270 , piece_id=LUZ6HPESCVZDYNDVHR2QIS6HM2IIRK3IQB6BSV2VVVB3OFRZVMFQ, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.215879389 (12h41m37.959936254s) [noise=true , address=104.36.82.60:4339 , piece_id=FI3A54HMEOKZJ2FC2ZOLNQXW6NR7AUD26HBVFFGEGTG3AYRZD73A, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.216161781 (12h41m37.960218643s) [noise=true , address=104.36.82.61:4397 , piece_id=7FJT5CLASEMVAP4HSGHP3XQTNBKVRP2STW2YF6H2UDLHFPUHKMMQ, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.216489433 (12h41m37.960546284s) [noise=true , address=104.36.82.17:4344 , piece_id=CCKNSMXAJAO3QUJDDZ7POWA7NZ63KACQQAJRU37RPG5YMXVMOVZA, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.216758002 (12h41m37.960814865s) [noise=true , address=104.36.82.47:4525 , piece_id=62GQ6IOY4OBZAPRBVZ2OLLA7R4DVQVMWNQZAR67BTPQUX2HUEZRA, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.217037043 (12h41m37.961093908s) [noise=true , address=104.36.82.13:4191 , piece_id=NUNWAQKF465J5JQLYQBOCH75SAXPR5F3OPBDDTC6AWWBSKB6MSUA, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.217313215 (12h41m37.961370077s) [noise=true , address=64.49.13.140:4509 , piece_id=CNKDQOVU53HJ6EDOSZOXRSFOKQBNKWERZSFJQA5F24FZBR6XEEQA, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.217587290 (12h41m37.961644153s) [noise=true , address=45.91.116.9:4292 , piece_id=NO3S4IIMN5JWUNMMNQS6SDFLCDPW3MF3LK4BDZ4IVIZMRWBHSTQQ, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.217852877 (12h41m37.961909742s) [noise=true , address=104.36.82.43:4849 , piece_id=ECUDD3LRMPQMXZW6MRFLFHVPAO3RHCV6MWZYOJSASL4ILT54TFFA, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.218138868 (12h41m37.962195731s) [noise=true , address=64.49.13.26:4956 , piece_id=VHWLGKKB5MBWQOD3KVNXE3YYCK5TZ56UU25BTACIPJQZJA7PHA7Q, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.218358245 (12h41m37.962415106s) [noise=true , address=104.36.82.62:4435 , piece_id=O3PP5IRA3MLOLM35Z2U2BLM7D6GPH6WMHWANUF7AAGTW7IW6CZBQ, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.218693596 (12h41m37.962750459s) [noise=true , address=79.127.205.233:4647 , piece_id=6YIT5HDQJ6CN63ZFANACICFS4M2OQEPOSFIV36QXS5VLJZ2BRR7A, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.218971752 (12h41m37.963028615s) [noise=true , address=104.36.82.60:4354 , piece_id=A3YTMCWULXHPBRVL6TDQOBNYTQQWNKB2UZPKAPDH2CCG5SMV6QTA, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.219339785 (12h41m37.963396657s) [noise=true , address=104.36.82.9:4036 , piece_id=UG2DKHGMS5NC3BTN7QQU23JXBUQ7UFVN6FZKPBZHYUOQ6SEJV5WA, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.820211995 (12h41m38.564268846s) [upload=13066]: Next returned nil segment Aug 7 22:21:16.820729442 (12h41m38.564786312s) [upload=13066]: Waiting for error group managing segments... Aug 7 22:21:16.974534776 (12h41m38.71859164s) [noise=true , address=104.36.82.61:4397 , piece_id=7FJT5CLASEMVAP4HSGHP3XQTNBKVRP2STW2YF6H2UDLHFPUHKMMQ, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:16.995935209 (12h41m38.739992059s) [noise=true , address=104.36.82.9:4036 , piece_id=UG2DKHGMS5NC3BTN7QQU23JXBUQ7UFVN6FZKPBZHYUOQ6SEJV5WA, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.005410191 (12h41m38.749467042s) [noise=true , address=104.36.82.60:4354 , piece_id=A3YTMCWULXHPBRVL6TDQOBNYTQQWNKB2UZPKAPDH2CCG5SMV6QTA, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.012303228 (12h41m38.756360079s) [noise=true , address=104.36.82.17:4344 , piece_id=CCKNSMXAJAO3QUJDDZ7POWA7NZ63KACQQAJRU37RPG5YMXVMOVZA, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.047899041 (12h41m38.791955903s) [noise=true , address=104.36.82.13:4191 , piece_id=NUNWAQKF465J5JQLYQBOCH75SAXPR5F3OPBDDTC6AWWBSKB6MSUA, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.065632282 (12h41m38.809689142s) [noise=true , address=104.36.82.60:4339 , piece_id=FI3A54HMEOKZJ2FC2ZOLNQXW6NR7AUD26HBVFFGEGTG3AYRZD73A, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.074330593 (12h41m38.818387457s) [noise=true , address=104.36.82.62:4435 , piece_id=O3PP5IRA3MLOLM35Z2U2BLM7D6GPH6WMHWANUF7AAGTW7IW6CZBQ, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.205659931 (12h41m38.949716794s) [noise=true , address=104.36.82.59:4336 , piece_id=5AVF5EEMC334PPHVOSYDAYZ5CKQ4F5LE6BRD5G446NRRJNYQBOYQ, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.304344891 (12h41m39.048401752s) [noise=true , address=45.91.116.5:4327 , piece_id=IGBMWNLDFIKQY7AJ3UJ6JILAFLRQ5DZAE6FFO3EJ4NDPVWZXCK5A, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.476320837 (12h41m39.220377687s) [noise=true , address=64.49.13.140:4509 , piece_id=CNKDQOVU53HJ6EDOSZOXRSFOKQBNKWERZSFJQA5F24FZBR6XEEQA, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.498349653 (12h41m39.242406517s) [noise=true , address=64.49.13.26:4956 , piece_id=VHWLGKKB5MBWQOD3KVNXE3YYCK5TZ56UU25BTACIPJQZJA7PHA7Q, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.566667384 (12h41m39.310724234s) [noise=true , address=104.36.82.47:4525 , piece_id=62GQ6IOY4OBZAPRBVZ2OLLA7R4DVQVMWNQZAR67BTPQUX2HUEZRA, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.572654381 (12h41m39.316711251s) [noise=true , address=79.127.205.233:4647 , piece_id=6YIT5HDQJ6CN63ZFANACICFS4M2OQEPOSFIV36QXS5VLJZ2BRR7A, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:18.002510167 (12h41m39.746567013s) [noise=true , address=45.91.116.9:4292 , piece_id=NO3S4IIMN5JWUNMMNQS6SDFLCDPW3MF3LK4BDZ4IVIZMRWBHSTQQ, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:18.002534446 (12h41m39.746591307s) [seg_pos={0 0}, upload=13066]: Segment reached optimal threshold of 14 pieces. Aug 7 22:21:18.002684920 (12h41m39.746741766s) [noise=true , address=104.36.82.43:4849 , piece_id=ECUDD3LRMPQMXZW6MRFLFHVPAO3RHCV6MWZYOJSASL4ILT54TFFA, seg_pos={0 0}, upload=13066]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12goRcYW39Qqga4HW1i6cg6VGKJFUxnyTJBe6HRnZiLAU3LonWn): protocol: expected piece hash; context canceled; EOF Aug 7 22:21:18.002753282 (12h41m39.746810128s) [noise=true , address=104.36.82.58:4270 , piece_id=LUZ6HPESCVZDYNDVHR2QIS6HM2IIRK3IQB6BSV2VVVB3OFRZVMFQ, seg_pos={0 0}, upload=13066]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sJfMr1nMTpizd5PZ2bZ6M8tZWhFL2ukdJ3cNSRQtdUHNJhg81): protocol: expected piece hash; context canceled; EOF Aug 7 22:21:18.002945708 (12h41m39.747002555s) [noise=true , address=104.36.82.2:4754 , piece_id=PQJBMG6FQ3HUTE26IBB5R3WQCAJTFKMGECOXSJPV2VDHP33JOPUA, seg_pos={0 0}, upload=13066]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125aPwoHeFabk6zZ6bkGNj3USJYgTQG3RB1QhhXssj7Vik2AuCs): protocol: expected piece hash; context canceled; EOF Aug 7 22:21:18.003059994 (12h41m39.74711684s) [noise=true , address=104.36.82.44:4904 , piece_id=RWD3KVEEHT5BUVYBAHRHZFCBTCZUTNHRD6PU7OWOUIF2QVBZXSSQ, seg_pos={0 0}, upload=13066]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: write tcp 139.59.144.22:47438->104.36.82.44:4904: use of closed network connection; write tcp 139.59.144.22:47438->104.36.82.44:4904: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47438->104.36.82.44:4904: use of closed network connection Aug 7 22:21:18.003080731 (12h41m39.747137592s) [seg_pos={0 0}, upload=13066]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 22:21:18.003298604 (12h41m39.747355464s) [upload=13066]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 22:21:18.003304952 (12h41m39.747361812s) [upload=13066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:30:15.962677772 (12h50m37.706734624s) [upload=13066]: Done uploading segments... Aug 7 22:30:15.962709891 (12h50m37.706766753s) [upload=13066]: Upload finished. err: metaclient: context deadline exceeded Duration: 3.125695895s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 3 out of 60 is failed (threshold=3) [.........................................XXX................] duration: 352.709306ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 22:36:03 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 353.084861ms edge gateway vhost us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 19 out of 60 is failed (threshold=3) [.X.X...........................XXXXXXXXXXXX.XXXX.X..........] duration: 3.066635415s FAIL (2025-08-07 22:35:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.066653418s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 18 out of 60 is failed (threshold=3) [................................XXXXXXXXXXXXXXXXXX..........] duration: 2.71132235s FAIL (2025-08-07 22:35:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.711384418s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 19 out of 60 is failed (threshold=3) [XXXXXXXXXXX............................X.XXXXX.XX...........] duration: 2.340090027s FAIL (2025-08-07 22:35:57 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.340118478s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 16 out of 60 is failed (threshold=3) [...............................X.XXXXXXXXXXXXXXX............] duration: 968.088082ms FAIL (2025-08-07 22:35:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 4 (12h38m8.493437894s) [upload=13019]: Uploading segments... Aug 7 22:17:46.749423663 (12h38m8.493480532s) [upload=13019]: Got next segment. Inline: true Aug 7 22:17:46.749428712 (12h38m8.493485573s) [upload=13019]: Waiting for error group managing segments... Aug 7 22:17:46.749471140 (12h38m8.49352799s) [upload=13019]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:17:46.749476259 (12h38m8.493533119s) [upload=13019]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:17:46.749479952 (12h38m8.493536802s) [upload=13019]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:18:46.471549080 (12h39m8.215605941s) [upload=13019]: Done uploading segments... Aug 7 22:18:46.471571547 (12h39m8.215628407s) [upload=13019]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:18:46.904761993 (12h39m8.648818857s) [upload=13033]: Starting upload Aug 7 22:18:46.904856076 (12h39m8.648912937s) [upload=13033]: Done starting upload Aug 7 22:18:46.904960146 (12h39m8.649017s) [upload=13033]: Uploading segments... Aug 7 22:18:46.905047245 (12h39m8.649104114s) [upload=13033]: Got next segment. Inline: true Aug 7 22:18:46.905056198 (12h39m8.649113053s) [upload=13033]: Waiting for error group managing segments... Aug 7 22:18:46.905116687 (12h39m8.649173538s) [upload=13033]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:18:46.905122460 (12h39m8.649179309s) [upload=13033]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:18:46.905126367 (12h39m8.649183281s) [upload=13033]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:19:46.626010888 (12h40m8.370067739s) [upload=13033]: Done uploading segments... Aug 7 22:19:46.626032620 (12h40m8.370089471s) [upload=13033]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:19:59.643579230 (12h40m21.387636081s) [upload=13050]: Starting upload Aug 7 22:19:59.643668904 (12h40m21.387725771s) [upload=13050]: Done starting upload Aug 7 22:19:59.643785033 (12h40m21.38784188s) [upload=13050]: Uploading segments... Aug 7 22:19:59.643889787 (12h40m21.387946639s) [upload=13050]: Got next segment. Inline: true Aug 7 22:19:59.643897379 (12h40m21.387954229s) [upload=13050]: Waiting for error group managing segments... Aug 7 22:19:59.643957450 (12h40m21.388014313s) [upload=13050]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:19:59.643962932 (12h40m21.388019793s) [upload=13050]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:19:59.643966291 (12h40m21.388023151s) [upload=13050]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:20:59.369875577 (12h41m21.113932442s) [upload=13050]: Done uploading segments... Aug 7 22:20:59.369905671 (12h41m21.113962518s) [upload=13050]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:20:59.787948456 (12h41m21.532005324s) [upload=13064]: Starting upload Aug 7 22:20:59.788068842 (12h41m21.53212571s) [upload=13064]: Done starting upload Aug 7 22:20:59.788168587 (12h41m21.532225457s) [upload=13064]: Uploading segments... Aug 7 22:20:59.788311286 (12h41m21.532368151s) [upload=13064]: Got next segment. Inline: true Aug 7 22:20:59.788318759 (12h41m21.532375633s) [upload=13064]: Waiting for error group managing segments... Aug 7 22:20:59.788388410 (12h41m21.532445276s) [upload=13064]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:20:59.788394823 (12h41m21.532451689s) [upload=13064]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:20:59.788398689 (12h41m21.532455554s) [upload=13064]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:21:59.519361969 (12h42m21.263418825s) [upload=13064]: Done uploading segments... Aug 7 22:21:59.519387607 (12h42m21.263444468s) [upload=13064]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:21:59.945997169 (12h42m21.69005402s) [upload=13075]: Starting upload Aug 7 22:21:59.946071844 (12h42m21.690129359s) [upload=13075]: Done starting upload Aug 7 22:21:59.946178905 (12h42m21.690235754s) [upload=13075]: Uploading segments... Aug 7 22:21:59.946230067 (12h42m21.690286936s) [upload=13075]: Got next segment. Inline: true Aug 7 22:21:59.946235138 (12h42m21.690292002s) [upload=13075]: Waiting for error group managing segments... Aug 7 22:21:59.946295164 (12h42m21.690352024s) [upload=13075]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:21:59.946300535 (12h42m21.690357396s) [upload=13075]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:21:59.946303920 (12h42m21.69036078s) [upload=13075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:22:59.677780729 (12h43m21.421837589s) [upload=13075]: Done uploading segments... Aug 7 22:22:59.677798007 (12h43m21.421854868s) [upload=13075]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:23:00.094834123 (12h43m21.838890994s) [upload=13088]: Starting upload Aug 7 22:23:00.094984258 (12h43m21.83904116s) [upload=13088]: Done starting upload Aug 7 22:23:00.095111897 (12h43m21.839168758s) [upload=13088]: Uploading segments... Aug 7 22:23:00.095477827 (12h43m21.839534697s) [upload=13088]: Got next segment. Inline: true Aug 7 22:23:00.095487112 (12h43m21.839543982s) [upload=13088]: Waiting for error group managing segments... Aug 7 22:23:00.095559737 (12h43m21.839616605s) [upload=13088]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:23:00.095567952 (12h43m21.839624822s) [upload=13088]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:23:00.095571963 (12h43m21.839628824s) [upload=13088]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:23:59.816074411 (12h44m21.560131259s) [upload=13088]: Done uploading segments... Aug 7 22:23:59.816094923 (12h44m21.560162185s) [upload=13088]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:24:00.243449247 (12h44m21.987506098s) [upload=13101]: Starting upload Aug 7 22:24:00.243529153 (12h44m21.987586015s) [upload=13101]: Done starting upload Aug 7 22:24:00.243603428 (12h44m21.987660281s) [upload=13101]: Uploading segments... Aug 7 22:24:00.243664741 (12h44m21.98772161s) [upload=13101]: Got next segment. Inline: true Aug 7 22:24:00.243670702 (12h44m21.987727562s) [upload=13101]: Waiting for error group managing segments... Aug 7 22:24:00.243730265 (12h44m21.987787124s) [upload=13101]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:24:00.243736084 (12h44m21.987792946s) [upload=13101]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:24:00.243739669 (12h44m21.987796531s) [upload=13101]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:24:59.965587620 (12h45m21.709644469s) [upload=13101]: Done uploading segments... Aug 7 22:24:59.965609247 (12h45m21.709666095s) [upload=13101]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:25:00.417995504 (12h45m22.162052369s) [upload=13115]: Starting upload Aug 7 22:25:00.418148792 (12h45m22.162205658s) [upload=13115]: Done starting upload Aug 7 22:25:00.418313994 (12h45m22.162370855s) [upload=13115]: Uploading segments... Aug 7 22:25:00.418427015 (12h45m22.162483886s) [upload=13115]: Got next segment. Inline: true Aug 7 22:25:00.418453990 (12h45m22.162510859s) [upload=13115]: Waiting for error group managing segments... Aug 7 22:25:00.418542094 (12h45m22.16259896s) [upload=13115]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:25:00.418549113 (12h45m22.162605982s) [upload=13115]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:25:00.418555718 (12h45m22.162612588s) [upload=13115]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:26:00.143481421 (12h46m21.887538319s) [upload=13115]: Done uploading segments... Aug 7 22:26:00.143508788 (12h46m21.88756568s) [upload=13115]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:26:00.572183011 (12h46m22.316239858s) [upload=13133]: Starting upload Aug 7 22:26:00.572257919 (12h46m22.316314778s) [upload=13133]: Done starting upload Aug 7 22:26:00.572347104 (12h46m22.316403949s) [upload=13133]: Uploading segments... Aug 7 22:26:00.572414569 (12h46m22.316471429s) [upload=13133]: Got next segment. Inline: true Aug 7 22:26:00.572419354 (12h46m22.316476214s) [upload=13133]: Waiting for error group managing segments... Aug 7 22:26:00.572457012 (12h46m22.316513858s) [upload=13133]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:26:00.572461785 (12h46m22.316518631s) [upload=13133]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:26:00.572465017 (12h46m22.316521878s) [upload=13133]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:27:00.300711265 (12h47m22.044768125s) [upload=13133]: Done uploading segments... Aug 7 22:27:00.300733422 (12h47m22.044790282s) [upload=13133]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:27:00.736249379 (12h47m22.48030624s) [upload=13147]: Starting upload Aug 7 22:27:00.736365836 (12h47m22.480422697s) [upload=13147]: Done starting upload Aug 7 22:27:00.736456413 (12h47m22.480513264s) [upload=13147]: Uploading segments... Aug 7 22:27:00.736517869 (12h47m22.480574739s) [upload=13147]: Got next segment. Inline: true Aug 7 22:27:00.736526521 (12h47m22.480583386s) [upload=13147]: Waiting for error group managing segments... Aug 7 22:27:00.736600220 (12h47m22.480657069s) [upload=13147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:27:00.736605984 (12h47m22.480662844s) [upload=13147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:27:00.736609354 (12h47m22.480666214s) [upload=13147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:28:00.459012227 (12h48m22.203069105s) [upload=13147]: Done uploading segments... Aug 7 22:28:00.459039460 (12h48m22.203096312s) [upload=13147]: Upload finished. err: metaclient: context deadline exceeded Duration: 968.108549ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 19 out of 60 is failed (threshold=3) [............................XXXXXXXXXXXXXXXX.XXX............] duration: 1.023411792s FAIL (2025-08-07 22:35:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.023445691s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [...................................XXXXXXXXXXXXX..X.........] duration: 2.862983579s FAIL (2025-08-07 22:35:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== 7 22:27:44.139472223 (12h48m5.883529083s) [upload=13160]: Got next segment. Inline: false Aug 7 22:27:44.139508151 (12h48m5.883565011s) [upload=13160]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:27:44.139513399 (12h48m5.883570259s) [upload=13160]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:27:44.245907064 (12h48m5.989963927s) [seg_pos={0 0}, upload=13160]: Begin upload segment... Aug 7 22:27:44.246039735 (12h48m5.990096597s) [noise=true , address=104.36.82.10:4079 , piece_id=2ADIUNAB5FPKLXMW46HBB75SJLYOKA4N3TYS6J72WZDKLQTSP23A, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.246596012 (12h48m5.990652882s) [noise=true , address=104.36.82.4:4801 , piece_id=U6XCDALXGOTJHRL2P4V7DOPIJJF6VUKB5RIJYOP3WXFTC6HQKQXQ, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.247006251 (12h48m5.991063115s) [noise=true , address=77.90.31.12:4590 , piece_id=IYZTGAPBVXXQNQVDNR4NZRPDNNIBETBQ26DZUZ2SODA2D2VQUXTA, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.247197178 (12h48m5.991254041s) [seg_pos={0 0}, upload=13160]: Done begin upload segment. Aug 7 22:27:44.247225152 (12h48m5.991282018s) [noise=true , address=104.36.82.5:4837 , piece_id=XDA3KTWSH5OJPHK4IFOPINMJAR6VJLL7B7WNO3MV2VT3HQCGJ2KA, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.247751444 (12h48m5.99180831s) [noise=true , address=104.36.82.6:4876 , piece_id=45DZFNAUCX6EJNPT4WBVLEIO2GZROYIF2NQ254LU32T6AE7P4ZRA, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.247956514 (12h48m5.992013382s) [noise=true , address=104.36.82.11:4115 , piece_id=B2C72IS3FGY7DT4SXLUG5FYHCN5ECAMCAIVT2ZNVCXML2RVCQZFQ, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.248499391 (12h48m5.992556252s) [noise=true , address=45.91.116.12:4225 , piece_id=N7R42AWZKTHBBPZY24OVKSBRTPGVBLJ3HBYB2ONRZ56F6YZK2XWQ, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.248606386 (12h48m5.992663249s) [noise=true , address=77.90.31.13:4628 , piece_id=FSARBSWO4VFKOYHINODNYG7BC4JEH6KX5Z3IOYIJ4DGQ3DCZNDJA, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.249110151 (12h48m5.99316702s) [noise=true , address=45.91.116.9:4261 , piece_id=ZESRB2XAQ4WPIB2HQKYLZKJDXDE65NXMRX4M5JE3EKRHROCT4JKA, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.249195913 (12h48m5.993252781s) [noise=true , address=104.36.82.13:4185 , piece_id=QI54Y53UTBR7JEQ4NPD5R37EC47364LEUWETTCGI3WQUKSHUTP5A, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.249587895 (12h48m5.993644759s) [noise=true , address=45.91.116.8:4047 , piece_id=LDURVPEWVV7OEPVY4L4ZER4H2EDUPJS45N55Y57VZUTLSTKGXSRA, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.249756005 (12h48m5.993812872s) [noise=true , address=77.90.31.11:4553 , piece_id=ZDJ5BW2XPNHEBNB6FBVWAH66T6SVKAUY3724YY6HZ25VBLKYFTIQ, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.250801847 (12h48m5.99485871s) [noise=true , address=77.90.31.10:4517 , piece_id=QHV63D3QBJ5GVDWNHGWJTLXQNHGK4PZ4OCT4UWCEQKHMYBDL26WA, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.251094602 (12h48m5.995151463s) [noise=true , address=77.90.31.18:4806 , piece_id=ZJLVNSXETRBXYNHIDLN7TSQU6OQVWTCOQFJLPWAMCPY2JHBTHVQA, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.251408354 (12h48m5.995465217s) [noise=true , address=104.36.82.10:4078 , piece_id=UCDTQMJTYXAMJXGRPKBGTJZ4OCF4P2BSCR6WR3KSXCK6S5PWWZ7A, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.251693979 (12h48m5.995750839s) [noise=true , address=45.91.116.5:4297 , piece_id=QEQZU67YDQXNW7ACIRVXJKX2HIKMZMAP23TCAMTB2RDK3RFKLLRQ, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.928589281 (12h48m6.672646155s) [upload=13160]: Next returned nil segment Aug 7 22:27:44.928610126 (12h48m6.672666995s) [upload=13160]: Waiting for error group managing segments... Aug 7 22:27:45.055730069 (12h48m6.799786923s) [noise=true , address=104.36.82.11:4115 , piece_id=B2C72IS3FGY7DT4SXLUG5FYHCN5ECAMCAIVT2ZNVCXML2RVCQZFQ, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.058230571 (12h48m6.802287421s) [noise=true , address=104.36.82.4:4801 , piece_id=U6XCDALXGOTJHRL2P4V7DOPIJJF6VUKB5RIJYOP3WXFTC6HQKQXQ, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.072176193 (12h48m6.816233043s) [noise=true , address=104.36.82.6:4876 , piece_id=45DZFNAUCX6EJNPT4WBVLEIO2GZROYIF2NQ254LU32T6AE7P4ZRA, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.076144433 (12h48m6.82020128s) [noise=true , address=104.36.82.5:4837 , piece_id=XDA3KTWSH5OJPHK4IFOPINMJAR6VJLL7B7WNO3MV2VT3HQCGJ2KA, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.083969611 (12h48m6.828026457s) [noise=true , address=104.36.82.13:4185 , piece_id=QI54Y53UTBR7JEQ4NPD5R37EC47364LEUWETTCGI3WQUKSHUTP5A, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.090499988 (12h48m6.834556849s) [noise=true , address=104.36.82.10:4078 , piece_id=UCDTQMJTYXAMJXGRPKBGTJZ4OCF4P2BSCR6WR3KSXCK6S5PWWZ7A, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.129675895 (12h48m6.873732757s) [noise=true , address=45.91.116.12:4225 , piece_id=N7R42AWZKTHBBPZY24OVKSBRTPGVBLJ3HBYB2ONRZ56F6YZK2XWQ, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.148504731 (12h48m6.892561581s) [noise=true , address=77.90.31.10:4517 , piece_id=QHV63D3QBJ5GVDWNHGWJTLXQNHGK4PZ4OCT4UWCEQKHMYBDL26WA, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.183435281 (12h48m6.927492128s) [noise=true , address=104.36.82.10:4079 , piece_id=2ADIUNAB5FPKLXMW46HBB75SJLYOKA4N3TYS6J72WZDKLQTSP23A, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.188056276 (12h48m6.932113137s) [noise=true , address=77.90.31.12:4590 , piece_id=IYZTGAPBVXXQNQVDNR4NZRPDNNIBETBQ26DZUZ2SODA2D2VQUXTA, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.189109860 (12h48m6.93316672s) [noise=true , address=77.90.31.13:4628 , piece_id=FSARBSWO4VFKOYHINODNYG7BC4JEH6KX5Z3IOYIJ4DGQ3DCZNDJA, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.191395584 (12h48m6.93545243s) [noise=true , address=45.91.116.9:4261 , piece_id=ZESRB2XAQ4WPIB2HQKYLZKJDXDE65NXMRX4M5JE3EKRHROCT4JKA, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.212167167 (12h48m6.956224014s) [noise=true , address=77.90.31.11:4553 , piece_id=ZDJ5BW2XPNHEBNB6FBVWAH66T6SVKAUY3724YY6HZ25VBLKYFTIQ, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.212191124 (12h48m6.956247983s) [seg_pos={0 0}, upload=13160]: Segment reached optimal threshold of 13 pieces. Aug 7 22:27:45.212404510 (12h48m6.956461359s) [noise=true , address=45.91.116.8:4047 , piece_id=LDURVPEWVV7OEPVY4L4ZER4H2EDUPJS45N55Y57VZUTLSTKGXSRA, seg_pos={0 0}, upload=13160]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: context canceled; context canceled Aug 7 22:27:45.212444310 (12h48m6.956501156s) [noise=true , address=77.90.31.18:4806 , piece_id=ZJLVNSXETRBXYNHIDLN7TSQU6OQVWTCOQFJLPWAMCPY2JHBTHVQA, seg_pos={0 0}, upload=13160]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: write tcp 139.59.144.22:44926->77.90.31.18:4806: use of closed network connection; write tcp 139.59.144.22:44926->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44926->77.90.31.18:4806: use of closed network connection Aug 7 22:27:45.212507508 (12h48m6.956564355s) [noise=true , address=45.91.116.5:4297 , piece_id=QEQZU67YDQXNW7ACIRVXJKX2HIKMZMAP23TCAMTB2RDK3RFKLLRQ, seg_pos={0 0}, upload=13160]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 139.59.144.22:41424->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:41424->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41424->45.91.116.5:4297: use of closed network connection Aug 7 22:27:45.212525428 (12h48m6.956582288s) [seg_pos={0 0}, upload=13160]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 22:27:45.212587349 (12h48m6.956644209s) [upload=13160]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 22:27:45.212591472 (12h48m6.956648332s) [upload=13160]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:27:45.467451329 (12h48m7.21150818s) [upload=13160]: Done uploading segments... Aug 7 22:27:45.467470231 (12h48m7.211527081s) [upload=13160]: Upload finished. err: <nil> Aug 7 22:28:46.300540067 (12h49m8.044596928s) [upload=13178]: Starting upload Aug 7 22:28:46.300629117 (12h49m8.044685979s) [upload=13178]: Done starting upload Aug 7 22:28:46.300797752 (12h49m8.044854603s) [upload=13178]: Uploading segments... Aug 7 22:28:46.301190738 (12h49m8.045247598s) [upload=13178]: Got next segment. Inline: false Aug 7 22:28:46.301268000 (12h49m8.04532486s) [upload=13178]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:28:46.301275318 (12h49m8.045332178s) [upload=13178]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:29:16.009299446 (12h49m37.753356315s) [upload=13178]: Next segment err: context deadline exceeded Aug 7 22:29:16.009371909 (12h49m37.753428759s) [upload=13178]: Done uploading segments... Aug 7 22:29:16.009392178 (12h49m37.753449035s) [upload=13178]: Upload finished. err: context deadline exceeded Duration: 2.86301595s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 17 out of 60 is failed (threshold=3) [...........................XXXXXXXXXXXX.XXXX....X...........] duration: 1.090722755s FAIL (2025-08-07 22:35:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== [upload=13077]: Starting upload Aug 7 22:22:03.530157956 (12h42m25.27421483s) [upload=13077]: Done starting upload Aug 7 22:22:03.530254340 (12h42m25.274311199s) [upload=13077]: Uploading segments... Aug 7 22:22:03.530319521 (12h42m25.274376398s) [upload=13077]: Got next segment. Inline: true Aug 7 22:22:03.530326839 (12h42m25.27438371s) [upload=13077]: Waiting for error group managing segments... Aug 7 22:22:03.530387033 (12h42m25.274443903s) [upload=13077]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:22:03.530393764 (12h42m25.274450635s) [upload=13077]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:22:03.530398068 (12h42m25.274454934s) [upload=13077]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:22:03.668101987 (12h42m25.412158856s) [upload=13077]: Done uploading segments... Aug 7 22:22:03.668146181 (12h42m25.412203031s) [upload=13077]: Upload finished. err: <nil> Aug 7 22:22:50.781175997 (12h43m12.525232844s) [upload=13085]: Starting upload Aug 7 22:22:50.781238492 (12h43m12.525295338s) [upload=13085]: Done starting upload Aug 7 22:22:50.781295832 (12h43m12.525352678s) [upload=13085]: Uploading segments... Aug 7 22:22:50.781339974 (12h43m12.525396844s) [upload=13085]: Got next segment. Inline: true Aug 7 22:22:50.781344619 (12h43m12.525401479s) [upload=13085]: Waiting for error group managing segments... Aug 7 22:22:50.781382529 (12h43m12.525439375s) [upload=13085]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:22:50.781387267 (12h43m12.525444113s) [upload=13085]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:22:50.781389841 (12h43m12.525446704s) [upload=13085]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:23:20.513298723 (12h43m42.257355587s) [upload=13085]: Done uploading segments... Aug 7 22:23:20.513325654 (12h43m42.25738252s) [upload=13085]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:23:51.254516692 (12h44m12.998573555s) [upload=13099]: Starting upload Aug 7 22:23:51.254606307 (12h44m12.998663169s) [upload=13099]: Done starting upload Aug 7 22:23:51.254669521 (12h44m12.998726371s) [upload=13099]: Uploading segments... Aug 7 22:23:51.254709863 (12h44m12.998766733s) [upload=13099]: Got next segment. Inline: true Aug 7 22:23:51.254715030 (12h44m12.99877189s) [upload=13099]: Waiting for error group managing segments... Aug 7 22:23:51.254768832 (12h44m12.998825693s) [upload=13099]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:23:51.254774395 (12h44m12.998831255s) [upload=13099]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:23:51.254778260 (12h44m12.998835121s) [upload=13099]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:24:20.980586635 (12h44m42.724643486s) [upload=13099]: Done uploading segments... Aug 7 22:24:20.980611487 (12h44m42.724668337s) [upload=13099]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:24:22.809871679 (12h44m44.55392853s) [upload=13108]: Starting upload Aug 7 22:24:22.809959685 (12h44m44.554016545s) [upload=13108]: Done starting upload Aug 7 22:24:22.810026016 (12h44m44.55408288s) [upload=13108]: Uploading segments... Aug 7 22:24:22.810084984 (12h44m44.554141856s) [upload=13108]: Got next segment. Inline: true Aug 7 22:24:22.810090937 (12h44m44.554147807s) [upload=13108]: Waiting for error group managing segments... Aug 7 22:24:22.810180251 (12h44m44.554237121s) [upload=13108]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:24:22.810187976 (12h44m44.554244835s) [upload=13108]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:24:22.810192661 (12h44m44.55424953s) [upload=13108]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:24:52.535275009 (12h45m14.279331864s) [upload=13108]: Done uploading segments... Aug 7 22:24:52.535303649 (12h45m14.279360515s) [upload=13108]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:24:52.960601883 (12h45m14.70465875s) [upload=13112]: Starting upload Aug 7 22:24:52.960708121 (12h45m14.704764984s) [upload=13112]: Done starting upload Aug 7 22:24:52.960845573 (12h45m14.704902419s) [upload=13112]: Uploading segments... Aug 7 22:24:52.960931513 (12h45m14.704988384s) [upload=13112]: Got next segment. Inline: true Aug 7 22:24:52.960939376 (12h45m14.704996243s) [upload=13112]: Waiting for error group managing segments... Aug 7 22:24:52.961019433 (12h45m14.705076313s) [upload=13112]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:24:52.961026755 (12h45m14.705083622s) [upload=13112]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:24:52.961030905 (12h45m14.705087769s) [upload=13112]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:25:22.687567107 (12h45m44.431623957s) [upload=13112]: Done uploading segments... Aug 7 22:25:22.687590061 (12h45m44.431646912s) [upload=13112]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:25:23.131013864 (12h45m44.875070712s) [upload=13122]: Starting upload Aug 7 22:25:23.131088619 (12h45m44.875145482s) [upload=13122]: Done starting upload Aug 7 22:25:23.131275088 (12h45m44.875331941s) [upload=13122]: Uploading segments... Aug 7 22:25:23.131376992 (12h45m44.875433843s) [upload=13122]: Got next segment. Inline: true Aug 7 22:25:23.131382965 (12h45m44.875439816s) [upload=13122]: Waiting for error group managing segments... Aug 7 22:25:23.131434882 (12h45m44.875491733s) [upload=13122]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:25:23.131440120 (12h45m44.875496982s) [upload=13122]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:25:23.131443668 (12h45m44.875500519s) [upload=13122]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:25:23.272670449 (12h45m45.016727299s) [upload=13122]: Done uploading segments... Aug 7 22:25:23.272702241 (12h45m45.016759091s) [upload=13122]: Upload finished. err: <nil> Aug 7 22:26:25.811198326 (12h46m47.555255193s) [upload=13140]: Starting upload Aug 7 22:26:25.811285415 (12h46m47.555342279s) [upload=13140]: Done starting upload Aug 7 22:26:25.811369732 (12h46m47.555426582s) [upload=13140]: Uploading segments... Aug 7 22:26:25.811467114 (12h46m47.55552398s) [upload=13140]: Got next segment. Inline: true Aug 7 22:26:25.811473701 (12h46m47.555530552s) [upload=13140]: Waiting for error group managing segments... Aug 7 22:26:25.811539126 (12h46m47.555595979s) [upload=13140]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:26:25.811545282 (12h46m47.555602146s) [upload=13140]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:26:25.811548939 (12h46m47.555605802s) [upload=13140]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:26:25.968679249 (12h46m47.712736113s) [upload=13140]: Done uploading segments... Aug 7 22:26:25.968709074 (12h46m47.712765928s) [upload=13140]: Upload finished. err: <nil> Aug 7 22:27:27.799853559 (12h47m49.543910409s) [upload=13154]: Starting upload Aug 7 22:27:27.799998197 (12h47m49.544055055s) [upload=13154]: Done starting upload Aug 7 22:27:27.800097878 (12h47m49.544154732s) [upload=13154]: Uploading segments... Aug 7 22:27:27.800202613 (12h47m49.544259463s) [upload=13154]: Got next segment. Inline: true Aug 7 22:27:27.800221436 (12h47m49.544278297s) [upload=13154]: Waiting for error group managing segments... Aug 7 22:27:27.800305006 (12h47m49.544361867s) [upload=13154]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:27:27.800310790 (12h47m49.54436765s) [upload=13154]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:27:27.800314678 (12h47m49.544371539s) [upload=13154]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:27:27.950105257 (12h47m49.694162117s) [upload=13154]: Done uploading segments... Aug 7 22:27:27.950124372 (12h47m49.694181233s) [upload=13154]: Upload finished. err: <nil> Aug 7 22:27:59.550898410 (12h48m21.29495526s) [upload=13165]: Starting upload Aug 7 22:27:59.550970829 (12h48m21.295027691s) [upload=13165]: Done starting upload Aug 7 22:27:59.551049380 (12h48m21.295106232s) [upload=13165]: Uploading segments... Aug 7 22:27:59.551283044 (12h48m21.295339906s) [upload=13165]: Got next segment. Inline: true Aug 7 22:27:59.551293209 (12h48m21.295350069s) [upload=13165]: Waiting for error group managing segments... Aug 7 22:27:59.551359587 (12h48m21.295416438s) [upload=13165]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:27:59.551365951 (12h48m21.295422812s) [upload=13165]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:27:59.551369453 (12h48m21.295426314s) [upload=13165]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:27:59.691525895 (12h48m21.435582778s) [upload=13165]: Done uploading segments... Aug 7 22:27:59.691551086 (12h48m21.435607933s) [upload=13165]: Upload finished. err: <nil> Aug 7 22:29:06.342665932 (12h49m28.086722791s) [upload=13183]: Starting upload Aug 7 22:29:06.342756156 (12h49m28.086813016s) [upload=13183]: Done starting upload Aug 7 22:29:06.342875471 (12h49m28.086932321s) [upload=13183]: Uploading segments... Aug 7 22:29:06.343020027 (12h49m28.087076894s) [upload=13183]: Got next segment. Inline: true Aug 7 22:29:06.343029056 (12h49m28.087085926s) [upload=13183]: Waiting for error group managing segments... Aug 7 22:29:06.343276816 (12h49m28.087333721s) [upload=13183]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:29:06.343287054 (12h49m28.087343919s) [upload=13183]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:29:06.343290895 (12h49m28.08734776s) [upload=13183]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:29:36.070785151 (12h49m57.814842003s) [upload=13183]: Done uploading segments... Aug 7 22:29:36.070807148 (12h49m57.814863999s) [upload=13183]: Upload finished. err: metaclient: context deadline exceeded Duration: 1.090749862s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 12 out of 60 is failed (threshold=3) [...........XXXX..............................XXXXXXXX.......] duration: 3.184878175s FAIL (2025-08-07 22:35:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== s... Aug 7 22:21:14.636082751 (12h41m36.380139602s) [upload=13066]: Got next segment. Inline: false Aug 7 22:21:14.636136774 (12h41m36.380193634s) [upload=13066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:21:14.636142452 (12h41m36.380199312s) [upload=13066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:21:16.211357551 (12h41m37.955414411s) [seg_pos={0 0}, upload=13066]: Begin upload segment... Aug 7 22:21:16.211426917 (12h41m37.955483784s) [noise=true , address=45.91.116.5:4327 , piece_id=IGBMWNLDFIKQY7AJ3UJ6JILAFLRQ5DZAE6FFO3EJ4NDPVWZXCK5A, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.214054689 (12h41m37.95811156s) [seg_pos={0 0}, upload=13066]: Done begin upload segment. Aug 7 22:21:16.214143921 (12h41m37.958200791s) [noise=true , address=104.36.82.2:4754 , piece_id=PQJBMG6FQ3HUTE26IBB5R3WQCAJTFKMGECOXSJPV2VDHP33JOPUA, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.214660710 (12h41m37.95871758s) [noise=true , address=104.36.82.59:4336 , piece_id=5AVF5EEMC334PPHVOSYDAYZ5CKQ4F5LE6BRD5G446NRRJNYQBOYQ, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.215094607 (12h41m37.959151469s) [noise=true , address=104.36.82.44:4904 , piece_id=RWD3KVEEHT5BUVYBAHRHZFCBTCZUTNHRD6PU7OWOUIF2QVBZXSSQ, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.215554381 (12h41m37.959611251s) [noise=true , address=104.36.82.58:4270 , piece_id=LUZ6HPESCVZDYNDVHR2QIS6HM2IIRK3IQB6BSV2VVVB3OFRZVMFQ, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.215879389 (12h41m37.959936254s) [noise=true , address=104.36.82.60:4339 , piece_id=FI3A54HMEOKZJ2FC2ZOLNQXW6NR7AUD26HBVFFGEGTG3AYRZD73A, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.216161781 (12h41m37.960218643s) [noise=true , address=104.36.82.61:4397 , piece_id=7FJT5CLASEMVAP4HSGHP3XQTNBKVRP2STW2YF6H2UDLHFPUHKMMQ, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.216489433 (12h41m37.960546284s) [noise=true , address=104.36.82.17:4344 , piece_id=CCKNSMXAJAO3QUJDDZ7POWA7NZ63KACQQAJRU37RPG5YMXVMOVZA, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.216758002 (12h41m37.960814865s) [noise=true , address=104.36.82.47:4525 , piece_id=62GQ6IOY4OBZAPRBVZ2OLLA7R4DVQVMWNQZAR67BTPQUX2HUEZRA, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.217037043 (12h41m37.961093908s) [noise=true , address=104.36.82.13:4191 , piece_id=NUNWAQKF465J5JQLYQBOCH75SAXPR5F3OPBDDTC6AWWBSKB6MSUA, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.217313215 (12h41m37.961370077s) [noise=true , address=64.49.13.140:4509 , piece_id=CNKDQOVU53HJ6EDOSZOXRSFOKQBNKWERZSFJQA5F24FZBR6XEEQA, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.217587290 (12h41m37.961644153s) [noise=true , address=45.91.116.9:4292 , piece_id=NO3S4IIMN5JWUNMMNQS6SDFLCDPW3MF3LK4BDZ4IVIZMRWBHSTQQ, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.217852877 (12h41m37.961909742s) [noise=true , address=104.36.82.43:4849 , piece_id=ECUDD3LRMPQMXZW6MRFLFHVPAO3RHCV6MWZYOJSASL4ILT54TFFA, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.218138868 (12h41m37.962195731s) [noise=true , address=64.49.13.26:4956 , piece_id=VHWLGKKB5MBWQOD3KVNXE3YYCK5TZ56UU25BTACIPJQZJA7PHA7Q, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.218358245 (12h41m37.962415106s) [noise=true , address=104.36.82.62:4435 , piece_id=O3PP5IRA3MLOLM35Z2U2BLM7D6GPH6WMHWANUF7AAGTW7IW6CZBQ, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.218693596 (12h41m37.962750459s) [noise=true , address=79.127.205.233:4647 , piece_id=6YIT5HDQJ6CN63ZFANACICFS4M2OQEPOSFIV36QXS5VLJZ2BRR7A, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.218971752 (12h41m37.963028615s) [noise=true , address=104.36.82.60:4354 , piece_id=A3YTMCWULXHPBRVL6TDQOBNYTQQWNKB2UZPKAPDH2CCG5SMV6QTA, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.219339785 (12h41m37.963396657s) [noise=true , address=104.36.82.9:4036 , piece_id=UG2DKHGMS5NC3BTN7QQU23JXBUQ7UFVN6FZKPBZHYUOQ6SEJV5WA, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.820211995 (12h41m38.564268846s) [upload=13066]: Next returned nil segment Aug 7 22:21:16.820729442 (12h41m38.564786312s) [upload=13066]: Waiting for error group managing segments... Aug 7 22:21:16.974534776 (12h41m38.71859164s) [noise=true , address=104.36.82.61:4397 , piece_id=7FJT5CLASEMVAP4HSGHP3XQTNBKVRP2STW2YF6H2UDLHFPUHKMMQ, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:16.995935209 (12h41m38.739992059s) [noise=true , address=104.36.82.9:4036 , piece_id=UG2DKHGMS5NC3BTN7QQU23JXBUQ7UFVN6FZKPBZHYUOQ6SEJV5WA, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.005410191 (12h41m38.749467042s) [noise=true , address=104.36.82.60:4354 , piece_id=A3YTMCWULXHPBRVL6TDQOBNYTQQWNKB2UZPKAPDH2CCG5SMV6QTA, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.012303228 (12h41m38.756360079s) [noise=true , address=104.36.82.17:4344 , piece_id=CCKNSMXAJAO3QUJDDZ7POWA7NZ63KACQQAJRU37RPG5YMXVMOVZA, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.047899041 (12h41m38.791955903s) [noise=true , address=104.36.82.13:4191 , piece_id=NUNWAQKF465J5JQLYQBOCH75SAXPR5F3OPBDDTC6AWWBSKB6MSUA, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.065632282 (12h41m38.809689142s) [noise=true , address=104.36.82.60:4339 , piece_id=FI3A54HMEOKZJ2FC2ZOLNQXW6NR7AUD26HBVFFGEGTG3AYRZD73A, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.074330593 (12h41m38.818387457s) [noise=true , address=104.36.82.62:4435 , piece_id=O3PP5IRA3MLOLM35Z2U2BLM7D6GPH6WMHWANUF7AAGTW7IW6CZBQ, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.205659931 (12h41m38.949716794s) [noise=true , address=104.36.82.59:4336 , piece_id=5AVF5EEMC334PPHVOSYDAYZ5CKQ4F5LE6BRD5G446NRRJNYQBOYQ, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.304344891 (12h41m39.048401752s) [noise=true , address=45.91.116.5:4327 , piece_id=IGBMWNLDFIKQY7AJ3UJ6JILAFLRQ5DZAE6FFO3EJ4NDPVWZXCK5A, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.476320837 (12h41m39.220377687s) [noise=true , address=64.49.13.140:4509 , piece_id=CNKDQOVU53HJ6EDOSZOXRSFOKQBNKWERZSFJQA5F24FZBR6XEEQA, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.498349653 (12h41m39.242406517s) [noise=true , address=64.49.13.26:4956 , piece_id=VHWLGKKB5MBWQOD3KVNXE3YYCK5TZ56UU25BTACIPJQZJA7PHA7Q, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.566667384 (12h41m39.310724234s) [noise=true , address=104.36.82.47:4525 , piece_id=62GQ6IOY4OBZAPRBVZ2OLLA7R4DVQVMWNQZAR67BTPQUX2HUEZRA, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.572654381 (12h41m39.316711251s) [noise=true , address=79.127.205.233:4647 , piece_id=6YIT5HDQJ6CN63ZFANACICFS4M2OQEPOSFIV36QXS5VLJZ2BRR7A, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:18.002510167 (12h41m39.746567013s) [noise=true , address=45.91.116.9:4292 , piece_id=NO3S4IIMN5JWUNMMNQS6SDFLCDPW3MF3LK4BDZ4IVIZMRWBHSTQQ, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:18.002534446 (12h41m39.746591307s) [seg_pos={0 0}, upload=13066]: Segment reached optimal threshold of 14 pieces. Aug 7 22:21:18.002684920 (12h41m39.746741766s) [noise=true , address=104.36.82.43:4849 , piece_id=ECUDD3LRMPQMXZW6MRFLFHVPAO3RHCV6MWZYOJSASL4ILT54TFFA, seg_pos={0 0}, upload=13066]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12goRcYW39Qqga4HW1i6cg6VGKJFUxnyTJBe6HRnZiLAU3LonWn): protocol: expected piece hash; context canceled; EOF Aug 7 22:21:18.002753282 (12h41m39.746810128s) [noise=true , address=104.36.82.58:4270 , piece_id=LUZ6HPESCVZDYNDVHR2QIS6HM2IIRK3IQB6BSV2VVVB3OFRZVMFQ, seg_pos={0 0}, upload=13066]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sJfMr1nMTpizd5PZ2bZ6M8tZWhFL2ukdJ3cNSRQtdUHNJhg81): protocol: expected piece hash; context canceled; EOF Aug 7 22:21:18.002945708 (12h41m39.747002555s) [noise=true , address=104.36.82.2:4754 , piece_id=PQJBMG6FQ3HUTE26IBB5R3WQCAJTFKMGECOXSJPV2VDHP33JOPUA, seg_pos={0 0}, upload=13066]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125aPwoHeFabk6zZ6bkGNj3USJYgTQG3RB1QhhXssj7Vik2AuCs): protocol: expected piece hash; context canceled; EOF Aug 7 22:21:18.003059994 (12h41m39.74711684s) [noise=true , address=104.36.82.44:4904 , piece_id=RWD3KVEEHT5BUVYBAHRHZFCBTCZUTNHRD6PU7OWOUIF2QVBZXSSQ, seg_pos={0 0}, upload=13066]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: write tcp 139.59.144.22:47438->104.36.82.44:4904: use of closed network connection; write tcp 139.59.144.22:47438->104.36.82.44:4904: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47438->104.36.82.44:4904: use of closed network connection Aug 7 22:21:18.003080731 (12h41m39.747137592s) [seg_pos={0 0}, upload=13066]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 22:21:18.003298604 (12h41m39.747355464s) [upload=13066]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 22:21:18.003304952 (12h41m39.747361812s) [upload=13066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:30:15.962677772 (12h50m37.706734624s) [upload=13066]: Done uploading segments... Aug 7 22:30:15.962709891 (12h50m37.706766753s) [upload=13066]: Upload finished. err: metaclient: context deadline exceeded Duration: 3.184907974s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [.....X...X..........................X...............X.......] duration: 1.169946205s FAIL (2025-08-07 22:35:46 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== d=13120]: Done uploading segments... Aug 7 22:25:13.463635961 (12h45m35.207692812s) [upload=13120]: Upload finished. err: <nil> Aug 7 22:25:36.676221958 (12h45m58.42027881s) [upload=13126]: Starting upload Aug 7 22:25:36.676280024 (12h45m58.420336886s) [upload=13126]: Done starting upload Aug 7 22:25:36.676348826 (12h45m58.420405681s) [upload=13126]: Uploading segments... Aug 7 22:25:36.676415761 (12h45m58.42047263s) [upload=13126]: Got next segment. Inline: true Aug 7 22:25:36.676421620 (12h45m58.420478481s) [upload=13126]: Waiting for error group managing segments... Aug 7 22:25:36.676463453 (12h45m58.420520304s) [upload=13126]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:25:36.676469083 (12h45m58.420525944s) [upload=13126]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:25:36.676472693 (12h45m58.420529553s) [upload=13126]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:25:36.846956085 (12h45m58.591012935s) [upload=13126]: Done uploading segments... Aug 7 22:25:36.846976507 (12h45m58.591033359s) [upload=13126]: Upload finished. err: <nil> Aug 7 22:26:18.836000763 (12h46m40.580057613s) [upload=13137]: Starting upload Aug 7 22:26:18.836038438 (12h46m40.580095288s) [upload=13137]: Done starting upload Aug 7 22:26:18.836098090 (12h46m40.58015494s) [upload=13137]: Uploading segments... Aug 7 22:26:18.836143460 (12h46m40.580200331s) [upload=13137]: Got next segment. Inline: true Aug 7 22:26:18.836148461 (12h46m40.580205309s) [upload=13137]: Waiting for error group managing segments... Aug 7 22:26:18.836191879 (12h46m40.580248725s) [upload=13137]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:26:18.836197487 (12h46m40.580254348s) [upload=13137]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:26:18.836205083 (12h46m40.580261941s) [upload=13137]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:26:18.978690589 (12h46m40.722747448s) [upload=13137]: Done uploading segments... Aug 7 22:26:18.978706599 (12h46m40.722763445s) [upload=13137]: Upload finished. err: <nil> Aug 7 22:27:06.613274662 (12h47m28.357331513s) [upload=13150]: Starting upload Aug 7 22:27:06.613358594 (12h47m28.357415443s) [upload=13150]: Done starting upload Aug 7 22:27:06.613451706 (12h47m28.357508565s) [upload=13150]: Uploading segments... Aug 7 22:27:06.613532713 (12h47m28.357589583s) [upload=13150]: Got next segment. Inline: true Aug 7 22:27:06.613540195 (12h47m28.357597165s) [upload=13150]: Waiting for error group managing segments... Aug 7 22:27:06.613608323 (12h47m28.357665185s) [upload=13150]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:27:06.613616998 (12h47m28.35767387s) [upload=13150]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:27:06.613621123 (12h47m28.357677991s) [upload=13150]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:27:06.756270932 (12h47m28.500327782s) [upload=13150]: Done uploading segments... Aug 7 22:27:06.756289996 (12h47m28.500346845s) [upload=13150]: Upload finished. err: <nil> Aug 7 22:27:39.275964882 (12h48m1.020021742s) [upload=13159]: Starting upload Aug 7 22:27:39.276031028 (12h48m1.020087874s) [upload=13159]: Done starting upload Aug 7 22:27:39.276100015 (12h48m1.020156863s) [upload=13159]: Uploading segments... Aug 7 22:27:39.276162849 (12h48m1.020219721s) [upload=13159]: Got next segment. Inline: true Aug 7 22:27:39.276168125 (12h48m1.020224985s) [upload=13159]: Waiting for error group managing segments... Aug 7 22:27:39.276216009 (12h48m1.020272869s) [upload=13159]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:27:39.276220617 (12h48m1.020277478s) [upload=13159]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:27:39.276223278 (12h48m1.020280137s) [upload=13159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:27:39.491372003 (12h48m1.235428853s) [upload=13159]: Done uploading segments... Aug 7 22:27:39.491394004 (12h48m1.235450857s) [upload=13159]: Upload finished. err: <nil> Aug 7 22:28:03.413359947 (12h48m25.157416798s) [upload=13167]: Starting upload Aug 7 22:28:03.413424697 (12h48m25.157481559s) [upload=13167]: Done starting upload Aug 7 22:28:03.413546352 (12h48m25.157603207s) [upload=13167]: Uploading segments... Aug 7 22:28:03.413695999 (12h48m25.157752868s) [upload=13167]: Got next segment. Inline: true Aug 7 22:28:03.413705607 (12h48m25.157762476s) [upload=13167]: Waiting for error group managing segments... Aug 7 22:28:03.413788849 (12h48m25.157845716s) [upload=13167]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:28:03.413816554 (12h48m25.157873423s) [upload=13167]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:28:03.413820681 (12h48m25.157877544s) [upload=13167]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:28:03.561945313 (12h48m25.306002181s) [upload=13167]: Done uploading segments... Aug 7 22:28:03.561970962 (12h48m25.306027825s) [upload=13167]: Upload finished. err: <nil> Aug 7 22:28:25.535904284 (12h48m47.279961133s) [upload=13172]: Starting upload Aug 7 22:28:25.535961986 (12h48m47.280018836s) [upload=13172]: Done starting upload Aug 7 22:28:25.536032757 (12h48m47.280089607s) [upload=13172]: Uploading segments... Aug 7 22:28:25.536074131 (12h48m47.280131001s) [upload=13172]: Got next segment. Inline: true Aug 7 22:28:25.536089717 (12h48m47.280146578s) [upload=13172]: Waiting for error group managing segments... Aug 7 22:28:25.536134532 (12h48m47.280191382s) [upload=13172]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:28:25.536151821 (12h48m47.280208671s) [upload=13172]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:28:25.536166195 (12h48m47.280223058s) [upload=13172]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:28:26.345427619 (12h48m48.089484469s) [upload=13172]: Done uploading segments... Aug 7 22:28:26.345447034 (12h48m48.089503895s) [upload=13172]: Upload finished. err: <nil> Aug 7 22:28:49.552061521 (12h49m11.296118389s) [upload=13179]: Starting upload Aug 7 22:28:49.552134904 (12h49m11.296191767s) [upload=13179]: Done starting upload Aug 7 22:28:49.552208829 (12h49m11.296265676s) [upload=13179]: Uploading segments... Aug 7 22:28:49.552262194 (12h49m11.296319062s) [upload=13179]: Got next segment. Inline: true Aug 7 22:28:49.552268137 (12h49m11.296324998s) [upload=13179]: Waiting for error group managing segments... Aug 7 22:28:49.552316820 (12h49m11.296373667s) [upload=13179]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:28:49.552323169 (12h49m11.29638003s) [upload=13179]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:28:49.552327059 (12h49m11.29638392s) [upload=13179]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:28:49.693951093 (12h49m11.438007952s) [upload=13179]: Done uploading segments... Aug 7 22:28:49.693988901 (12h49m11.438045762s) [upload=13179]: Upload finished. err: <nil> Aug 7 22:29:09.820328663 (12h49m31.564385513s) [upload=13184]: Starting upload Aug 7 22:29:09.820384335 (12h49m31.564441196s) [upload=13184]: Done starting upload Aug 7 22:29:09.820445932 (12h49m31.564502786s) [upload=13184]: Uploading segments... Aug 7 22:29:09.820487651 (12h49m31.564544517s) [upload=13184]: Got next segment. Inline: true Aug 7 22:29:09.820492611 (12h49m31.564549472s) [upload=13184]: Waiting for error group managing segments... Aug 7 22:29:09.820534051 (12h49m31.564590901s) [upload=13184]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:29:09.820539062 (12h49m31.564595922s) [upload=13184]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:29:09.820542725 (12h49m31.564599585s) [upload=13184]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:29:10.028668214 (12h49m31.772725062s) [upload=13184]: Done uploading segments... Aug 7 22:29:10.028692776 (12h49m31.772749623s) [upload=13184]: Upload finished. err: <nil> Aug 7 22:30:21.625872621 (12h50m43.369929471s) [upload=13201]: Starting upload Aug 7 22:30:21.625976134 (12h50m43.370033002s) [upload=13201]: Done starting upload Aug 7 22:30:21.626054127 (12h50m43.370110973s) [upload=13201]: Uploading segments... Aug 7 22:30:21.626095581 (12h50m43.370152447s) [upload=13201]: Got next segment. Inline: true Aug 7 22:30:21.626101049 (12h50m43.370157915s) [upload=13201]: Waiting for error group managing segments... Aug 7 22:30:21.626141797 (12h50m43.370198644s) [upload=13201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:30:21.626147151 (12h50m43.370204015s) [upload=13201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:30:21.626149721 (12h50m43.370206581s) [upload=13201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:30:21.760255105 (12h50m43.504311956s) [upload=13201]: Done uploading segments... Aug 7 22:30:21.760273763 (12h50m43.504330614s) [upload=13201]: Upload finished. err: <nil> Aug 7 22:30:47.060687315 (12h51m8.804744175s) [upload=13211]: Starting upload Aug 7 22:30:47.060743560 (12h51m8.80480042s) [upload=13211]: Done starting upload Aug 7 22:30:47.060819066 (12h51m8.804875915s) [upload=13211]: Uploading segments... Aug 7 22:30:47.060872995 (12h51m8.804929861s) [upload=13211]: Got next segment. Inline: true Aug 7 22:30:47.060878710 (12h51m8.80493557s) [upload=13211]: Waiting for error group managing segments... Aug 7 22:30:47.060932110 (12h51m8.804988961s) [upload=13211]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:30:47.060937671 (12h51m8.80499453s) [upload=13211]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:30:47.060940905 (12h51m8.804997769s) [upload=13211]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:30:48.162928472 (12h51m9.906985333s) [upload=13211]: Done uploading segments... Aug 7 22:30:48.162948603 (12h51m9.907005457s) [upload=13211]: Upload finished. err: <nil> Duration: 1.169990384s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 2 out of 60 is failed (threshold=3) [..................................X............X............] duration: 186.28383ms remote: "tcp:185.244.226.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 22:35:39 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 186.387171ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [....................................X.......................] duration: 106.920646ms remote: "tcp:185.244.226.2:443" target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt FAIL (2025-08-07 22:35:31 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 106.946512ms edge linksharing bgp us1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 3 out of 60 is failed (threshold=3) [..........................................XXX...............] duration: 295.533087ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 22:35:28 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 295.647788ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 21 out of 60 is failed (threshold=3) [...........................X.XXXXXXXXXXXXXXXX.X.X.XX........] duration: 876.906286ms FAIL (2025-08-07 22:35:21 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 876.932206ms core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 19 out of 60 is failed (threshold=3) [..X.X...........................XXXXXXXXXXXX.XXXX.X.........] duration: 2.960273012s FAIL (2025-08-07 22:35:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.96028433s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [X...................................XXXXXXXXXXXXX..X........] duration: 2.991840226s FAIL (2025-08-07 22:35:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== 7 22:27:44.139472223 (12h48m5.883529083s) [upload=13160]: Got next segment. Inline: false Aug 7 22:27:44.139508151 (12h48m5.883565011s) [upload=13160]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:27:44.139513399 (12h48m5.883570259s) [upload=13160]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:27:44.245907064 (12h48m5.989963927s) [seg_pos={0 0}, upload=13160]: Begin upload segment... Aug 7 22:27:44.246039735 (12h48m5.990096597s) [noise=true , address=104.36.82.10:4079 , piece_id=2ADIUNAB5FPKLXMW46HBB75SJLYOKA4N3TYS6J72WZDKLQTSP23A, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.246596012 (12h48m5.990652882s) [noise=true , address=104.36.82.4:4801 , piece_id=U6XCDALXGOTJHRL2P4V7DOPIJJF6VUKB5RIJYOP3WXFTC6HQKQXQ, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.247006251 (12h48m5.991063115s) [noise=true , address=77.90.31.12:4590 , piece_id=IYZTGAPBVXXQNQVDNR4NZRPDNNIBETBQ26DZUZ2SODA2D2VQUXTA, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.247197178 (12h48m5.991254041s) [seg_pos={0 0}, upload=13160]: Done begin upload segment. Aug 7 22:27:44.247225152 (12h48m5.991282018s) [noise=true , address=104.36.82.5:4837 , piece_id=XDA3KTWSH5OJPHK4IFOPINMJAR6VJLL7B7WNO3MV2VT3HQCGJ2KA, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.247751444 (12h48m5.99180831s) [noise=true , address=104.36.82.6:4876 , piece_id=45DZFNAUCX6EJNPT4WBVLEIO2GZROYIF2NQ254LU32T6AE7P4ZRA, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.247956514 (12h48m5.992013382s) [noise=true , address=104.36.82.11:4115 , piece_id=B2C72IS3FGY7DT4SXLUG5FYHCN5ECAMCAIVT2ZNVCXML2RVCQZFQ, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.248499391 (12h48m5.992556252s) [noise=true , address=45.91.116.12:4225 , piece_id=N7R42AWZKTHBBPZY24OVKSBRTPGVBLJ3HBYB2ONRZ56F6YZK2XWQ, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.248606386 (12h48m5.992663249s) [noise=true , address=77.90.31.13:4628 , piece_id=FSARBSWO4VFKOYHINODNYG7BC4JEH6KX5Z3IOYIJ4DGQ3DCZNDJA, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.249110151 (12h48m5.99316702s) [noise=true , address=45.91.116.9:4261 , piece_id=ZESRB2XAQ4WPIB2HQKYLZKJDXDE65NXMRX4M5JE3EKRHROCT4JKA, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.249195913 (12h48m5.993252781s) [noise=true , address=104.36.82.13:4185 , piece_id=QI54Y53UTBR7JEQ4NPD5R37EC47364LEUWETTCGI3WQUKSHUTP5A, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.249587895 (12h48m5.993644759s) [noise=true , address=45.91.116.8:4047 , piece_id=LDURVPEWVV7OEPVY4L4ZER4H2EDUPJS45N55Y57VZUTLSTKGXSRA, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.249756005 (12h48m5.993812872s) [noise=true , address=77.90.31.11:4553 , piece_id=ZDJ5BW2XPNHEBNB6FBVWAH66T6SVKAUY3724YY6HZ25VBLKYFTIQ, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.250801847 (12h48m5.99485871s) [noise=true , address=77.90.31.10:4517 , piece_id=QHV63D3QBJ5GVDWNHGWJTLXQNHGK4PZ4OCT4UWCEQKHMYBDL26WA, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.251094602 (12h48m5.995151463s) [noise=true , address=77.90.31.18:4806 , piece_id=ZJLVNSXETRBXYNHIDLN7TSQU6OQVWTCOQFJLPWAMCPY2JHBTHVQA, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.251408354 (12h48m5.995465217s) [noise=true , address=104.36.82.10:4078 , piece_id=UCDTQMJTYXAMJXGRPKBGTJZ4OCF4P2BSCR6WR3KSXCK6S5PWWZ7A, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.251693979 (12h48m5.995750839s) [noise=true , address=45.91.116.5:4297 , piece_id=QEQZU67YDQXNW7ACIRVXJKX2HIKMZMAP23TCAMTB2RDK3RFKLLRQ, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.928589281 (12h48m6.672646155s) [upload=13160]: Next returned nil segment Aug 7 22:27:44.928610126 (12h48m6.672666995s) [upload=13160]: Waiting for error group managing segments... Aug 7 22:27:45.055730069 (12h48m6.799786923s) [noise=true , address=104.36.82.11:4115 , piece_id=B2C72IS3FGY7DT4SXLUG5FYHCN5ECAMCAIVT2ZNVCXML2RVCQZFQ, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.058230571 (12h48m6.802287421s) [noise=true , address=104.36.82.4:4801 , piece_id=U6XCDALXGOTJHRL2P4V7DOPIJJF6VUKB5RIJYOP3WXFTC6HQKQXQ, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.072176193 (12h48m6.816233043s) [noise=true , address=104.36.82.6:4876 , piece_id=45DZFNAUCX6EJNPT4WBVLEIO2GZROYIF2NQ254LU32T6AE7P4ZRA, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.076144433 (12h48m6.82020128s) [noise=true , address=104.36.82.5:4837 , piece_id=XDA3KTWSH5OJPHK4IFOPINMJAR6VJLL7B7WNO3MV2VT3HQCGJ2KA, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.083969611 (12h48m6.828026457s) [noise=true , address=104.36.82.13:4185 , piece_id=QI54Y53UTBR7JEQ4NPD5R37EC47364LEUWETTCGI3WQUKSHUTP5A, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.090499988 (12h48m6.834556849s) [noise=true , address=104.36.82.10:4078 , piece_id=UCDTQMJTYXAMJXGRPKBGTJZ4OCF4P2BSCR6WR3KSXCK6S5PWWZ7A, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.129675895 (12h48m6.873732757s) [noise=true , address=45.91.116.12:4225 , piece_id=N7R42AWZKTHBBPZY24OVKSBRTPGVBLJ3HBYB2ONRZ56F6YZK2XWQ, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.148504731 (12h48m6.892561581s) [noise=true , address=77.90.31.10:4517 , piece_id=QHV63D3QBJ5GVDWNHGWJTLXQNHGK4PZ4OCT4UWCEQKHMYBDL26WA, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.183435281 (12h48m6.927492128s) [noise=true , address=104.36.82.10:4079 , piece_id=2ADIUNAB5FPKLXMW46HBB75SJLYOKA4N3TYS6J72WZDKLQTSP23A, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.188056276 (12h48m6.932113137s) [noise=true , address=77.90.31.12:4590 , piece_id=IYZTGAPBVXXQNQVDNR4NZRPDNNIBETBQ26DZUZ2SODA2D2VQUXTA, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.189109860 (12h48m6.93316672s) [noise=true , address=77.90.31.13:4628 , piece_id=FSARBSWO4VFKOYHINODNYG7BC4JEH6KX5Z3IOYIJ4DGQ3DCZNDJA, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.191395584 (12h48m6.93545243s) [noise=true , address=45.91.116.9:4261 , piece_id=ZESRB2XAQ4WPIB2HQKYLZKJDXDE65NXMRX4M5JE3EKRHROCT4JKA, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.212167167 (12h48m6.956224014s) [noise=true , address=77.90.31.11:4553 , piece_id=ZDJ5BW2XPNHEBNB6FBVWAH66T6SVKAUY3724YY6HZ25VBLKYFTIQ, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.212191124 (12h48m6.956247983s) [seg_pos={0 0}, upload=13160]: Segment reached optimal threshold of 13 pieces. Aug 7 22:27:45.212404510 (12h48m6.956461359s) [noise=true , address=45.91.116.8:4047 , piece_id=LDURVPEWVV7OEPVY4L4ZER4H2EDUPJS45N55Y57VZUTLSTKGXSRA, seg_pos={0 0}, upload=13160]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: context canceled; context canceled Aug 7 22:27:45.212444310 (12h48m6.956501156s) [noise=true , address=77.90.31.18:4806 , piece_id=ZJLVNSXETRBXYNHIDLN7TSQU6OQVWTCOQFJLPWAMCPY2JHBTHVQA, seg_pos={0 0}, upload=13160]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: write tcp 139.59.144.22:44926->77.90.31.18:4806: use of closed network connection; write tcp 139.59.144.22:44926->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44926->77.90.31.18:4806: use of closed network connection Aug 7 22:27:45.212507508 (12h48m6.956564355s) [noise=true , address=45.91.116.5:4297 , piece_id=QEQZU67YDQXNW7ACIRVXJKX2HIKMZMAP23TCAMTB2RDK3RFKLLRQ, seg_pos={0 0}, upload=13160]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 139.59.144.22:41424->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:41424->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41424->45.91.116.5:4297: use of closed network connection Aug 7 22:27:45.212525428 (12h48m6.956582288s) [seg_pos={0 0}, upload=13160]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 22:27:45.212587349 (12h48m6.956644209s) [upload=13160]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 22:27:45.212591472 (12h48m6.956648332s) [upload=13160]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:27:45.467451329 (12h48m7.21150818s) [upload=13160]: Done uploading segments... Aug 7 22:27:45.467470231 (12h48m7.211527081s) [upload=13160]: Upload finished. err: <nil> Aug 7 22:28:46.300540067 (12h49m8.044596928s) [upload=13178]: Starting upload Aug 7 22:28:46.300629117 (12h49m8.044685979s) [upload=13178]: Done starting upload Aug 7 22:28:46.300797752 (12h49m8.044854603s) [upload=13178]: Uploading segments... Aug 7 22:28:46.301190738 (12h49m8.045247598s) [upload=13178]: Got next segment. Inline: false Aug 7 22:28:46.301268000 (12h49m8.04532486s) [upload=13178]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:28:46.301275318 (12h49m8.045332178s) [upload=13178]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:29:16.009299446 (12h49m37.753356315s) [upload=13178]: Next segment err: context deadline exceeded Aug 7 22:29:16.009371909 (12h49m37.753428759s) [upload=13178]: Done uploading segments... Aug 7 22:29:16.009392178 (12h49m37.753449035s) [upload=13178]: Upload finished. err: context deadline exceeded Duration: 2.991870554s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [......X...X..........................X...............X......] duration: 978.400338ms FAIL (2025-08-07 22:35:16 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== d=13120]: Done uploading segments... Aug 7 22:25:13.463635961 (12h45m35.207692812s) [upload=13120]: Upload finished. err: <nil> Aug 7 22:25:36.676221958 (12h45m58.42027881s) [upload=13126]: Starting upload Aug 7 22:25:36.676280024 (12h45m58.420336886s) [upload=13126]: Done starting upload Aug 7 22:25:36.676348826 (12h45m58.420405681s) [upload=13126]: Uploading segments... Aug 7 22:25:36.676415761 (12h45m58.42047263s) [upload=13126]: Got next segment. Inline: true Aug 7 22:25:36.676421620 (12h45m58.420478481s) [upload=13126]: Waiting for error group managing segments... Aug 7 22:25:36.676463453 (12h45m58.420520304s) [upload=13126]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:25:36.676469083 (12h45m58.420525944s) [upload=13126]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:25:36.676472693 (12h45m58.420529553s) [upload=13126]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:25:36.846956085 (12h45m58.591012935s) [upload=13126]: Done uploading segments... Aug 7 22:25:36.846976507 (12h45m58.591033359s) [upload=13126]: Upload finished. err: <nil> Aug 7 22:26:18.836000763 (12h46m40.580057613s) [upload=13137]: Starting upload Aug 7 22:26:18.836038438 (12h46m40.580095288s) [upload=13137]: Done starting upload Aug 7 22:26:18.836098090 (12h46m40.58015494s) [upload=13137]: Uploading segments... Aug 7 22:26:18.836143460 (12h46m40.580200331s) [upload=13137]: Got next segment. Inline: true Aug 7 22:26:18.836148461 (12h46m40.580205309s) [upload=13137]: Waiting for error group managing segments... Aug 7 22:26:18.836191879 (12h46m40.580248725s) [upload=13137]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:26:18.836197487 (12h46m40.580254348s) [upload=13137]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:26:18.836205083 (12h46m40.580261941s) [upload=13137]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:26:18.978690589 (12h46m40.722747448s) [upload=13137]: Done uploading segments... Aug 7 22:26:18.978706599 (12h46m40.722763445s) [upload=13137]: Upload finished. err: <nil> Aug 7 22:27:06.613274662 (12h47m28.357331513s) [upload=13150]: Starting upload Aug 7 22:27:06.613358594 (12h47m28.357415443s) [upload=13150]: Done starting upload Aug 7 22:27:06.613451706 (12h47m28.357508565s) [upload=13150]: Uploading segments... Aug 7 22:27:06.613532713 (12h47m28.357589583s) [upload=13150]: Got next segment. Inline: true Aug 7 22:27:06.613540195 (12h47m28.357597165s) [upload=13150]: Waiting for error group managing segments... Aug 7 22:27:06.613608323 (12h47m28.357665185s) [upload=13150]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:27:06.613616998 (12h47m28.35767387s) [upload=13150]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:27:06.613621123 (12h47m28.357677991s) [upload=13150]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:27:06.756270932 (12h47m28.500327782s) [upload=13150]: Done uploading segments... Aug 7 22:27:06.756289996 (12h47m28.500346845s) [upload=13150]: Upload finished. err: <nil> Aug 7 22:27:39.275964882 (12h48m1.020021742s) [upload=13159]: Starting upload Aug 7 22:27:39.276031028 (12h48m1.020087874s) [upload=13159]: Done starting upload Aug 7 22:27:39.276100015 (12h48m1.020156863s) [upload=13159]: Uploading segments... Aug 7 22:27:39.276162849 (12h48m1.020219721s) [upload=13159]: Got next segment. Inline: true Aug 7 22:27:39.276168125 (12h48m1.020224985s) [upload=13159]: Waiting for error group managing segments... Aug 7 22:27:39.276216009 (12h48m1.020272869s) [upload=13159]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:27:39.276220617 (12h48m1.020277478s) [upload=13159]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:27:39.276223278 (12h48m1.020280137s) [upload=13159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:27:39.491372003 (12h48m1.235428853s) [upload=13159]: Done uploading segments... Aug 7 22:27:39.491394004 (12h48m1.235450857s) [upload=13159]: Upload finished. err: <nil> Aug 7 22:28:03.413359947 (12h48m25.157416798s) [upload=13167]: Starting upload Aug 7 22:28:03.413424697 (12h48m25.157481559s) [upload=13167]: Done starting upload Aug 7 22:28:03.413546352 (12h48m25.157603207s) [upload=13167]: Uploading segments... Aug 7 22:28:03.413695999 (12h48m25.157752868s) [upload=13167]: Got next segment. Inline: true Aug 7 22:28:03.413705607 (12h48m25.157762476s) [upload=13167]: Waiting for error group managing segments... Aug 7 22:28:03.413788849 (12h48m25.157845716s) [upload=13167]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:28:03.413816554 (12h48m25.157873423s) [upload=13167]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:28:03.413820681 (12h48m25.157877544s) [upload=13167]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:28:03.561945313 (12h48m25.306002181s) [upload=13167]: Done uploading segments... Aug 7 22:28:03.561970962 (12h48m25.306027825s) [upload=13167]: Upload finished. err: <nil> Aug 7 22:28:25.535904284 (12h48m47.279961133s) [upload=13172]: Starting upload Aug 7 22:28:25.535961986 (12h48m47.280018836s) [upload=13172]: Done starting upload Aug 7 22:28:25.536032757 (12h48m47.280089607s) [upload=13172]: Uploading segments... Aug 7 22:28:25.536074131 (12h48m47.280131001s) [upload=13172]: Got next segment. Inline: true Aug 7 22:28:25.536089717 (12h48m47.280146578s) [upload=13172]: Waiting for error group managing segments... Aug 7 22:28:25.536134532 (12h48m47.280191382s) [upload=13172]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:28:25.536151821 (12h48m47.280208671s) [upload=13172]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:28:25.536166195 (12h48m47.280223058s) [upload=13172]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:28:26.345427619 (12h48m48.089484469s) [upload=13172]: Done uploading segments... Aug 7 22:28:26.345447034 (12h48m48.089503895s) [upload=13172]: Upload finished. err: <nil> Aug 7 22:28:49.552061521 (12h49m11.296118389s) [upload=13179]: Starting upload Aug 7 22:28:49.552134904 (12h49m11.296191767s) [upload=13179]: Done starting upload Aug 7 22:28:49.552208829 (12h49m11.296265676s) [upload=13179]: Uploading segments... Aug 7 22:28:49.552262194 (12h49m11.296319062s) [upload=13179]: Got next segment. Inline: true Aug 7 22:28:49.552268137 (12h49m11.296324998s) [upload=13179]: Waiting for error group managing segments... Aug 7 22:28:49.552316820 (12h49m11.296373667s) [upload=13179]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:28:49.552323169 (12h49m11.29638003s) [upload=13179]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:28:49.552327059 (12h49m11.29638392s) [upload=13179]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:28:49.693951093 (12h49m11.438007952s) [upload=13179]: Done uploading segments... Aug 7 22:28:49.693988901 (12h49m11.438045762s) [upload=13179]: Upload finished. err: <nil> Aug 7 22:29:09.820328663 (12h49m31.564385513s) [upload=13184]: Starting upload Aug 7 22:29:09.820384335 (12h49m31.564441196s) [upload=13184]: Done starting upload Aug 7 22:29:09.820445932 (12h49m31.564502786s) [upload=13184]: Uploading segments... Aug 7 22:29:09.820487651 (12h49m31.564544517s) [upload=13184]: Got next segment. Inline: true Aug 7 22:29:09.820492611 (12h49m31.564549472s) [upload=13184]: Waiting for error group managing segments... Aug 7 22:29:09.820534051 (12h49m31.564590901s) [upload=13184]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:29:09.820539062 (12h49m31.564595922s) [upload=13184]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:29:09.820542725 (12h49m31.564599585s) [upload=13184]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:29:10.028668214 (12h49m31.772725062s) [upload=13184]: Done uploading segments... Aug 7 22:29:10.028692776 (12h49m31.772749623s) [upload=13184]: Upload finished. err: <nil> Aug 7 22:30:21.625872621 (12h50m43.369929471s) [upload=13201]: Starting upload Aug 7 22:30:21.625976134 (12h50m43.370033002s) [upload=13201]: Done starting upload Aug 7 22:30:21.626054127 (12h50m43.370110973s) [upload=13201]: Uploading segments... Aug 7 22:30:21.626095581 (12h50m43.370152447s) [upload=13201]: Got next segment. Inline: true Aug 7 22:30:21.626101049 (12h50m43.370157915s) [upload=13201]: Waiting for error group managing segments... Aug 7 22:30:21.626141797 (12h50m43.370198644s) [upload=13201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:30:21.626147151 (12h50m43.370204015s) [upload=13201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:30:21.626149721 (12h50m43.370206581s) [upload=13201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:30:21.760255105 (12h50m43.504311956s) [upload=13201]: Done uploading segments... Aug 7 22:30:21.760273763 (12h50m43.504330614s) [upload=13201]: Upload finished. err: <nil> Aug 7 22:30:47.060687315 (12h51m8.804744175s) [upload=13211]: Starting upload Aug 7 22:30:47.060743560 (12h51m8.80480042s) [upload=13211]: Done starting upload Aug 7 22:30:47.060819066 (12h51m8.804875915s) [upload=13211]: Uploading segments... Aug 7 22:30:47.060872995 (12h51m8.804929861s) [upload=13211]: Got next segment. Inline: true Aug 7 22:30:47.060878710 (12h51m8.80493557s) [upload=13211]: Waiting for error group managing segments... Aug 7 22:30:47.060932110 (12h51m8.804988961s) [upload=13211]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:30:47.060937671 (12h51m8.80499453s) [upload=13211]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:30:47.060940905 (12h51m8.804997769s) [upload=13211]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:30:48.162928472 (12h51m9.906985333s) [upload=13211]: Done uploading segments... Aug 7 22:30:48.162948603 (12h51m9.907005457s) [upload=13211]: Upload finished. err: <nil> Duration: 978.42838ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 12 out of 60 is failed (threshold=3) [............XXXX..............................XXXXXXXX......] duration: 3.745021293s FAIL (2025-08-07 22:35:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== s... Aug 7 22:21:14.636082751 (12h41m36.380139602s) [upload=13066]: Got next segment. Inline: false Aug 7 22:21:14.636136774 (12h41m36.380193634s) [upload=13066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:21:14.636142452 (12h41m36.380199312s) [upload=13066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:21:16.211357551 (12h41m37.955414411s) [seg_pos={0 0}, upload=13066]: Begin upload segment... Aug 7 22:21:16.211426917 (12h41m37.955483784s) [noise=true , address=45.91.116.5:4327 , piece_id=IGBMWNLDFIKQY7AJ3UJ6JILAFLRQ5DZAE6FFO3EJ4NDPVWZXCK5A, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.214054689 (12h41m37.95811156s) [seg_pos={0 0}, upload=13066]: Done begin upload segment. Aug 7 22:21:16.214143921 (12h41m37.958200791s) [noise=true , address=104.36.82.2:4754 , piece_id=PQJBMG6FQ3HUTE26IBB5R3WQCAJTFKMGECOXSJPV2VDHP33JOPUA, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.214660710 (12h41m37.95871758s) [noise=true , address=104.36.82.59:4336 , piece_id=5AVF5EEMC334PPHVOSYDAYZ5CKQ4F5LE6BRD5G446NRRJNYQBOYQ, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.215094607 (12h41m37.959151469s) [noise=true , address=104.36.82.44:4904 , piece_id=RWD3KVEEHT5BUVYBAHRHZFCBTCZUTNHRD6PU7OWOUIF2QVBZXSSQ, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.215554381 (12h41m37.959611251s) [noise=true , address=104.36.82.58:4270 , piece_id=LUZ6HPESCVZDYNDVHR2QIS6HM2IIRK3IQB6BSV2VVVB3OFRZVMFQ, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.215879389 (12h41m37.959936254s) [noise=true , address=104.36.82.60:4339 , piece_id=FI3A54HMEOKZJ2FC2ZOLNQXW6NR7AUD26HBVFFGEGTG3AYRZD73A, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.216161781 (12h41m37.960218643s) [noise=true , address=104.36.82.61:4397 , piece_id=7FJT5CLASEMVAP4HSGHP3XQTNBKVRP2STW2YF6H2UDLHFPUHKMMQ, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.216489433 (12h41m37.960546284s) [noise=true , address=104.36.82.17:4344 , piece_id=CCKNSMXAJAO3QUJDDZ7POWA7NZ63KACQQAJRU37RPG5YMXVMOVZA, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.216758002 (12h41m37.960814865s) [noise=true , address=104.36.82.47:4525 , piece_id=62GQ6IOY4OBZAPRBVZ2OLLA7R4DVQVMWNQZAR67BTPQUX2HUEZRA, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.217037043 (12h41m37.961093908s) [noise=true , address=104.36.82.13:4191 , piece_id=NUNWAQKF465J5JQLYQBOCH75SAXPR5F3OPBDDTC6AWWBSKB6MSUA, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.217313215 (12h41m37.961370077s) [noise=true , address=64.49.13.140:4509 , piece_id=CNKDQOVU53HJ6EDOSZOXRSFOKQBNKWERZSFJQA5F24FZBR6XEEQA, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.217587290 (12h41m37.961644153s) [noise=true , address=45.91.116.9:4292 , piece_id=NO3S4IIMN5JWUNMMNQS6SDFLCDPW3MF3LK4BDZ4IVIZMRWBHSTQQ, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.217852877 (12h41m37.961909742s) [noise=true , address=104.36.82.43:4849 , piece_id=ECUDD3LRMPQMXZW6MRFLFHVPAO3RHCV6MWZYOJSASL4ILT54TFFA, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.218138868 (12h41m37.962195731s) [noise=true , address=64.49.13.26:4956 , piece_id=VHWLGKKB5MBWQOD3KVNXE3YYCK5TZ56UU25BTACIPJQZJA7PHA7Q, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.218358245 (12h41m37.962415106s) [noise=true , address=104.36.82.62:4435 , piece_id=O3PP5IRA3MLOLM35Z2U2BLM7D6GPH6WMHWANUF7AAGTW7IW6CZBQ, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.218693596 (12h41m37.962750459s) [noise=true , address=79.127.205.233:4647 , piece_id=6YIT5HDQJ6CN63ZFANACICFS4M2OQEPOSFIV36QXS5VLJZ2BRR7A, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.218971752 (12h41m37.963028615s) [noise=true , address=104.36.82.60:4354 , piece_id=A3YTMCWULXHPBRVL6TDQOBNYTQQWNKB2UZPKAPDH2CCG5SMV6QTA, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.219339785 (12h41m37.963396657s) [noise=true , address=104.36.82.9:4036 , piece_id=UG2DKHGMS5NC3BTN7QQU23JXBUQ7UFVN6FZKPBZHYUOQ6SEJV5WA, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.820211995 (12h41m38.564268846s) [upload=13066]: Next returned nil segment Aug 7 22:21:16.820729442 (12h41m38.564786312s) [upload=13066]: Waiting for error group managing segments... Aug 7 22:21:16.974534776 (12h41m38.71859164s) [noise=true , address=104.36.82.61:4397 , piece_id=7FJT5CLASEMVAP4HSGHP3XQTNBKVRP2STW2YF6H2UDLHFPUHKMMQ, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:16.995935209 (12h41m38.739992059s) [noise=true , address=104.36.82.9:4036 , piece_id=UG2DKHGMS5NC3BTN7QQU23JXBUQ7UFVN6FZKPBZHYUOQ6SEJV5WA, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.005410191 (12h41m38.749467042s) [noise=true , address=104.36.82.60:4354 , piece_id=A3YTMCWULXHPBRVL6TDQOBNYTQQWNKB2UZPKAPDH2CCG5SMV6QTA, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.012303228 (12h41m38.756360079s) [noise=true , address=104.36.82.17:4344 , piece_id=CCKNSMXAJAO3QUJDDZ7POWA7NZ63KACQQAJRU37RPG5YMXVMOVZA, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.047899041 (12h41m38.791955903s) [noise=true , address=104.36.82.13:4191 , piece_id=NUNWAQKF465J5JQLYQBOCH75SAXPR5F3OPBDDTC6AWWBSKB6MSUA, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.065632282 (12h41m38.809689142s) [noise=true , address=104.36.82.60:4339 , piece_id=FI3A54HMEOKZJ2FC2ZOLNQXW6NR7AUD26HBVFFGEGTG3AYRZD73A, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.074330593 (12h41m38.818387457s) [noise=true , address=104.36.82.62:4435 , piece_id=O3PP5IRA3MLOLM35Z2U2BLM7D6GPH6WMHWANUF7AAGTW7IW6CZBQ, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.205659931 (12h41m38.949716794s) [noise=true , address=104.36.82.59:4336 , piece_id=5AVF5EEMC334PPHVOSYDAYZ5CKQ4F5LE6BRD5G446NRRJNYQBOYQ, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.304344891 (12h41m39.048401752s) [noise=true , address=45.91.116.5:4327 , piece_id=IGBMWNLDFIKQY7AJ3UJ6JILAFLRQ5DZAE6FFO3EJ4NDPVWZXCK5A, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.476320837 (12h41m39.220377687s) [noise=true , address=64.49.13.140:4509 , piece_id=CNKDQOVU53HJ6EDOSZOXRSFOKQBNKWERZSFJQA5F24FZBR6XEEQA, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.498349653 (12h41m39.242406517s) [noise=true , address=64.49.13.26:4956 , piece_id=VHWLGKKB5MBWQOD3KVNXE3YYCK5TZ56UU25BTACIPJQZJA7PHA7Q, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.566667384 (12h41m39.310724234s) [noise=true , address=104.36.82.47:4525 , piece_id=62GQ6IOY4OBZAPRBVZ2OLLA7R4DVQVMWNQZAR67BTPQUX2HUEZRA, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.572654381 (12h41m39.316711251s) [noise=true , address=79.127.205.233:4647 , piece_id=6YIT5HDQJ6CN63ZFANACICFS4M2OQEPOSFIV36QXS5VLJZ2BRR7A, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:18.002510167 (12h41m39.746567013s) [noise=true , address=45.91.116.9:4292 , piece_id=NO3S4IIMN5JWUNMMNQS6SDFLCDPW3MF3LK4BDZ4IVIZMRWBHSTQQ, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:18.002534446 (12h41m39.746591307s) [seg_pos={0 0}, upload=13066]: Segment reached optimal threshold of 14 pieces. Aug 7 22:21:18.002684920 (12h41m39.746741766s) [noise=true , address=104.36.82.43:4849 , piece_id=ECUDD3LRMPQMXZW6MRFLFHVPAO3RHCV6MWZYOJSASL4ILT54TFFA, seg_pos={0 0}, upload=13066]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12goRcYW39Qqga4HW1i6cg6VGKJFUxnyTJBe6HRnZiLAU3LonWn): protocol: expected piece hash; context canceled; EOF Aug 7 22:21:18.002753282 (12h41m39.746810128s) [noise=true , address=104.36.82.58:4270 , piece_id=LUZ6HPESCVZDYNDVHR2QIS6HM2IIRK3IQB6BSV2VVVB3OFRZVMFQ, seg_pos={0 0}, upload=13066]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sJfMr1nMTpizd5PZ2bZ6M8tZWhFL2ukdJ3cNSRQtdUHNJhg81): protocol: expected piece hash; context canceled; EOF Aug 7 22:21:18.002945708 (12h41m39.747002555s) [noise=true , address=104.36.82.2:4754 , piece_id=PQJBMG6FQ3HUTE26IBB5R3WQCAJTFKMGECOXSJPV2VDHP33JOPUA, seg_pos={0 0}, upload=13066]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125aPwoHeFabk6zZ6bkGNj3USJYgTQG3RB1QhhXssj7Vik2AuCs): protocol: expected piece hash; context canceled; EOF Aug 7 22:21:18.003059994 (12h41m39.74711684s) [noise=true , address=104.36.82.44:4904 , piece_id=RWD3KVEEHT5BUVYBAHRHZFCBTCZUTNHRD6PU7OWOUIF2QVBZXSSQ, seg_pos={0 0}, upload=13066]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: write tcp 139.59.144.22:47438->104.36.82.44:4904: use of closed network connection; write tcp 139.59.144.22:47438->104.36.82.44:4904: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47438->104.36.82.44:4904: use of closed network connection Aug 7 22:21:18.003080731 (12h41m39.747137592s) [seg_pos={0 0}, upload=13066]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 22:21:18.003298604 (12h41m39.747355464s) [upload=13066]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 22:21:18.003304952 (12h41m39.747361812s) [upload=13066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:30:15.962677772 (12h50m37.706734624s) [upload=13066]: Done uploading segments... Aug 7 22:30:15.962709891 (12h50m37.706766753s) [upload=13066]: Upload finished. err: metaclient: context deadline exceeded Duration: 3.745049829s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.....X..X...........X......................X........X.......] duration: 3.092491173s FAIL (2025-08-07 22:35:14 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== pload=13204]: Uploading segments... Aug 7 22:30:31.937644158 (12h50m53.681701018s) [upload=13204]: Got next segment. Inline: false Aug 7 22:30:31.937701034 (12h50m53.681757885s) [upload=13204]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:30:31.937725980 (12h50m53.681782841s) [upload=13204]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:30:32.237884006 (12h50m53.981940856s) [seg_pos={0 0}, upload=13204]: Begin upload segment... Aug 7 22:30:32.237975798 (12h50m53.982032665s) [seg_pos={0 0}, upload=13204]: Done begin upload segment. Aug 7 22:30:32.238021241 (12h50m53.982078102s) [noise=true , address=104.36.82.34:4549 , piece_id=7XWTCOONFR2KIRHVGUD3SRVR37PCP52ENQJR743AUEKVQBFWPLZQ, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.238037974 (12h50m53.982094843s) [noise=true , address=104.36.82.37:4673 , piece_id=AVJWSEWIKELDXA3DV62DMV3RZLHJ4EJR4W3V6IGGXFUQHAMWFPJA, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.238498130 (12h50m53.982554995s) [noise=true , address=104.36.82.17:4346 , piece_id=VDEOT6B6CUYIMFY3WMMTJVNJTXURII72Q7NBY2DAVHUNN4EHWC4A, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.238509513 (12h50m53.982566376s) [noise=true , address=104.36.82.12:4179 , piece_id=K7X5PR5NYK7LR3BDCQS66YAGYGWVFDXSG4TM42WIHR4XURDJWJ6Q, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.238837373 (12h50m53.982894233s) [noise=true , address=104.36.82.47:4527 , piece_id=RS7VNDZTLKOR4OSKZ3N5W3ELXLDN3BRXJNWHZEFBYW2LNM3NLZVA, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.238877922 (12h50m53.982934787s) [noise=true , address=104.36.82.58:4288 , piece_id=XTZGZNZEJBM5RYKYJXJYY5ED2MFA2ADMTJB7FMILXDR3W7E7YOPQ, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.239360083 (12h50m53.983416953s) [noise=true , address=104.36.82.44:4873 , piece_id=G7J764SMJNHCLSGTTCSDYISA6UNAFENBKL44CXJOH2TC46C56EFQ, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.239441321 (12h50m53.983498189s) [noise=true , address=104.36.82.39:4759 , piece_id=POKLEQJD3YYQSEJFZWNGKXULGIJ5JCQHHLMDBC3TAFYGZEWIXY7Q, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.239783955 (12h50m53.983840827s) [noise=true , address=79.127.205.245:4220 , piece_id=LX7ZN5QBMAPUFCIIOEOW4YXX72WGXH3A5ICGT5A6LVUCYLBLOCDA, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.239893333 (12h50m53.983950196s) [noise=true , address=104.36.82.23:4479 , piece_id=FCBSRKC6XA7MA7EIJ3GCB7XPTPC43OWYRXTJFFFJXU4I6LNRQPGA, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.240152214 (12h50m53.984209078s) [noise=true , address=104.36.82.59:4320 , piece_id=PLFQLBPXXXYN4EIVX5Z56WO7YCHMYKH5EBPNQV3KAEC4Q7NNTJZA, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.240280854 (12h50m53.984337724s) [noise=true , address=77.90.31.17:4777 , piece_id=5WWWEOD7W4KTC7WDC6QC5IHM4AICAYMJRQBM3LGVBYROMYZSMZBQ, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.240446242 (12h50m53.984503111s) [noise=true , address=104.36.82.46:4496 , piece_id=EQE7MYB2HOMCJNKQE4XZE62KTDQUUJKILDCDONOI2YKAURSM4I4A, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.240677807 (12h50m53.984734657s) [noise=true , address=104.36.82.6:4884 , piece_id=J66M7GUJFLMQXPUT23IIWFRFHYY3J726DRUS4PNI4GMCL5ORNHOQ, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.240762860 (12h50m53.984819723s) [noise=true , address=64.49.13.26:4946 , piece_id=KB7JRCJHFMVHCJKG4NZQ5QJ3O5IFOWW2IBBWSP6TQ6DSTWBABT5A, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.241124053 (12h50m53.985180903s) [noise=true , address=104.36.82.60:4346 , piece_id=NIUOHVNXXWV2K7GZJFVKBR2ZIGTJBIYLC45G6ADMBDLJKOKIVEVQ, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.241500880 (12h50m53.985557731s) [noise=true , address=104.36.82.8:4982 , piece_id=Z5TUKKS3PLZFFUOGHJENT7JIB4FJBS5WWRBEYTQ2USSGVGN56OUA, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.241813854 (12h50m53.985870704s) [noise=true , address=104.36.82.19:4406 , piece_id=6Z6NKSRY63B3XDEPE2H3X5HPY6JYTYHLOVEQX5DY2X4AKURLTGZA, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.832548156 (12h50m54.576605008s) [upload=13204]: Next returned nil segment Aug 7 22:30:32.832566279 (12h50m54.576623131s) [upload=13204]: Waiting for error group managing segments... Aug 7 22:30:32.986982167 (12h50m54.731039021s) [noise=true , address=104.36.82.44:4873 , piece_id=G7J764SMJNHCLSGTTCSDYISA6UNAFENBKL44CXJOH2TC46C56EFQ, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:32.995526266 (12h50m54.739583133s) [noise=true , address=104.36.82.39:4759 , piece_id=POKLEQJD3YYQSEJFZWNGKXULGIJ5JCQHHLMDBC3TAFYGZEWIXY7Q, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.029305286 (12h50m54.773362132s) [noise=true , address=104.36.82.46:4496 , piece_id=EQE7MYB2HOMCJNKQE4XZE62KTDQUUJKILDCDONOI2YKAURSM4I4A, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.030396152 (12h50m54.774453021s) [noise=true , address=104.36.82.23:4479 , piece_id=FCBSRKC6XA7MA7EIJ3GCB7XPTPC43OWYRXTJFFFJXU4I6LNRQPGA, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.058142476 (12h50m54.802199375s) [noise=true , address=104.36.82.19:4406 , piece_id=6Z6NKSRY63B3XDEPE2H3X5HPY6JYTYHLOVEQX5DY2X4AKURLTGZA, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.175489481 (12h50m54.919546344s) [noise=true , address=104.36.82.59:4320 , piece_id=PLFQLBPXXXYN4EIVX5Z56WO7YCHMYKH5EBPNQV3KAEC4Q7NNTJZA, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.206541092 (12h50m54.950597958s) [noise=true , address=77.90.31.17:4777 , piece_id=5WWWEOD7W4KTC7WDC6QC5IHM4AICAYMJRQBM3LGVBYROMYZSMZBQ, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.207881440 (12h50m54.951938305s) [noise=true , address=104.36.82.12:4179 , piece_id=K7X5PR5NYK7LR3BDCQS66YAGYGWVFDXSG4TM42WIHR4XURDJWJ6Q, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.223413715 (12h50m54.967470563s) [noise=true , address=104.36.82.34:4549 , piece_id=7XWTCOONFR2KIRHVGUD3SRVR37PCP52ENQJR743AUEKVQBFWPLZQ, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.235435210 (12h50m54.979492061s) [noise=true , address=104.36.82.8:4982 , piece_id=Z5TUKKS3PLZFFUOGHJENT7JIB4FJBS5WWRBEYTQ2USSGVGN56OUA, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.309334498 (12h50m55.053391362s) [noise=true , address=104.36.82.17:4346 , piece_id=VDEOT6B6CUYIMFY3WMMTJVNJTXURII72Q7NBY2DAVHUNN4EHWC4A, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.502128838 (12h50m55.246185692s) [noise=true , address=64.49.13.26:4946 , piece_id=KB7JRCJHFMVHCJKG4NZQ5QJ3O5IFOWW2IBBWSP6TQ6DSTWBABT5A, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.544691546 (12h50m55.288748396s) [noise=true , address=79.127.205.245:4220 , piece_id=LX7ZN5QBMAPUFCIIOEOW4YXX72WGXH3A5ICGT5A6LVUCYLBLOCDA, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.633287759 (12h50m55.377344611s) [noise=true , address=104.36.82.60:4346 , piece_id=NIUOHVNXXWV2K7GZJFVKBR2ZIGTJBIYLC45G6ADMBDLJKOKIVEVQ, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.633308567 (12h50m55.377365418s) [seg_pos={0 0}, upload=13204]: Segment reached optimal threshold of 14 pieces. Aug 7 22:30:33.633410618 (12h50m55.377467467s) [noise=true , address=104.36.82.6:4884 , piece_id=J66M7GUJFLMQXPUT23IIWFRFHYY3J726DRUS4PNI4GMCL5ORNHOQ, seg_pos={0 0}, upload=13204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1c4THqUrQMeqY94EGiZSNKQDDnvycEZbGMrYxrb4s4XuaRe8br): protocol: expected piece hash; context canceled; EOF Aug 7 22:30:33.633759470 (12h50m55.377816321s) [noise=true , address=104.36.82.47:4527 , piece_id=RS7VNDZTLKOR4OSKZ3N5W3ELXLDN3BRXJNWHZEFBYW2LNM3NLZVA, seg_pos={0 0}, upload=13204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17xmkbFoj3PTJnAygsjRAcfdd17AfSkPP2xdZp68GGPrvfMuU6): protocol: expected piece hash; context canceled; EOF Aug 7 22:30:33.633848731 (12h50m55.377905589s) [noise=true , address=104.36.82.58:4288 , piece_id=XTZGZNZEJBM5RYKYJXJYY5ED2MFA2ADMTJB7FMILXDR3W7E7YOPQ, seg_pos={0 0}, upload=13204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuahEyfok8Hk4fBNcdrZQWoQZiCXfNWdbfqPMnbEPmdZQ3R1o): protocol: expected piece hash; context canceled; EOF Aug 7 22:30:33.634152395 (12h50m55.378209251s) [noise=true , address=104.36.82.37:4673 , piece_id=AVJWSEWIKELDXA3DV62DMV3RZLHJ4EJR4W3V6IGGXFUQHAMWFPJA, seg_pos={0 0}, upload=13204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NkBjmh5iL2BtckR19qgXQ4wnfvnbpA9erSuMCTEijwX7ZdHGL): protocol: write tcp 139.59.144.22:44486->104.36.82.37:4673: use of closed network connection; write tcp 139.59.144.22:44486->104.36.82.37:4673: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44486->104.36.82.37:4673: use of closed network connection Aug 7 22:30:33.634174413 (12h50m55.378231287s) [seg_pos={0 0}, upload=13204]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 22:30:33.634256322 (12h50m55.378313188s) [upload=13204]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 22:30:33.634262633 (12h50m55.378319509s) [upload=13204]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:30:34.036796742 (12h50m55.7808536s) [upload=13204]: Done uploading segments... Aug 7 22:30:34.036819686 (12h50m55.780876545s) [upload=13204]: Upload finished. err: <nil> Duration: 3.092518867s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 3 out of 60 is failed (threshold=3) [...........................................XXX..............] duration: 333.418192ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 22:35:03 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 333.504725ms edge gateway vhost us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 18 out of 60 is failed (threshold=3) [.................................XXXXXXXXXXXXXXXXXX.........] duration: 2.745720067s FAIL (2025-08-07 22:35:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.745748444s edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 2 out of 60 is failed (threshold=3) [...................................X............X...........] duration: 222.912684ms remote: "tcp:185.244.226.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 22:35:00 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 222.94535ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 20 out of 60 is failed (threshold=3) [XXXXXXXXXXXX............................X.XXXXX.XX..........] duration: 2.27103941s FAIL (2025-08-07 22:34:58 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.271076418s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [......X..X...........X......................X........X......] duration: 3.057689444s FAIL (2025-08-07 22:34:50 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== pload=13204]: Uploading segments... Aug 7 22:30:31.937644158 (12h50m53.681701018s) [upload=13204]: Got next segment. Inline: false Aug 7 22:30:31.937701034 (12h50m53.681757885s) [upload=13204]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:30:31.937725980 (12h50m53.681782841s) [upload=13204]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:30:32.237884006 (12h50m53.981940856s) [seg_pos={0 0}, upload=13204]: Begin upload segment... Aug 7 22:30:32.237975798 (12h50m53.982032665s) [seg_pos={0 0}, upload=13204]: Done begin upload segment. Aug 7 22:30:32.238021241 (12h50m53.982078102s) [noise=true , address=104.36.82.34:4549 , piece_id=7XWTCOONFR2KIRHVGUD3SRVR37PCP52ENQJR743AUEKVQBFWPLZQ, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.238037974 (12h50m53.982094843s) [noise=true , address=104.36.82.37:4673 , piece_id=AVJWSEWIKELDXA3DV62DMV3RZLHJ4EJR4W3V6IGGXFUQHAMWFPJA, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.238498130 (12h50m53.982554995s) [noise=true , address=104.36.82.17:4346 , piece_id=VDEOT6B6CUYIMFY3WMMTJVNJTXURII72Q7NBY2DAVHUNN4EHWC4A, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.238509513 (12h50m53.982566376s) [noise=true , address=104.36.82.12:4179 , piece_id=K7X5PR5NYK7LR3BDCQS66YAGYGWVFDXSG4TM42WIHR4XURDJWJ6Q, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.238837373 (12h50m53.982894233s) [noise=true , address=104.36.82.47:4527 , piece_id=RS7VNDZTLKOR4OSKZ3N5W3ELXLDN3BRXJNWHZEFBYW2LNM3NLZVA, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.238877922 (12h50m53.982934787s) [noise=true , address=104.36.82.58:4288 , piece_id=XTZGZNZEJBM5RYKYJXJYY5ED2MFA2ADMTJB7FMILXDR3W7E7YOPQ, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.239360083 (12h50m53.983416953s) [noise=true , address=104.36.82.44:4873 , piece_id=G7J764SMJNHCLSGTTCSDYISA6UNAFENBKL44CXJOH2TC46C56EFQ, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.239441321 (12h50m53.983498189s) [noise=true , address=104.36.82.39:4759 , piece_id=POKLEQJD3YYQSEJFZWNGKXULGIJ5JCQHHLMDBC3TAFYGZEWIXY7Q, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.239783955 (12h50m53.983840827s) [noise=true , address=79.127.205.245:4220 , piece_id=LX7ZN5QBMAPUFCIIOEOW4YXX72WGXH3A5ICGT5A6LVUCYLBLOCDA, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.239893333 (12h50m53.983950196s) [noise=true , address=104.36.82.23:4479 , piece_id=FCBSRKC6XA7MA7EIJ3GCB7XPTPC43OWYRXTJFFFJXU4I6LNRQPGA, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.240152214 (12h50m53.984209078s) [noise=true , address=104.36.82.59:4320 , piece_id=PLFQLBPXXXYN4EIVX5Z56WO7YCHMYKH5EBPNQV3KAEC4Q7NNTJZA, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.240280854 (12h50m53.984337724s) [noise=true , address=77.90.31.17:4777 , piece_id=5WWWEOD7W4KTC7WDC6QC5IHM4AICAYMJRQBM3LGVBYROMYZSMZBQ, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.240446242 (12h50m53.984503111s) [noise=true , address=104.36.82.46:4496 , piece_id=EQE7MYB2HOMCJNKQE4XZE62KTDQUUJKILDCDONOI2YKAURSM4I4A, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.240677807 (12h50m53.984734657s) [noise=true , address=104.36.82.6:4884 , piece_id=J66M7GUJFLMQXPUT23IIWFRFHYY3J726DRUS4PNI4GMCL5ORNHOQ, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.240762860 (12h50m53.984819723s) [noise=true , address=64.49.13.26:4946 , piece_id=KB7JRCJHFMVHCJKG4NZQ5QJ3O5IFOWW2IBBWSP6TQ6DSTWBABT5A, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.241124053 (12h50m53.985180903s) [noise=true , address=104.36.82.60:4346 , piece_id=NIUOHVNXXWV2K7GZJFVKBR2ZIGTJBIYLC45G6ADMBDLJKOKIVEVQ, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.241500880 (12h50m53.985557731s) [noise=true , address=104.36.82.8:4982 , piece_id=Z5TUKKS3PLZFFUOGHJENT7JIB4FJBS5WWRBEYTQ2USSGVGN56OUA, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.241813854 (12h50m53.985870704s) [noise=true , address=104.36.82.19:4406 , piece_id=6Z6NKSRY63B3XDEPE2H3X5HPY6JYTYHLOVEQX5DY2X4AKURLTGZA, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.832548156 (12h50m54.576605008s) [upload=13204]: Next returned nil segment Aug 7 22:30:32.832566279 (12h50m54.576623131s) [upload=13204]: Waiting for error group managing segments... Aug 7 22:30:32.986982167 (12h50m54.731039021s) [noise=true , address=104.36.82.44:4873 , piece_id=G7J764SMJNHCLSGTTCSDYISA6UNAFENBKL44CXJOH2TC46C56EFQ, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:32.995526266 (12h50m54.739583133s) [noise=true , address=104.36.82.39:4759 , piece_id=POKLEQJD3YYQSEJFZWNGKXULGIJ5JCQHHLMDBC3TAFYGZEWIXY7Q, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.029305286 (12h50m54.773362132s) [noise=true , address=104.36.82.46:4496 , piece_id=EQE7MYB2HOMCJNKQE4XZE62KTDQUUJKILDCDONOI2YKAURSM4I4A, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.030396152 (12h50m54.774453021s) [noise=true , address=104.36.82.23:4479 , piece_id=FCBSRKC6XA7MA7EIJ3GCB7XPTPC43OWYRXTJFFFJXU4I6LNRQPGA, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.058142476 (12h50m54.802199375s) [noise=true , address=104.36.82.19:4406 , piece_id=6Z6NKSRY63B3XDEPE2H3X5HPY6JYTYHLOVEQX5DY2X4AKURLTGZA, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.175489481 (12h50m54.919546344s) [noise=true , address=104.36.82.59:4320 , piece_id=PLFQLBPXXXYN4EIVX5Z56WO7YCHMYKH5EBPNQV3KAEC4Q7NNTJZA, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.206541092 (12h50m54.950597958s) [noise=true , address=77.90.31.17:4777 , piece_id=5WWWEOD7W4KTC7WDC6QC5IHM4AICAYMJRQBM3LGVBYROMYZSMZBQ, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.207881440 (12h50m54.951938305s) [noise=true , address=104.36.82.12:4179 , piece_id=K7X5PR5NYK7LR3BDCQS66YAGYGWVFDXSG4TM42WIHR4XURDJWJ6Q, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.223413715 (12h50m54.967470563s) [noise=true , address=104.36.82.34:4549 , piece_id=7XWTCOONFR2KIRHVGUD3SRVR37PCP52ENQJR743AUEKVQBFWPLZQ, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.235435210 (12h50m54.979492061s) [noise=true , address=104.36.82.8:4982 , piece_id=Z5TUKKS3PLZFFUOGHJENT7JIB4FJBS5WWRBEYTQ2USSGVGN56OUA, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.309334498 (12h50m55.053391362s) [noise=true , address=104.36.82.17:4346 , piece_id=VDEOT6B6CUYIMFY3WMMTJVNJTXURII72Q7NBY2DAVHUNN4EHWC4A, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.502128838 (12h50m55.246185692s) [noise=true , address=64.49.13.26:4946 , piece_id=KB7JRCJHFMVHCJKG4NZQ5QJ3O5IFOWW2IBBWSP6TQ6DSTWBABT5A, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.544691546 (12h50m55.288748396s) [noise=true , address=79.127.205.245:4220 , piece_id=LX7ZN5QBMAPUFCIIOEOW4YXX72WGXH3A5ICGT5A6LVUCYLBLOCDA, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.633287759 (12h50m55.377344611s) [noise=true , address=104.36.82.60:4346 , piece_id=NIUOHVNXXWV2K7GZJFVKBR2ZIGTJBIYLC45G6ADMBDLJKOKIVEVQ, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.633308567 (12h50m55.377365418s) [seg_pos={0 0}, upload=13204]: Segment reached optimal threshold of 14 pieces. Aug 7 22:30:33.633410618 (12h50m55.377467467s) [noise=true , address=104.36.82.6:4884 , piece_id=J66M7GUJFLMQXPUT23IIWFRFHYY3J726DRUS4PNI4GMCL5ORNHOQ, seg_pos={0 0}, upload=13204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1c4THqUrQMeqY94EGiZSNKQDDnvycEZbGMrYxrb4s4XuaRe8br): protocol: expected piece hash; context canceled; EOF Aug 7 22:30:33.633759470 (12h50m55.377816321s) [noise=true , address=104.36.82.47:4527 , piece_id=RS7VNDZTLKOR4OSKZ3N5W3ELXLDN3BRXJNWHZEFBYW2LNM3NLZVA, seg_pos={0 0}, upload=13204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17xmkbFoj3PTJnAygsjRAcfdd17AfSkPP2xdZp68GGPrvfMuU6): protocol: expected piece hash; context canceled; EOF Aug 7 22:30:33.633848731 (12h50m55.377905589s) [noise=true , address=104.36.82.58:4288 , piece_id=XTZGZNZEJBM5RYKYJXJYY5ED2MFA2ADMTJB7FMILXDR3W7E7YOPQ, seg_pos={0 0}, upload=13204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuahEyfok8Hk4fBNcdrZQWoQZiCXfNWdbfqPMnbEPmdZQ3R1o): protocol: expected piece hash; context canceled; EOF Aug 7 22:30:33.634152395 (12h50m55.378209251s) [noise=true , address=104.36.82.37:4673 , piece_id=AVJWSEWIKELDXA3DV62DMV3RZLHJ4EJR4W3V6IGGXFUQHAMWFPJA, seg_pos={0 0}, upload=13204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NkBjmh5iL2BtckR19qgXQ4wnfvnbpA9erSuMCTEijwX7ZdHGL): protocol: write tcp 139.59.144.22:44486->104.36.82.37:4673: use of closed network connection; write tcp 139.59.144.22:44486->104.36.82.37:4673: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44486->104.36.82.37:4673: use of closed network connection Aug 7 22:30:33.634174413 (12h50m55.378231287s) [seg_pos={0 0}, upload=13204]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 22:30:33.634256322 (12h50m55.378313188s) [upload=13204]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 22:30:33.634262633 (12h50m55.378319509s) [upload=13204]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:30:34.036796742 (12h50m55.7808536s) [upload=13204]: Done uploading segments... Aug 7 22:30:34.036819686 (12h50m55.780876545s) [upload=13204]: Upload finished. err: <nil> Duration: 3.057719017s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 17 out of 60 is failed (threshold=3) [............................XXXXXXXXXXXX.XXXX....X..........] duration: 932.053325ms FAIL (2025-08-07 22:34:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== [upload=13077]: Starting upload Aug 7 22:22:03.530157956 (12h42m25.27421483s) [upload=13077]: Done starting upload Aug 7 22:22:03.530254340 (12h42m25.274311199s) [upload=13077]: Uploading segments... Aug 7 22:22:03.530319521 (12h42m25.274376398s) [upload=13077]: Got next segment. Inline: true Aug 7 22:22:03.530326839 (12h42m25.27438371s) [upload=13077]: Waiting for error group managing segments... Aug 7 22:22:03.530387033 (12h42m25.274443903s) [upload=13077]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:22:03.530393764 (12h42m25.274450635s) [upload=13077]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:22:03.530398068 (12h42m25.274454934s) [upload=13077]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:22:03.668101987 (12h42m25.412158856s) [upload=13077]: Done uploading segments... Aug 7 22:22:03.668146181 (12h42m25.412203031s) [upload=13077]: Upload finished. err: <nil> Aug 7 22:22:50.781175997 (12h43m12.525232844s) [upload=13085]: Starting upload Aug 7 22:22:50.781238492 (12h43m12.525295338s) [upload=13085]: Done starting upload Aug 7 22:22:50.781295832 (12h43m12.525352678s) [upload=13085]: Uploading segments... Aug 7 22:22:50.781339974 (12h43m12.525396844s) [upload=13085]: Got next segment. Inline: true Aug 7 22:22:50.781344619 (12h43m12.525401479s) [upload=13085]: Waiting for error group managing segments... Aug 7 22:22:50.781382529 (12h43m12.525439375s) [upload=13085]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:22:50.781387267 (12h43m12.525444113s) [upload=13085]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:22:50.781389841 (12h43m12.525446704s) [upload=13085]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:23:20.513298723 (12h43m42.257355587s) [upload=13085]: Done uploading segments... Aug 7 22:23:20.513325654 (12h43m42.25738252s) [upload=13085]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:23:51.254516692 (12h44m12.998573555s) [upload=13099]: Starting upload Aug 7 22:23:51.254606307 (12h44m12.998663169s) [upload=13099]: Done starting upload Aug 7 22:23:51.254669521 (12h44m12.998726371s) [upload=13099]: Uploading segments... Aug 7 22:23:51.254709863 (12h44m12.998766733s) [upload=13099]: Got next segment. Inline: true Aug 7 22:23:51.254715030 (12h44m12.99877189s) [upload=13099]: Waiting for error group managing segments... Aug 7 22:23:51.254768832 (12h44m12.998825693s) [upload=13099]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:23:51.254774395 (12h44m12.998831255s) [upload=13099]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:23:51.254778260 (12h44m12.998835121s) [upload=13099]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:24:20.980586635 (12h44m42.724643486s) [upload=13099]: Done uploading segments... Aug 7 22:24:20.980611487 (12h44m42.724668337s) [upload=13099]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:24:22.809871679 (12h44m44.55392853s) [upload=13108]: Starting upload Aug 7 22:24:22.809959685 (12h44m44.554016545s) [upload=13108]: Done starting upload Aug 7 22:24:22.810026016 (12h44m44.55408288s) [upload=13108]: Uploading segments... Aug 7 22:24:22.810084984 (12h44m44.554141856s) [upload=13108]: Got next segment. Inline: true Aug 7 22:24:22.810090937 (12h44m44.554147807s) [upload=13108]: Waiting for error group managing segments... Aug 7 22:24:22.810180251 (12h44m44.554237121s) [upload=13108]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:24:22.810187976 (12h44m44.554244835s) [upload=13108]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:24:22.810192661 (12h44m44.55424953s) [upload=13108]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:24:52.535275009 (12h45m14.279331864s) [upload=13108]: Done uploading segments... Aug 7 22:24:52.535303649 (12h45m14.279360515s) [upload=13108]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:24:52.960601883 (12h45m14.70465875s) [upload=13112]: Starting upload Aug 7 22:24:52.960708121 (12h45m14.704764984s) [upload=13112]: Done starting upload Aug 7 22:24:52.960845573 (12h45m14.704902419s) [upload=13112]: Uploading segments... Aug 7 22:24:52.960931513 (12h45m14.704988384s) [upload=13112]: Got next segment. Inline: true Aug 7 22:24:52.960939376 (12h45m14.704996243s) [upload=13112]: Waiting for error group managing segments... Aug 7 22:24:52.961019433 (12h45m14.705076313s) [upload=13112]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:24:52.961026755 (12h45m14.705083622s) [upload=13112]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:24:52.961030905 (12h45m14.705087769s) [upload=13112]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:25:22.687567107 (12h45m44.431623957s) [upload=13112]: Done uploading segments... Aug 7 22:25:22.687590061 (12h45m44.431646912s) [upload=13112]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:25:23.131013864 (12h45m44.875070712s) [upload=13122]: Starting upload Aug 7 22:25:23.131088619 (12h45m44.875145482s) [upload=13122]: Done starting upload Aug 7 22:25:23.131275088 (12h45m44.875331941s) [upload=13122]: Uploading segments... Aug 7 22:25:23.131376992 (12h45m44.875433843s) [upload=13122]: Got next segment. Inline: true Aug 7 22:25:23.131382965 (12h45m44.875439816s) [upload=13122]: Waiting for error group managing segments... Aug 7 22:25:23.131434882 (12h45m44.875491733s) [upload=13122]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:25:23.131440120 (12h45m44.875496982s) [upload=13122]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:25:23.131443668 (12h45m44.875500519s) [upload=13122]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:25:23.272670449 (12h45m45.016727299s) [upload=13122]: Done uploading segments... Aug 7 22:25:23.272702241 (12h45m45.016759091s) [upload=13122]: Upload finished. err: <nil> Aug 7 22:26:25.811198326 (12h46m47.555255193s) [upload=13140]: Starting upload Aug 7 22:26:25.811285415 (12h46m47.555342279s) [upload=13140]: Done starting upload Aug 7 22:26:25.811369732 (12h46m47.555426582s) [upload=13140]: Uploading segments... Aug 7 22:26:25.811467114 (12h46m47.55552398s) [upload=13140]: Got next segment. Inline: true Aug 7 22:26:25.811473701 (12h46m47.555530552s) [upload=13140]: Waiting for error group managing segments... Aug 7 22:26:25.811539126 (12h46m47.555595979s) [upload=13140]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:26:25.811545282 (12h46m47.555602146s) [upload=13140]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:26:25.811548939 (12h46m47.555605802s) [upload=13140]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:26:25.968679249 (12h46m47.712736113s) [upload=13140]: Done uploading segments... Aug 7 22:26:25.968709074 (12h46m47.712765928s) [upload=13140]: Upload finished. err: <nil> Aug 7 22:27:27.799853559 (12h47m49.543910409s) [upload=13154]: Starting upload Aug 7 22:27:27.799998197 (12h47m49.544055055s) [upload=13154]: Done starting upload Aug 7 22:27:27.800097878 (12h47m49.544154732s) [upload=13154]: Uploading segments... Aug 7 22:27:27.800202613 (12h47m49.544259463s) [upload=13154]: Got next segment. Inline: true Aug 7 22:27:27.800221436 (12h47m49.544278297s) [upload=13154]: Waiting for error group managing segments... Aug 7 22:27:27.800305006 (12h47m49.544361867s) [upload=13154]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:27:27.800310790 (12h47m49.54436765s) [upload=13154]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:27:27.800314678 (12h47m49.544371539s) [upload=13154]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:27:27.950105257 (12h47m49.694162117s) [upload=13154]: Done uploading segments... Aug 7 22:27:27.950124372 (12h47m49.694181233s) [upload=13154]: Upload finished. err: <nil> Aug 7 22:27:59.550898410 (12h48m21.29495526s) [upload=13165]: Starting upload Aug 7 22:27:59.550970829 (12h48m21.295027691s) [upload=13165]: Done starting upload Aug 7 22:27:59.551049380 (12h48m21.295106232s) [upload=13165]: Uploading segments... Aug 7 22:27:59.551283044 (12h48m21.295339906s) [upload=13165]: Got next segment. Inline: true Aug 7 22:27:59.551293209 (12h48m21.295350069s) [upload=13165]: Waiting for error group managing segments... Aug 7 22:27:59.551359587 (12h48m21.295416438s) [upload=13165]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:27:59.551365951 (12h48m21.295422812s) [upload=13165]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:27:59.551369453 (12h48m21.295426314s) [upload=13165]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:27:59.691525895 (12h48m21.435582778s) [upload=13165]: Done uploading segments... Aug 7 22:27:59.691551086 (12h48m21.435607933s) [upload=13165]: Upload finished. err: <nil> Aug 7 22:29:06.342665932 (12h49m28.086722791s) [upload=13183]: Starting upload Aug 7 22:29:06.342756156 (12h49m28.086813016s) [upload=13183]: Done starting upload Aug 7 22:29:06.342875471 (12h49m28.086932321s) [upload=13183]: Uploading segments... Aug 7 22:29:06.343020027 (12h49m28.087076894s) [upload=13183]: Got next segment. Inline: true Aug 7 22:29:06.343029056 (12h49m28.087085926s) [upload=13183]: Waiting for error group managing segments... Aug 7 22:29:06.343276816 (12h49m28.087333721s) [upload=13183]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:29:06.343287054 (12h49m28.087343919s) [upload=13183]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:29:06.343290895 (12h49m28.08734776s) [upload=13183]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:29:36.070785151 (12h49m57.814842003s) [upload=13183]: Done uploading segments... Aug 7 22:29:36.070807148 (12h49m57.814863999s) [upload=13183]: Upload finished. err: metaclient: context deadline exceeded Duration: 932.086191ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 19 out of 60 is failed (threshold=3) [.............................XXXXXXXXXXXXXXXX.XXX...........] duration: 1.053451191s FAIL (2025-08-07 22:34:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.053482378s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 16 out of 60 is failed (threshold=3) [................................X.XXXXXXXXXXXXXXX...........] duration: 951.282077ms FAIL (2025-08-07 22:34:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 4 (12h38m8.493437894s) [upload=13019]: Uploading segments... Aug 7 22:17:46.749423663 (12h38m8.493480532s) [upload=13019]: Got next segment. Inline: true Aug 7 22:17:46.749428712 (12h38m8.493485573s) [upload=13019]: Waiting for error group managing segments... Aug 7 22:17:46.749471140 (12h38m8.49352799s) [upload=13019]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:17:46.749476259 (12h38m8.493533119s) [upload=13019]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:17:46.749479952 (12h38m8.493536802s) [upload=13019]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:18:46.471549080 (12h39m8.215605941s) [upload=13019]: Done uploading segments... Aug 7 22:18:46.471571547 (12h39m8.215628407s) [upload=13019]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:18:46.904761993 (12h39m8.648818857s) [upload=13033]: Starting upload Aug 7 22:18:46.904856076 (12h39m8.648912937s) [upload=13033]: Done starting upload Aug 7 22:18:46.904960146 (12h39m8.649017s) [upload=13033]: Uploading segments... Aug 7 22:18:46.905047245 (12h39m8.649104114s) [upload=13033]: Got next segment. Inline: true Aug 7 22:18:46.905056198 (12h39m8.649113053s) [upload=13033]: Waiting for error group managing segments... Aug 7 22:18:46.905116687 (12h39m8.649173538s) [upload=13033]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:18:46.905122460 (12h39m8.649179309s) [upload=13033]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:18:46.905126367 (12h39m8.649183281s) [upload=13033]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:19:46.626010888 (12h40m8.370067739s) [upload=13033]: Done uploading segments... Aug 7 22:19:46.626032620 (12h40m8.370089471s) [upload=13033]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:19:59.643579230 (12h40m21.387636081s) [upload=13050]: Starting upload Aug 7 22:19:59.643668904 (12h40m21.387725771s) [upload=13050]: Done starting upload Aug 7 22:19:59.643785033 (12h40m21.38784188s) [upload=13050]: Uploading segments... Aug 7 22:19:59.643889787 (12h40m21.387946639s) [upload=13050]: Got next segment. Inline: true Aug 7 22:19:59.643897379 (12h40m21.387954229s) [upload=13050]: Waiting for error group managing segments... Aug 7 22:19:59.643957450 (12h40m21.388014313s) [upload=13050]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:19:59.643962932 (12h40m21.388019793s) [upload=13050]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:19:59.643966291 (12h40m21.388023151s) [upload=13050]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:20:59.369875577 (12h41m21.113932442s) [upload=13050]: Done uploading segments... Aug 7 22:20:59.369905671 (12h41m21.113962518s) [upload=13050]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:20:59.787948456 (12h41m21.532005324s) [upload=13064]: Starting upload Aug 7 22:20:59.788068842 (12h41m21.53212571s) [upload=13064]: Done starting upload Aug 7 22:20:59.788168587 (12h41m21.532225457s) [upload=13064]: Uploading segments... Aug 7 22:20:59.788311286 (12h41m21.532368151s) [upload=13064]: Got next segment. Inline: true Aug 7 22:20:59.788318759 (12h41m21.532375633s) [upload=13064]: Waiting for error group managing segments... Aug 7 22:20:59.788388410 (12h41m21.532445276s) [upload=13064]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:20:59.788394823 (12h41m21.532451689s) [upload=13064]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:20:59.788398689 (12h41m21.532455554s) [upload=13064]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:21:59.519361969 (12h42m21.263418825s) [upload=13064]: Done uploading segments... Aug 7 22:21:59.519387607 (12h42m21.263444468s) [upload=13064]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:21:59.945997169 (12h42m21.69005402s) [upload=13075]: Starting upload Aug 7 22:21:59.946071844 (12h42m21.690129359s) [upload=13075]: Done starting upload Aug 7 22:21:59.946178905 (12h42m21.690235754s) [upload=13075]: Uploading segments... Aug 7 22:21:59.946230067 (12h42m21.690286936s) [upload=13075]: Got next segment. Inline: true Aug 7 22:21:59.946235138 (12h42m21.690292002s) [upload=13075]: Waiting for error group managing segments... Aug 7 22:21:59.946295164 (12h42m21.690352024s) [upload=13075]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:21:59.946300535 (12h42m21.690357396s) [upload=13075]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:21:59.946303920 (12h42m21.69036078s) [upload=13075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:22:59.677780729 (12h43m21.421837589s) [upload=13075]: Done uploading segments... Aug 7 22:22:59.677798007 (12h43m21.421854868s) [upload=13075]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:23:00.094834123 (12h43m21.838890994s) [upload=13088]: Starting upload Aug 7 22:23:00.094984258 (12h43m21.83904116s) [upload=13088]: Done starting upload Aug 7 22:23:00.095111897 (12h43m21.839168758s) [upload=13088]: Uploading segments... Aug 7 22:23:00.095477827 (12h43m21.839534697s) [upload=13088]: Got next segment. Inline: true Aug 7 22:23:00.095487112 (12h43m21.839543982s) [upload=13088]: Waiting for error group managing segments... Aug 7 22:23:00.095559737 (12h43m21.839616605s) [upload=13088]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:23:00.095567952 (12h43m21.839624822s) [upload=13088]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:23:00.095571963 (12h43m21.839628824s) [upload=13088]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:23:59.816074411 (12h44m21.560131259s) [upload=13088]: Done uploading segments... Aug 7 22:23:59.816094923 (12h44m21.560162185s) [upload=13088]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:24:00.243449247 (12h44m21.987506098s) [upload=13101]: Starting upload Aug 7 22:24:00.243529153 (12h44m21.987586015s) [upload=13101]: Done starting upload Aug 7 22:24:00.243603428 (12h44m21.987660281s) [upload=13101]: Uploading segments... Aug 7 22:24:00.243664741 (12h44m21.98772161s) [upload=13101]: Got next segment. Inline: true Aug 7 22:24:00.243670702 (12h44m21.987727562s) [upload=13101]: Waiting for error group managing segments... Aug 7 22:24:00.243730265 (12h44m21.987787124s) [upload=13101]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:24:00.243736084 (12h44m21.987792946s) [upload=13101]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:24:00.243739669 (12h44m21.987796531s) [upload=13101]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:24:59.965587620 (12h45m21.709644469s) [upload=13101]: Done uploading segments... Aug 7 22:24:59.965609247 (12h45m21.709666095s) [upload=13101]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:25:00.417995504 (12h45m22.162052369s) [upload=13115]: Starting upload Aug 7 22:25:00.418148792 (12h45m22.162205658s) [upload=13115]: Done starting upload Aug 7 22:25:00.418313994 (12h45m22.162370855s) [upload=13115]: Uploading segments... Aug 7 22:25:00.418427015 (12h45m22.162483886s) [upload=13115]: Got next segment. Inline: true Aug 7 22:25:00.418453990 (12h45m22.162510859s) [upload=13115]: Waiting for error group managing segments... Aug 7 22:25:00.418542094 (12h45m22.16259896s) [upload=13115]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:25:00.418549113 (12h45m22.162605982s) [upload=13115]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:25:00.418555718 (12h45m22.162612588s) [upload=13115]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:26:00.143481421 (12h46m21.887538319s) [upload=13115]: Done uploading segments... Aug 7 22:26:00.143508788 (12h46m21.88756568s) [upload=13115]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:26:00.572183011 (12h46m22.316239858s) [upload=13133]: Starting upload Aug 7 22:26:00.572257919 (12h46m22.316314778s) [upload=13133]: Done starting upload Aug 7 22:26:00.572347104 (12h46m22.316403949s) [upload=13133]: Uploading segments... Aug 7 22:26:00.572414569 (12h46m22.316471429s) [upload=13133]: Got next segment. Inline: true Aug 7 22:26:00.572419354 (12h46m22.316476214s) [upload=13133]: Waiting for error group managing segments... Aug 7 22:26:00.572457012 (12h46m22.316513858s) [upload=13133]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:26:00.572461785 (12h46m22.316518631s) [upload=13133]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:26:00.572465017 (12h46m22.316521878s) [upload=13133]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:27:00.300711265 (12h47m22.044768125s) [upload=13133]: Done uploading segments... Aug 7 22:27:00.300733422 (12h47m22.044790282s) [upload=13133]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:27:00.736249379 (12h47m22.48030624s) [upload=13147]: Starting upload Aug 7 22:27:00.736365836 (12h47m22.480422697s) [upload=13147]: Done starting upload Aug 7 22:27:00.736456413 (12h47m22.480513264s) [upload=13147]: Uploading segments... Aug 7 22:27:00.736517869 (12h47m22.480574739s) [upload=13147]: Got next segment. Inline: true Aug 7 22:27:00.736526521 (12h47m22.480583386s) [upload=13147]: Waiting for error group managing segments... Aug 7 22:27:00.736600220 (12h47m22.480657069s) [upload=13147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:27:00.736605984 (12h47m22.480662844s) [upload=13147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:27:00.736609354 (12h47m22.480666214s) [upload=13147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:28:00.459012227 (12h48m22.203069105s) [upload=13147]: Done uploading segments... Aug 7 22:28:00.459039460 (12h48m22.203096312s) [upload=13147]: Upload finished. err: metaclient: context deadline exceeded Duration: 951.313803ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 19 out of 60 is failed (threshold=3) [...X.X...........................XXXXXXXXXXXX.XXXX.X........] duration: 3.446454876s FAIL (2025-08-07 22:34:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.446478155s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.......X..X...........X......................X........X.....] duration: 3.212981972s FAIL (2025-08-07 22:34:34 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== pload=13204]: Uploading segments... Aug 7 22:30:31.937644158 (12h50m53.681701018s) [upload=13204]: Got next segment. Inline: false Aug 7 22:30:31.937701034 (12h50m53.681757885s) [upload=13204]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:30:31.937725980 (12h50m53.681782841s) [upload=13204]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:30:32.237884006 (12h50m53.981940856s) [seg_pos={0 0}, upload=13204]: Begin upload segment... Aug 7 22:30:32.237975798 (12h50m53.982032665s) [seg_pos={0 0}, upload=13204]: Done begin upload segment. Aug 7 22:30:32.238021241 (12h50m53.982078102s) [noise=true , address=104.36.82.34:4549 , piece_id=7XWTCOONFR2KIRHVGUD3SRVR37PCP52ENQJR743AUEKVQBFWPLZQ, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.238037974 (12h50m53.982094843s) [noise=true , address=104.36.82.37:4673 , piece_id=AVJWSEWIKELDXA3DV62DMV3RZLHJ4EJR4W3V6IGGXFUQHAMWFPJA, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.238498130 (12h50m53.982554995s) [noise=true , address=104.36.82.17:4346 , piece_id=VDEOT6B6CUYIMFY3WMMTJVNJTXURII72Q7NBY2DAVHUNN4EHWC4A, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.238509513 (12h50m53.982566376s) [noise=true , address=104.36.82.12:4179 , piece_id=K7X5PR5NYK7LR3BDCQS66YAGYGWVFDXSG4TM42WIHR4XURDJWJ6Q, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.238837373 (12h50m53.982894233s) [noise=true , address=104.36.82.47:4527 , piece_id=RS7VNDZTLKOR4OSKZ3N5W3ELXLDN3BRXJNWHZEFBYW2LNM3NLZVA, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.238877922 (12h50m53.982934787s) [noise=true , address=104.36.82.58:4288 , piece_id=XTZGZNZEJBM5RYKYJXJYY5ED2MFA2ADMTJB7FMILXDR3W7E7YOPQ, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.239360083 (12h50m53.983416953s) [noise=true , address=104.36.82.44:4873 , piece_id=G7J764SMJNHCLSGTTCSDYISA6UNAFENBKL44CXJOH2TC46C56EFQ, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.239441321 (12h50m53.983498189s) [noise=true , address=104.36.82.39:4759 , piece_id=POKLEQJD3YYQSEJFZWNGKXULGIJ5JCQHHLMDBC3TAFYGZEWIXY7Q, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.239783955 (12h50m53.983840827s) [noise=true , address=79.127.205.245:4220 , piece_id=LX7ZN5QBMAPUFCIIOEOW4YXX72WGXH3A5ICGT5A6LVUCYLBLOCDA, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.239893333 (12h50m53.983950196s) [noise=true , address=104.36.82.23:4479 , piece_id=FCBSRKC6XA7MA7EIJ3GCB7XPTPC43OWYRXTJFFFJXU4I6LNRQPGA, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.240152214 (12h50m53.984209078s) [noise=true , address=104.36.82.59:4320 , piece_id=PLFQLBPXXXYN4EIVX5Z56WO7YCHMYKH5EBPNQV3KAEC4Q7NNTJZA, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.240280854 (12h50m53.984337724s) [noise=true , address=77.90.31.17:4777 , piece_id=5WWWEOD7W4KTC7WDC6QC5IHM4AICAYMJRQBM3LGVBYROMYZSMZBQ, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.240446242 (12h50m53.984503111s) [noise=true , address=104.36.82.46:4496 , piece_id=EQE7MYB2HOMCJNKQE4XZE62KTDQUUJKILDCDONOI2YKAURSM4I4A, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.240677807 (12h50m53.984734657s) [noise=true , address=104.36.82.6:4884 , piece_id=J66M7GUJFLMQXPUT23IIWFRFHYY3J726DRUS4PNI4GMCL5ORNHOQ, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.240762860 (12h50m53.984819723s) [noise=true , address=64.49.13.26:4946 , piece_id=KB7JRCJHFMVHCJKG4NZQ5QJ3O5IFOWW2IBBWSP6TQ6DSTWBABT5A, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.241124053 (12h50m53.985180903s) [noise=true , address=104.36.82.60:4346 , piece_id=NIUOHVNXXWV2K7GZJFVKBR2ZIGTJBIYLC45G6ADMBDLJKOKIVEVQ, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.241500880 (12h50m53.985557731s) [noise=true , address=104.36.82.8:4982 , piece_id=Z5TUKKS3PLZFFUOGHJENT7JIB4FJBS5WWRBEYTQ2USSGVGN56OUA, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.241813854 (12h50m53.985870704s) [noise=true , address=104.36.82.19:4406 , piece_id=6Z6NKSRY63B3XDEPE2H3X5HPY6JYTYHLOVEQX5DY2X4AKURLTGZA, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.832548156 (12h50m54.576605008s) [upload=13204]: Next returned nil segment Aug 7 22:30:32.832566279 (12h50m54.576623131s) [upload=13204]: Waiting for error group managing segments... Aug 7 22:30:32.986982167 (12h50m54.731039021s) [noise=true , address=104.36.82.44:4873 , piece_id=G7J764SMJNHCLSGTTCSDYISA6UNAFENBKL44CXJOH2TC46C56EFQ, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:32.995526266 (12h50m54.739583133s) [noise=true , address=104.36.82.39:4759 , piece_id=POKLEQJD3YYQSEJFZWNGKXULGIJ5JCQHHLMDBC3TAFYGZEWIXY7Q, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.029305286 (12h50m54.773362132s) [noise=true , address=104.36.82.46:4496 , piece_id=EQE7MYB2HOMCJNKQE4XZE62KTDQUUJKILDCDONOI2YKAURSM4I4A, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.030396152 (12h50m54.774453021s) [noise=true , address=104.36.82.23:4479 , piece_id=FCBSRKC6XA7MA7EIJ3GCB7XPTPC43OWYRXTJFFFJXU4I6LNRQPGA, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.058142476 (12h50m54.802199375s) [noise=true , address=104.36.82.19:4406 , piece_id=6Z6NKSRY63B3XDEPE2H3X5HPY6JYTYHLOVEQX5DY2X4AKURLTGZA, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.175489481 (12h50m54.919546344s) [noise=true , address=104.36.82.59:4320 , piece_id=PLFQLBPXXXYN4EIVX5Z56WO7YCHMYKH5EBPNQV3KAEC4Q7NNTJZA, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.206541092 (12h50m54.950597958s) [noise=true , address=77.90.31.17:4777 , piece_id=5WWWEOD7W4KTC7WDC6QC5IHM4AICAYMJRQBM3LGVBYROMYZSMZBQ, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.207881440 (12h50m54.951938305s) [noise=true , address=104.36.82.12:4179 , piece_id=K7X5PR5NYK7LR3BDCQS66YAGYGWVFDXSG4TM42WIHR4XURDJWJ6Q, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.223413715 (12h50m54.967470563s) [noise=true , address=104.36.82.34:4549 , piece_id=7XWTCOONFR2KIRHVGUD3SRVR37PCP52ENQJR743AUEKVQBFWPLZQ, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.235435210 (12h50m54.979492061s) [noise=true , address=104.36.82.8:4982 , piece_id=Z5TUKKS3PLZFFUOGHJENT7JIB4FJBS5WWRBEYTQ2USSGVGN56OUA, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.309334498 (12h50m55.053391362s) [noise=true , address=104.36.82.17:4346 , piece_id=VDEOT6B6CUYIMFY3WMMTJVNJTXURII72Q7NBY2DAVHUNN4EHWC4A, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.502128838 (12h50m55.246185692s) [noise=true , address=64.49.13.26:4946 , piece_id=KB7JRCJHFMVHCJKG4NZQ5QJ3O5IFOWW2IBBWSP6TQ6DSTWBABT5A, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.544691546 (12h50m55.288748396s) [noise=true , address=79.127.205.245:4220 , piece_id=LX7ZN5QBMAPUFCIIOEOW4YXX72WGXH3A5ICGT5A6LVUCYLBLOCDA, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.633287759 (12h50m55.377344611s) [noise=true , address=104.36.82.60:4346 , piece_id=NIUOHVNXXWV2K7GZJFVKBR2ZIGTJBIYLC45G6ADMBDLJKOKIVEVQ, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.633308567 (12h50m55.377365418s) [seg_pos={0 0}, upload=13204]: Segment reached optimal threshold of 14 pieces. Aug 7 22:30:33.633410618 (12h50m55.377467467s) [noise=true , address=104.36.82.6:4884 , piece_id=J66M7GUJFLMQXPUT23IIWFRFHYY3J726DRUS4PNI4GMCL5ORNHOQ, seg_pos={0 0}, upload=13204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1c4THqUrQMeqY94EGiZSNKQDDnvycEZbGMrYxrb4s4XuaRe8br): protocol: expected piece hash; context canceled; EOF Aug 7 22:30:33.633759470 (12h50m55.377816321s) [noise=true , address=104.36.82.47:4527 , piece_id=RS7VNDZTLKOR4OSKZ3N5W3ELXLDN3BRXJNWHZEFBYW2LNM3NLZVA, seg_pos={0 0}, upload=13204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17xmkbFoj3PTJnAygsjRAcfdd17AfSkPP2xdZp68GGPrvfMuU6): protocol: expected piece hash; context canceled; EOF Aug 7 22:30:33.633848731 (12h50m55.377905589s) [noise=true , address=104.36.82.58:4288 , piece_id=XTZGZNZEJBM5RYKYJXJYY5ED2MFA2ADMTJB7FMILXDR3W7E7YOPQ, seg_pos={0 0}, upload=13204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuahEyfok8Hk4fBNcdrZQWoQZiCXfNWdbfqPMnbEPmdZQ3R1o): protocol: expected piece hash; context canceled; EOF Aug 7 22:30:33.634152395 (12h50m55.378209251s) [noise=true , address=104.36.82.37:4673 , piece_id=AVJWSEWIKELDXA3DV62DMV3RZLHJ4EJR4W3V6IGGXFUQHAMWFPJA, seg_pos={0 0}, upload=13204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NkBjmh5iL2BtckR19qgXQ4wnfvnbpA9erSuMCTEijwX7ZdHGL): protocol: write tcp 139.59.144.22:44486->104.36.82.37:4673: use of closed network connection; write tcp 139.59.144.22:44486->104.36.82.37:4673: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44486->104.36.82.37:4673: use of closed network connection Aug 7 22:30:33.634174413 (12h50m55.378231287s) [seg_pos={0 0}, upload=13204]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 22:30:33.634256322 (12h50m55.378313188s) [upload=13204]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 22:30:33.634262633 (12h50m55.378319509s) [upload=13204]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:30:34.036796742 (12h50m55.7808536s) [upload=13204]: Done uploading segments... Aug 7 22:30:34.036819686 (12h50m55.780876545s) [upload=13204]: Upload finished. err: <nil> Duration: 3.213010533s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 21 out of 60 is failed (threshold=3) [............................X.XXXXXXXXXXXXXXXX.X.X.XX.......] duration: 926.523834ms FAIL (2025-08-07 22:34:24 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 926.545493ms core uplink us1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.....................................X......................] duration: 111.3085ms remote: "tcp:185.244.226.2:443" target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt FAIL (2025-08-07 22:34:23 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 111.41257ms edge linksharing bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 12 out of 60 is failed (threshold=3) [.............XXXX..............................XXXXXXXX.....] duration: 3.567479819s FAIL (2025-08-07 22:34:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== s... Aug 7 22:21:14.636082751 (12h41m36.380139602s) [upload=13066]: Got next segment. Inline: false Aug 7 22:21:14.636136774 (12h41m36.380193634s) [upload=13066]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:21:14.636142452 (12h41m36.380199312s) [upload=13066]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:21:16.211357551 (12h41m37.955414411s) [seg_pos={0 0}, upload=13066]: Begin upload segment... Aug 7 22:21:16.211426917 (12h41m37.955483784s) [noise=true , address=45.91.116.5:4327 , piece_id=IGBMWNLDFIKQY7AJ3UJ6JILAFLRQ5DZAE6FFO3EJ4NDPVWZXCK5A, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.214054689 (12h41m37.95811156s) [seg_pos={0 0}, upload=13066]: Done begin upload segment. Aug 7 22:21:16.214143921 (12h41m37.958200791s) [noise=true , address=104.36.82.2:4754 , piece_id=PQJBMG6FQ3HUTE26IBB5R3WQCAJTFKMGECOXSJPV2VDHP33JOPUA, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.214660710 (12h41m37.95871758s) [noise=true , address=104.36.82.59:4336 , piece_id=5AVF5EEMC334PPHVOSYDAYZ5CKQ4F5LE6BRD5G446NRRJNYQBOYQ, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.215094607 (12h41m37.959151469s) [noise=true , address=104.36.82.44:4904 , piece_id=RWD3KVEEHT5BUVYBAHRHZFCBTCZUTNHRD6PU7OWOUIF2QVBZXSSQ, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.215554381 (12h41m37.959611251s) [noise=true , address=104.36.82.58:4270 , piece_id=LUZ6HPESCVZDYNDVHR2QIS6HM2IIRK3IQB6BSV2VVVB3OFRZVMFQ, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.215879389 (12h41m37.959936254s) [noise=true , address=104.36.82.60:4339 , piece_id=FI3A54HMEOKZJ2FC2ZOLNQXW6NR7AUD26HBVFFGEGTG3AYRZD73A, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.216161781 (12h41m37.960218643s) [noise=true , address=104.36.82.61:4397 , piece_id=7FJT5CLASEMVAP4HSGHP3XQTNBKVRP2STW2YF6H2UDLHFPUHKMMQ, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.216489433 (12h41m37.960546284s) [noise=true , address=104.36.82.17:4344 , piece_id=CCKNSMXAJAO3QUJDDZ7POWA7NZ63KACQQAJRU37RPG5YMXVMOVZA, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.216758002 (12h41m37.960814865s) [noise=true , address=104.36.82.47:4525 , piece_id=62GQ6IOY4OBZAPRBVZ2OLLA7R4DVQVMWNQZAR67BTPQUX2HUEZRA, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.217037043 (12h41m37.961093908s) [noise=true , address=104.36.82.13:4191 , piece_id=NUNWAQKF465J5JQLYQBOCH75SAXPR5F3OPBDDTC6AWWBSKB6MSUA, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.217313215 (12h41m37.961370077s) [noise=true , address=64.49.13.140:4509 , piece_id=CNKDQOVU53HJ6EDOSZOXRSFOKQBNKWERZSFJQA5F24FZBR6XEEQA, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.217587290 (12h41m37.961644153s) [noise=true , address=45.91.116.9:4292 , piece_id=NO3S4IIMN5JWUNMMNQS6SDFLCDPW3MF3LK4BDZ4IVIZMRWBHSTQQ, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.217852877 (12h41m37.961909742s) [noise=true , address=104.36.82.43:4849 , piece_id=ECUDD3LRMPQMXZW6MRFLFHVPAO3RHCV6MWZYOJSASL4ILT54TFFA, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.218138868 (12h41m37.962195731s) [noise=true , address=64.49.13.26:4956 , piece_id=VHWLGKKB5MBWQOD3KVNXE3YYCK5TZ56UU25BTACIPJQZJA7PHA7Q, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.218358245 (12h41m37.962415106s) [noise=true , address=104.36.82.62:4435 , piece_id=O3PP5IRA3MLOLM35Z2U2BLM7D6GPH6WMHWANUF7AAGTW7IW6CZBQ, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.218693596 (12h41m37.962750459s) [noise=true , address=79.127.205.233:4647 , piece_id=6YIT5HDQJ6CN63ZFANACICFS4M2OQEPOSFIV36QXS5VLJZ2BRR7A, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.218971752 (12h41m37.963028615s) [noise=true , address=104.36.82.60:4354 , piece_id=A3YTMCWULXHPBRVL6TDQOBNYTQQWNKB2UZPKAPDH2CCG5SMV6QTA, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.219339785 (12h41m37.963396657s) [noise=true , address=104.36.82.9:4036 , piece_id=UG2DKHGMS5NC3BTN7QQU23JXBUQ7UFVN6FZKPBZHYUOQ6SEJV5WA, seg_pos={0 0}, upload=13066]: Uploading piece... Aug 7 22:21:16.820211995 (12h41m38.564268846s) [upload=13066]: Next returned nil segment Aug 7 22:21:16.820729442 (12h41m38.564786312s) [upload=13066]: Waiting for error group managing segments... Aug 7 22:21:16.974534776 (12h41m38.71859164s) [noise=true , address=104.36.82.61:4397 , piece_id=7FJT5CLASEMVAP4HSGHP3XQTNBKVRP2STW2YF6H2UDLHFPUHKMMQ, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:16.995935209 (12h41m38.739992059s) [noise=true , address=104.36.82.9:4036 , piece_id=UG2DKHGMS5NC3BTN7QQU23JXBUQ7UFVN6FZKPBZHYUOQ6SEJV5WA, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.005410191 (12h41m38.749467042s) [noise=true , address=104.36.82.60:4354 , piece_id=A3YTMCWULXHPBRVL6TDQOBNYTQQWNKB2UZPKAPDH2CCG5SMV6QTA, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.012303228 (12h41m38.756360079s) [noise=true , address=104.36.82.17:4344 , piece_id=CCKNSMXAJAO3QUJDDZ7POWA7NZ63KACQQAJRU37RPG5YMXVMOVZA, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.047899041 (12h41m38.791955903s) [noise=true , address=104.36.82.13:4191 , piece_id=NUNWAQKF465J5JQLYQBOCH75SAXPR5F3OPBDDTC6AWWBSKB6MSUA, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.065632282 (12h41m38.809689142s) [noise=true , address=104.36.82.60:4339 , piece_id=FI3A54HMEOKZJ2FC2ZOLNQXW6NR7AUD26HBVFFGEGTG3AYRZD73A, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.074330593 (12h41m38.818387457s) [noise=true , address=104.36.82.62:4435 , piece_id=O3PP5IRA3MLOLM35Z2U2BLM7D6GPH6WMHWANUF7AAGTW7IW6CZBQ, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.205659931 (12h41m38.949716794s) [noise=true , address=104.36.82.59:4336 , piece_id=5AVF5EEMC334PPHVOSYDAYZ5CKQ4F5LE6BRD5G446NRRJNYQBOYQ, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.304344891 (12h41m39.048401752s) [noise=true , address=45.91.116.5:4327 , piece_id=IGBMWNLDFIKQY7AJ3UJ6JILAFLRQ5DZAE6FFO3EJ4NDPVWZXCK5A, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.476320837 (12h41m39.220377687s) [noise=true , address=64.49.13.140:4509 , piece_id=CNKDQOVU53HJ6EDOSZOXRSFOKQBNKWERZSFJQA5F24FZBR6XEEQA, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.498349653 (12h41m39.242406517s) [noise=true , address=64.49.13.26:4956 , piece_id=VHWLGKKB5MBWQOD3KVNXE3YYCK5TZ56UU25BTACIPJQZJA7PHA7Q, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.566667384 (12h41m39.310724234s) [noise=true , address=104.36.82.47:4525 , piece_id=62GQ6IOY4OBZAPRBVZ2OLLA7R4DVQVMWNQZAR67BTPQUX2HUEZRA, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:17.572654381 (12h41m39.316711251s) [noise=true , address=79.127.205.233:4647 , piece_id=6YIT5HDQJ6CN63ZFANACICFS4M2OQEPOSFIV36QXS5VLJZ2BRR7A, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:18.002510167 (12h41m39.746567013s) [noise=true , address=45.91.116.9:4292 , piece_id=NO3S4IIMN5JWUNMMNQS6SDFLCDPW3MF3LK4BDZ4IVIZMRWBHSTQQ, seg_pos={0 0}, upload=13066]: Done uploading piece. err: <nil> Aug 7 22:21:18.002534446 (12h41m39.746591307s) [seg_pos={0 0}, upload=13066]: Segment reached optimal threshold of 14 pieces. Aug 7 22:21:18.002684920 (12h41m39.746741766s) [noise=true , address=104.36.82.43:4849 , piece_id=ECUDD3LRMPQMXZW6MRFLFHVPAO3RHCV6MWZYOJSASL4ILT54TFFA, seg_pos={0 0}, upload=13066]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12goRcYW39Qqga4HW1i6cg6VGKJFUxnyTJBe6HRnZiLAU3LonWn): protocol: expected piece hash; context canceled; EOF Aug 7 22:21:18.002753282 (12h41m39.746810128s) [noise=true , address=104.36.82.58:4270 , piece_id=LUZ6HPESCVZDYNDVHR2QIS6HM2IIRK3IQB6BSV2VVVB3OFRZVMFQ, seg_pos={0 0}, upload=13066]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sJfMr1nMTpizd5PZ2bZ6M8tZWhFL2ukdJ3cNSRQtdUHNJhg81): protocol: expected piece hash; context canceled; EOF Aug 7 22:21:18.002945708 (12h41m39.747002555s) [noise=true , address=104.36.82.2:4754 , piece_id=PQJBMG6FQ3HUTE26IBB5R3WQCAJTFKMGECOXSJPV2VDHP33JOPUA, seg_pos={0 0}, upload=13066]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125aPwoHeFabk6zZ6bkGNj3USJYgTQG3RB1QhhXssj7Vik2AuCs): protocol: expected piece hash; context canceled; EOF Aug 7 22:21:18.003059994 (12h41m39.74711684s) [noise=true , address=104.36.82.44:4904 , piece_id=RWD3KVEEHT5BUVYBAHRHZFCBTCZUTNHRD6PU7OWOUIF2QVBZXSSQ, seg_pos={0 0}, upload=13066]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: write tcp 139.59.144.22:47438->104.36.82.44:4904: use of closed network connection; write tcp 139.59.144.22:47438->104.36.82.44:4904: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:47438->104.36.82.44:4904: use of closed network connection Aug 7 22:21:18.003080731 (12h41m39.747137592s) [seg_pos={0 0}, upload=13066]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 22:21:18.003298604 (12h41m39.747355464s) [upload=13066]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 22:21:18.003304952 (12h41m39.747361812s) [upload=13066]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:30:15.962677772 (12h50m37.706734624s) [upload=13066]: Done uploading segments... Aug 7 22:30:15.962709891 (12h50m37.706766753s) [upload=13066]: Upload finished. err: metaclient: context deadline exceeded Duration: 3.567717279s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [.......X...X..........................X...............X.....] duration: 1.201693757s FAIL (2025-08-07 22:34:18 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== d=13120]: Done uploading segments... Aug 7 22:25:13.463635961 (12h45m35.207692812s) [upload=13120]: Upload finished. err: <nil> Aug 7 22:25:36.676221958 (12h45m58.42027881s) [upload=13126]: Starting upload Aug 7 22:25:36.676280024 (12h45m58.420336886s) [upload=13126]: Done starting upload Aug 7 22:25:36.676348826 (12h45m58.420405681s) [upload=13126]: Uploading segments... Aug 7 22:25:36.676415761 (12h45m58.42047263s) [upload=13126]: Got next segment. Inline: true Aug 7 22:25:36.676421620 (12h45m58.420478481s) [upload=13126]: Waiting for error group managing segments... Aug 7 22:25:36.676463453 (12h45m58.420520304s) [upload=13126]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:25:36.676469083 (12h45m58.420525944s) [upload=13126]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:25:36.676472693 (12h45m58.420529553s) [upload=13126]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:25:36.846956085 (12h45m58.591012935s) [upload=13126]: Done uploading segments... Aug 7 22:25:36.846976507 (12h45m58.591033359s) [upload=13126]: Upload finished. err: <nil> Aug 7 22:26:18.836000763 (12h46m40.580057613s) [upload=13137]: Starting upload Aug 7 22:26:18.836038438 (12h46m40.580095288s) [upload=13137]: Done starting upload Aug 7 22:26:18.836098090 (12h46m40.58015494s) [upload=13137]: Uploading segments... Aug 7 22:26:18.836143460 (12h46m40.580200331s) [upload=13137]: Got next segment. Inline: true Aug 7 22:26:18.836148461 (12h46m40.580205309s) [upload=13137]: Waiting for error group managing segments... Aug 7 22:26:18.836191879 (12h46m40.580248725s) [upload=13137]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:26:18.836197487 (12h46m40.580254348s) [upload=13137]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:26:18.836205083 (12h46m40.580261941s) [upload=13137]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:26:18.978690589 (12h46m40.722747448s) [upload=13137]: Done uploading segments... Aug 7 22:26:18.978706599 (12h46m40.722763445s) [upload=13137]: Upload finished. err: <nil> Aug 7 22:27:06.613274662 (12h47m28.357331513s) [upload=13150]: Starting upload Aug 7 22:27:06.613358594 (12h47m28.357415443s) [upload=13150]: Done starting upload Aug 7 22:27:06.613451706 (12h47m28.357508565s) [upload=13150]: Uploading segments... Aug 7 22:27:06.613532713 (12h47m28.357589583s) [upload=13150]: Got next segment. Inline: true Aug 7 22:27:06.613540195 (12h47m28.357597165s) [upload=13150]: Waiting for error group managing segments... Aug 7 22:27:06.613608323 (12h47m28.357665185s) [upload=13150]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:27:06.613616998 (12h47m28.35767387s) [upload=13150]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:27:06.613621123 (12h47m28.357677991s) [upload=13150]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:27:06.756270932 (12h47m28.500327782s) [upload=13150]: Done uploading segments... Aug 7 22:27:06.756289996 (12h47m28.500346845s) [upload=13150]: Upload finished. err: <nil> Aug 7 22:27:39.275964882 (12h48m1.020021742s) [upload=13159]: Starting upload Aug 7 22:27:39.276031028 (12h48m1.020087874s) [upload=13159]: Done starting upload Aug 7 22:27:39.276100015 (12h48m1.020156863s) [upload=13159]: Uploading segments... Aug 7 22:27:39.276162849 (12h48m1.020219721s) [upload=13159]: Got next segment. Inline: true Aug 7 22:27:39.276168125 (12h48m1.020224985s) [upload=13159]: Waiting for error group managing segments... Aug 7 22:27:39.276216009 (12h48m1.020272869s) [upload=13159]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:27:39.276220617 (12h48m1.020277478s) [upload=13159]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:27:39.276223278 (12h48m1.020280137s) [upload=13159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:27:39.491372003 (12h48m1.235428853s) [upload=13159]: Done uploading segments... Aug 7 22:27:39.491394004 (12h48m1.235450857s) [upload=13159]: Upload finished. err: <nil> Aug 7 22:28:03.413359947 (12h48m25.157416798s) [upload=13167]: Starting upload Aug 7 22:28:03.413424697 (12h48m25.157481559s) [upload=13167]: Done starting upload Aug 7 22:28:03.413546352 (12h48m25.157603207s) [upload=13167]: Uploading segments... Aug 7 22:28:03.413695999 (12h48m25.157752868s) [upload=13167]: Got next segment. Inline: true Aug 7 22:28:03.413705607 (12h48m25.157762476s) [upload=13167]: Waiting for error group managing segments... Aug 7 22:28:03.413788849 (12h48m25.157845716s) [upload=13167]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:28:03.413816554 (12h48m25.157873423s) [upload=13167]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:28:03.413820681 (12h48m25.157877544s) [upload=13167]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:28:03.561945313 (12h48m25.306002181s) [upload=13167]: Done uploading segments... Aug 7 22:28:03.561970962 (12h48m25.306027825s) [upload=13167]: Upload finished. err: <nil> Aug 7 22:28:25.535904284 (12h48m47.279961133s) [upload=13172]: Starting upload Aug 7 22:28:25.535961986 (12h48m47.280018836s) [upload=13172]: Done starting upload Aug 7 22:28:25.536032757 (12h48m47.280089607s) [upload=13172]: Uploading segments... Aug 7 22:28:25.536074131 (12h48m47.280131001s) [upload=13172]: Got next segment. Inline: true Aug 7 22:28:25.536089717 (12h48m47.280146578s) [upload=13172]: Waiting for error group managing segments... Aug 7 22:28:25.536134532 (12h48m47.280191382s) [upload=13172]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:28:25.536151821 (12h48m47.280208671s) [upload=13172]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:28:25.536166195 (12h48m47.280223058s) [upload=13172]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:28:26.345427619 (12h48m48.089484469s) [upload=13172]: Done uploading segments... Aug 7 22:28:26.345447034 (12h48m48.089503895s) [upload=13172]: Upload finished. err: <nil> Aug 7 22:28:49.552061521 (12h49m11.296118389s) [upload=13179]: Starting upload Aug 7 22:28:49.552134904 (12h49m11.296191767s) [upload=13179]: Done starting upload Aug 7 22:28:49.552208829 (12h49m11.296265676s) [upload=13179]: Uploading segments... Aug 7 22:28:49.552262194 (12h49m11.296319062s) [upload=13179]: Got next segment. Inline: true Aug 7 22:28:49.552268137 (12h49m11.296324998s) [upload=13179]: Waiting for error group managing segments... Aug 7 22:28:49.552316820 (12h49m11.296373667s) [upload=13179]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:28:49.552323169 (12h49m11.29638003s) [upload=13179]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:28:49.552327059 (12h49m11.29638392s) [upload=13179]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:28:49.693951093 (12h49m11.438007952s) [upload=13179]: Done uploading segments... Aug 7 22:28:49.693988901 (12h49m11.438045762s) [upload=13179]: Upload finished. err: <nil> Aug 7 22:29:09.820328663 (12h49m31.564385513s) [upload=13184]: Starting upload Aug 7 22:29:09.820384335 (12h49m31.564441196s) [upload=13184]: Done starting upload Aug 7 22:29:09.820445932 (12h49m31.564502786s) [upload=13184]: Uploading segments... Aug 7 22:29:09.820487651 (12h49m31.564544517s) [upload=13184]: Got next segment. Inline: true Aug 7 22:29:09.820492611 (12h49m31.564549472s) [upload=13184]: Waiting for error group managing segments... Aug 7 22:29:09.820534051 (12h49m31.564590901s) [upload=13184]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:29:09.820539062 (12h49m31.564595922s) [upload=13184]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:29:09.820542725 (12h49m31.564599585s) [upload=13184]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:29:10.028668214 (12h49m31.772725062s) [upload=13184]: Done uploading segments... Aug 7 22:29:10.028692776 (12h49m31.772749623s) [upload=13184]: Upload finished. err: <nil> Aug 7 22:30:21.625872621 (12h50m43.369929471s) [upload=13201]: Starting upload Aug 7 22:30:21.625976134 (12h50m43.370033002s) [upload=13201]: Done starting upload Aug 7 22:30:21.626054127 (12h50m43.370110973s) [upload=13201]: Uploading segments... Aug 7 22:30:21.626095581 (12h50m43.370152447s) [upload=13201]: Got next segment. Inline: true Aug 7 22:30:21.626101049 (12h50m43.370157915s) [upload=13201]: Waiting for error group managing segments... Aug 7 22:30:21.626141797 (12h50m43.370198644s) [upload=13201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:30:21.626147151 (12h50m43.370204015s) [upload=13201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:30:21.626149721 (12h50m43.370206581s) [upload=13201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:30:21.760255105 (12h50m43.504311956s) [upload=13201]: Done uploading segments... Aug 7 22:30:21.760273763 (12h50m43.504330614s) [upload=13201]: Upload finished. err: <nil> Aug 7 22:30:47.060687315 (12h51m8.804744175s) [upload=13211]: Starting upload Aug 7 22:30:47.060743560 (12h51m8.80480042s) [upload=13211]: Done starting upload Aug 7 22:30:47.060819066 (12h51m8.804875915s) [upload=13211]: Uploading segments... Aug 7 22:30:47.060872995 (12h51m8.804929861s) [upload=13211]: Got next segment. Inline: true Aug 7 22:30:47.060878710 (12h51m8.80493557s) [upload=13211]: Waiting for error group managing segments... Aug 7 22:30:47.060932110 (12h51m8.804988961s) [upload=13211]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:30:47.060937671 (12h51m8.80499453s) [upload=13211]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:30:47.060940905 (12h51m8.804997769s) [upload=13211]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:30:48.162928472 (12h51m9.906985333s) [upload=13211]: Done uploading segments... Aug 7 22:30:48.162948603 (12h51m9.907005457s) [upload=13211]: Upload finished. err: <nil> Duration: 1.201723033s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 2 out of 60 is failed (threshold=3) [....................................X............X..........] duration: 113.123525ms remote: "tcp:185.244.226.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 22:34:17 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 113.202985ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 21 out of 60 is failed (threshold=3) [XXXXXXXXXXXXX............................X.XXXXX.XX.........] duration: 2.417386736s FAIL (2025-08-07 22:34:16 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.417418597s core uplink us1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 3 out of 60 is failed (threshold=3) [............................................XXX.............] duration: 326.749501ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 22:34:14 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 326.778963ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [.X...................................XXXXXXXXXXXXX..X.......] duration: 2.966060591s FAIL (2025-08-07 22:34:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== 7 22:27:44.139472223 (12h48m5.883529083s) [upload=13160]: Got next segment. Inline: false Aug 7 22:27:44.139508151 (12h48m5.883565011s) [upload=13160]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:27:44.139513399 (12h48m5.883570259s) [upload=13160]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:27:44.245907064 (12h48m5.989963927s) [seg_pos={0 0}, upload=13160]: Begin upload segment... Aug 7 22:27:44.246039735 (12h48m5.990096597s) [noise=true , address=104.36.82.10:4079 , piece_id=2ADIUNAB5FPKLXMW46HBB75SJLYOKA4N3TYS6J72WZDKLQTSP23A, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.246596012 (12h48m5.990652882s) [noise=true , address=104.36.82.4:4801 , piece_id=U6XCDALXGOTJHRL2P4V7DOPIJJF6VUKB5RIJYOP3WXFTC6HQKQXQ, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.247006251 (12h48m5.991063115s) [noise=true , address=77.90.31.12:4590 , piece_id=IYZTGAPBVXXQNQVDNR4NZRPDNNIBETBQ26DZUZ2SODA2D2VQUXTA, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.247197178 (12h48m5.991254041s) [seg_pos={0 0}, upload=13160]: Done begin upload segment. Aug 7 22:27:44.247225152 (12h48m5.991282018s) [noise=true , address=104.36.82.5:4837 , piece_id=XDA3KTWSH5OJPHK4IFOPINMJAR6VJLL7B7WNO3MV2VT3HQCGJ2KA, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.247751444 (12h48m5.99180831s) [noise=true , address=104.36.82.6:4876 , piece_id=45DZFNAUCX6EJNPT4WBVLEIO2GZROYIF2NQ254LU32T6AE7P4ZRA, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.247956514 (12h48m5.992013382s) [noise=true , address=104.36.82.11:4115 , piece_id=B2C72IS3FGY7DT4SXLUG5FYHCN5ECAMCAIVT2ZNVCXML2RVCQZFQ, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.248499391 (12h48m5.992556252s) [noise=true , address=45.91.116.12:4225 , piece_id=N7R42AWZKTHBBPZY24OVKSBRTPGVBLJ3HBYB2ONRZ56F6YZK2XWQ, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.248606386 (12h48m5.992663249s) [noise=true , address=77.90.31.13:4628 , piece_id=FSARBSWO4VFKOYHINODNYG7BC4JEH6KX5Z3IOYIJ4DGQ3DCZNDJA, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.249110151 (12h48m5.99316702s) [noise=true , address=45.91.116.9:4261 , piece_id=ZESRB2XAQ4WPIB2HQKYLZKJDXDE65NXMRX4M5JE3EKRHROCT4JKA, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.249195913 (12h48m5.993252781s) [noise=true , address=104.36.82.13:4185 , piece_id=QI54Y53UTBR7JEQ4NPD5R37EC47364LEUWETTCGI3WQUKSHUTP5A, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.249587895 (12h48m5.993644759s) [noise=true , address=45.91.116.8:4047 , piece_id=LDURVPEWVV7OEPVY4L4ZER4H2EDUPJS45N55Y57VZUTLSTKGXSRA, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.249756005 (12h48m5.993812872s) [noise=true , address=77.90.31.11:4553 , piece_id=ZDJ5BW2XPNHEBNB6FBVWAH66T6SVKAUY3724YY6HZ25VBLKYFTIQ, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.250801847 (12h48m5.99485871s) [noise=true , address=77.90.31.10:4517 , piece_id=QHV63D3QBJ5GVDWNHGWJTLXQNHGK4PZ4OCT4UWCEQKHMYBDL26WA, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.251094602 (12h48m5.995151463s) [noise=true , address=77.90.31.18:4806 , piece_id=ZJLVNSXETRBXYNHIDLN7TSQU6OQVWTCOQFJLPWAMCPY2JHBTHVQA, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.251408354 (12h48m5.995465217s) [noise=true , address=104.36.82.10:4078 , piece_id=UCDTQMJTYXAMJXGRPKBGTJZ4OCF4P2BSCR6WR3KSXCK6S5PWWZ7A, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.251693979 (12h48m5.995750839s) [noise=true , address=45.91.116.5:4297 , piece_id=QEQZU67YDQXNW7ACIRVXJKX2HIKMZMAP23TCAMTB2RDK3RFKLLRQ, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.928589281 (12h48m6.672646155s) [upload=13160]: Next returned nil segment Aug 7 22:27:44.928610126 (12h48m6.672666995s) [upload=13160]: Waiting for error group managing segments... Aug 7 22:27:45.055730069 (12h48m6.799786923s) [noise=true , address=104.36.82.11:4115 , piece_id=B2C72IS3FGY7DT4SXLUG5FYHCN5ECAMCAIVT2ZNVCXML2RVCQZFQ, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.058230571 (12h48m6.802287421s) [noise=true , address=104.36.82.4:4801 , piece_id=U6XCDALXGOTJHRL2P4V7DOPIJJF6VUKB5RIJYOP3WXFTC6HQKQXQ, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.072176193 (12h48m6.816233043s) [noise=true , address=104.36.82.6:4876 , piece_id=45DZFNAUCX6EJNPT4WBVLEIO2GZROYIF2NQ254LU32T6AE7P4ZRA, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.076144433 (12h48m6.82020128s) [noise=true , address=104.36.82.5:4837 , piece_id=XDA3KTWSH5OJPHK4IFOPINMJAR6VJLL7B7WNO3MV2VT3HQCGJ2KA, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.083969611 (12h48m6.828026457s) [noise=true , address=104.36.82.13:4185 , piece_id=QI54Y53UTBR7JEQ4NPD5R37EC47364LEUWETTCGI3WQUKSHUTP5A, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.090499988 (12h48m6.834556849s) [noise=true , address=104.36.82.10:4078 , piece_id=UCDTQMJTYXAMJXGRPKBGTJZ4OCF4P2BSCR6WR3KSXCK6S5PWWZ7A, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.129675895 (12h48m6.873732757s) [noise=true , address=45.91.116.12:4225 , piece_id=N7R42AWZKTHBBPZY24OVKSBRTPGVBLJ3HBYB2ONRZ56F6YZK2XWQ, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.148504731 (12h48m6.892561581s) [noise=true , address=77.90.31.10:4517 , piece_id=QHV63D3QBJ5GVDWNHGWJTLXQNHGK4PZ4OCT4UWCEQKHMYBDL26WA, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.183435281 (12h48m6.927492128s) [noise=true , address=104.36.82.10:4079 , piece_id=2ADIUNAB5FPKLXMW46HBB75SJLYOKA4N3TYS6J72WZDKLQTSP23A, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.188056276 (12h48m6.932113137s) [noise=true , address=77.90.31.12:4590 , piece_id=IYZTGAPBVXXQNQVDNR4NZRPDNNIBETBQ26DZUZ2SODA2D2VQUXTA, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.189109860 (12h48m6.93316672s) [noise=true , address=77.90.31.13:4628 , piece_id=FSARBSWO4VFKOYHINODNYG7BC4JEH6KX5Z3IOYIJ4DGQ3DCZNDJA, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.191395584 (12h48m6.93545243s) [noise=true , address=45.91.116.9:4261 , piece_id=ZESRB2XAQ4WPIB2HQKYLZKJDXDE65NXMRX4M5JE3EKRHROCT4JKA, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.212167167 (12h48m6.956224014s) [noise=true , address=77.90.31.11:4553 , piece_id=ZDJ5BW2XPNHEBNB6FBVWAH66T6SVKAUY3724YY6HZ25VBLKYFTIQ, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.212191124 (12h48m6.956247983s) [seg_pos={0 0}, upload=13160]: Segment reached optimal threshold of 13 pieces. Aug 7 22:27:45.212404510 (12h48m6.956461359s) [noise=true , address=45.91.116.8:4047 , piece_id=LDURVPEWVV7OEPVY4L4ZER4H2EDUPJS45N55Y57VZUTLSTKGXSRA, seg_pos={0 0}, upload=13160]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: context canceled; context canceled Aug 7 22:27:45.212444310 (12h48m6.956501156s) [noise=true , address=77.90.31.18:4806 , piece_id=ZJLVNSXETRBXYNHIDLN7TSQU6OQVWTCOQFJLPWAMCPY2JHBTHVQA, seg_pos={0 0}, upload=13160]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: write tcp 139.59.144.22:44926->77.90.31.18:4806: use of closed network connection; write tcp 139.59.144.22:44926->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44926->77.90.31.18:4806: use of closed network connection Aug 7 22:27:45.212507508 (12h48m6.956564355s) [noise=true , address=45.91.116.5:4297 , piece_id=QEQZU67YDQXNW7ACIRVXJKX2HIKMZMAP23TCAMTB2RDK3RFKLLRQ, seg_pos={0 0}, upload=13160]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 139.59.144.22:41424->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:41424->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41424->45.91.116.5:4297: use of closed network connection Aug 7 22:27:45.212525428 (12h48m6.956582288s) [seg_pos={0 0}, upload=13160]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 22:27:45.212587349 (12h48m6.956644209s) [upload=13160]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 22:27:45.212591472 (12h48m6.956648332s) [upload=13160]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:27:45.467451329 (12h48m7.21150818s) [upload=13160]: Done uploading segments... Aug 7 22:27:45.467470231 (12h48m7.211527081s) [upload=13160]: Upload finished. err: <nil> Aug 7 22:28:46.300540067 (12h49m8.044596928s) [upload=13178]: Starting upload Aug 7 22:28:46.300629117 (12h49m8.044685979s) [upload=13178]: Done starting upload Aug 7 22:28:46.300797752 (12h49m8.044854603s) [upload=13178]: Uploading segments... Aug 7 22:28:46.301190738 (12h49m8.045247598s) [upload=13178]: Got next segment. Inline: false Aug 7 22:28:46.301268000 (12h49m8.04532486s) [upload=13178]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:28:46.301275318 (12h49m8.045332178s) [upload=13178]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:29:16.009299446 (12h49m37.753356315s) [upload=13178]: Next segment err: context deadline exceeded Aug 7 22:29:16.009371909 (12h49m37.753428759s) [upload=13178]: Done uploading segments... Aug 7 22:29:16.009392178 (12h49m37.753449035s) [upload=13178]: Upload finished. err: context deadline exceeded Duration: 2.966102594s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [........X..X...........X......................X........X....] duration: 3.330613816s FAIL (2025-08-07 22:34:07 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== pload=13204]: Uploading segments... Aug 7 22:30:31.937644158 (12h50m53.681701018s) [upload=13204]: Got next segment. Inline: false Aug 7 22:30:31.937701034 (12h50m53.681757885s) [upload=13204]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:30:31.937725980 (12h50m53.681782841s) [upload=13204]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:30:32.237884006 (12h50m53.981940856s) [seg_pos={0 0}, upload=13204]: Begin upload segment... Aug 7 22:30:32.237975798 (12h50m53.982032665s) [seg_pos={0 0}, upload=13204]: Done begin upload segment. Aug 7 22:30:32.238021241 (12h50m53.982078102s) [noise=true , address=104.36.82.34:4549 , piece_id=7XWTCOONFR2KIRHVGUD3SRVR37PCP52ENQJR743AUEKVQBFWPLZQ, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.238037974 (12h50m53.982094843s) [noise=true , address=104.36.82.37:4673 , piece_id=AVJWSEWIKELDXA3DV62DMV3RZLHJ4EJR4W3V6IGGXFUQHAMWFPJA, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.238498130 (12h50m53.982554995s) [noise=true , address=104.36.82.17:4346 , piece_id=VDEOT6B6CUYIMFY3WMMTJVNJTXURII72Q7NBY2DAVHUNN4EHWC4A, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.238509513 (12h50m53.982566376s) [noise=true , address=104.36.82.12:4179 , piece_id=K7X5PR5NYK7LR3BDCQS66YAGYGWVFDXSG4TM42WIHR4XURDJWJ6Q, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.238837373 (12h50m53.982894233s) [noise=true , address=104.36.82.47:4527 , piece_id=RS7VNDZTLKOR4OSKZ3N5W3ELXLDN3BRXJNWHZEFBYW2LNM3NLZVA, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.238877922 (12h50m53.982934787s) [noise=true , address=104.36.82.58:4288 , piece_id=XTZGZNZEJBM5RYKYJXJYY5ED2MFA2ADMTJB7FMILXDR3W7E7YOPQ, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.239360083 (12h50m53.983416953s) [noise=true , address=104.36.82.44:4873 , piece_id=G7J764SMJNHCLSGTTCSDYISA6UNAFENBKL44CXJOH2TC46C56EFQ, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.239441321 (12h50m53.983498189s) [noise=true , address=104.36.82.39:4759 , piece_id=POKLEQJD3YYQSEJFZWNGKXULGIJ5JCQHHLMDBC3TAFYGZEWIXY7Q, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.239783955 (12h50m53.983840827s) [noise=true , address=79.127.205.245:4220 , piece_id=LX7ZN5QBMAPUFCIIOEOW4YXX72WGXH3A5ICGT5A6LVUCYLBLOCDA, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.239893333 (12h50m53.983950196s) [noise=true , address=104.36.82.23:4479 , piece_id=FCBSRKC6XA7MA7EIJ3GCB7XPTPC43OWYRXTJFFFJXU4I6LNRQPGA, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.240152214 (12h50m53.984209078s) [noise=true , address=104.36.82.59:4320 , piece_id=PLFQLBPXXXYN4EIVX5Z56WO7YCHMYKH5EBPNQV3KAEC4Q7NNTJZA, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.240280854 (12h50m53.984337724s) [noise=true , address=77.90.31.17:4777 , piece_id=5WWWEOD7W4KTC7WDC6QC5IHM4AICAYMJRQBM3LGVBYROMYZSMZBQ, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.240446242 (12h50m53.984503111s) [noise=true , address=104.36.82.46:4496 , piece_id=EQE7MYB2HOMCJNKQE4XZE62KTDQUUJKILDCDONOI2YKAURSM4I4A, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.240677807 (12h50m53.984734657s) [noise=true , address=104.36.82.6:4884 , piece_id=J66M7GUJFLMQXPUT23IIWFRFHYY3J726DRUS4PNI4GMCL5ORNHOQ, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.240762860 (12h50m53.984819723s) [noise=true , address=64.49.13.26:4946 , piece_id=KB7JRCJHFMVHCJKG4NZQ5QJ3O5IFOWW2IBBWSP6TQ6DSTWBABT5A, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.241124053 (12h50m53.985180903s) [noise=true , address=104.36.82.60:4346 , piece_id=NIUOHVNXXWV2K7GZJFVKBR2ZIGTJBIYLC45G6ADMBDLJKOKIVEVQ, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.241500880 (12h50m53.985557731s) [noise=true , address=104.36.82.8:4982 , piece_id=Z5TUKKS3PLZFFUOGHJENT7JIB4FJBS5WWRBEYTQ2USSGVGN56OUA, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.241813854 (12h50m53.985870704s) [noise=true , address=104.36.82.19:4406 , piece_id=6Z6NKSRY63B3XDEPE2H3X5HPY6JYTYHLOVEQX5DY2X4AKURLTGZA, seg_pos={0 0}, upload=13204]: Uploading piece... Aug 7 22:30:32.832548156 (12h50m54.576605008s) [upload=13204]: Next returned nil segment Aug 7 22:30:32.832566279 (12h50m54.576623131s) [upload=13204]: Waiting for error group managing segments... Aug 7 22:30:32.986982167 (12h50m54.731039021s) [noise=true , address=104.36.82.44:4873 , piece_id=G7J764SMJNHCLSGTTCSDYISA6UNAFENBKL44CXJOH2TC46C56EFQ, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:32.995526266 (12h50m54.739583133s) [noise=true , address=104.36.82.39:4759 , piece_id=POKLEQJD3YYQSEJFZWNGKXULGIJ5JCQHHLMDBC3TAFYGZEWIXY7Q, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.029305286 (12h50m54.773362132s) [noise=true , address=104.36.82.46:4496 , piece_id=EQE7MYB2HOMCJNKQE4XZE62KTDQUUJKILDCDONOI2YKAURSM4I4A, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.030396152 (12h50m54.774453021s) [noise=true , address=104.36.82.23:4479 , piece_id=FCBSRKC6XA7MA7EIJ3GCB7XPTPC43OWYRXTJFFFJXU4I6LNRQPGA, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.058142476 (12h50m54.802199375s) [noise=true , address=104.36.82.19:4406 , piece_id=6Z6NKSRY63B3XDEPE2H3X5HPY6JYTYHLOVEQX5DY2X4AKURLTGZA, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.175489481 (12h50m54.919546344s) [noise=true , address=104.36.82.59:4320 , piece_id=PLFQLBPXXXYN4EIVX5Z56WO7YCHMYKH5EBPNQV3KAEC4Q7NNTJZA, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.206541092 (12h50m54.950597958s) [noise=true , address=77.90.31.17:4777 , piece_id=5WWWEOD7W4KTC7WDC6QC5IHM4AICAYMJRQBM3LGVBYROMYZSMZBQ, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.207881440 (12h50m54.951938305s) [noise=true , address=104.36.82.12:4179 , piece_id=K7X5PR5NYK7LR3BDCQS66YAGYGWVFDXSG4TM42WIHR4XURDJWJ6Q, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.223413715 (12h50m54.967470563s) [noise=true , address=104.36.82.34:4549 , piece_id=7XWTCOONFR2KIRHVGUD3SRVR37PCP52ENQJR743AUEKVQBFWPLZQ, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.235435210 (12h50m54.979492061s) [noise=true , address=104.36.82.8:4982 , piece_id=Z5TUKKS3PLZFFUOGHJENT7JIB4FJBS5WWRBEYTQ2USSGVGN56OUA, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.309334498 (12h50m55.053391362s) [noise=true , address=104.36.82.17:4346 , piece_id=VDEOT6B6CUYIMFY3WMMTJVNJTXURII72Q7NBY2DAVHUNN4EHWC4A, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.502128838 (12h50m55.246185692s) [noise=true , address=64.49.13.26:4946 , piece_id=KB7JRCJHFMVHCJKG4NZQ5QJ3O5IFOWW2IBBWSP6TQ6DSTWBABT5A, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.544691546 (12h50m55.288748396s) [noise=true , address=79.127.205.245:4220 , piece_id=LX7ZN5QBMAPUFCIIOEOW4YXX72WGXH3A5ICGT5A6LVUCYLBLOCDA, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.633287759 (12h50m55.377344611s) [noise=true , address=104.36.82.60:4346 , piece_id=NIUOHVNXXWV2K7GZJFVKBR2ZIGTJBIYLC45G6ADMBDLJKOKIVEVQ, seg_pos={0 0}, upload=13204]: Done uploading piece. err: <nil> Aug 7 22:30:33.633308567 (12h50m55.377365418s) [seg_pos={0 0}, upload=13204]: Segment reached optimal threshold of 14 pieces. Aug 7 22:30:33.633410618 (12h50m55.377467467s) [noise=true , address=104.36.82.6:4884 , piece_id=J66M7GUJFLMQXPUT23IIWFRFHYY3J726DRUS4PNI4GMCL5ORNHOQ, seg_pos={0 0}, upload=13204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1c4THqUrQMeqY94EGiZSNKQDDnvycEZbGMrYxrb4s4XuaRe8br): protocol: expected piece hash; context canceled; EOF Aug 7 22:30:33.633759470 (12h50m55.377816321s) [noise=true , address=104.36.82.47:4527 , piece_id=RS7VNDZTLKOR4OSKZ3N5W3ELXLDN3BRXJNWHZEFBYW2LNM3NLZVA, seg_pos={0 0}, upload=13204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17xmkbFoj3PTJnAygsjRAcfdd17AfSkPP2xdZp68GGPrvfMuU6): protocol: expected piece hash; context canceled; EOF Aug 7 22:30:33.633848731 (12h50m55.377905589s) [noise=true , address=104.36.82.58:4288 , piece_id=XTZGZNZEJBM5RYKYJXJYY5ED2MFA2ADMTJB7FMILXDR3W7E7YOPQ, seg_pos={0 0}, upload=13204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuahEyfok8Hk4fBNcdrZQWoQZiCXfNWdbfqPMnbEPmdZQ3R1o): protocol: expected piece hash; context canceled; EOF Aug 7 22:30:33.634152395 (12h50m55.378209251s) [noise=true , address=104.36.82.37:4673 , piece_id=AVJWSEWIKELDXA3DV62DMV3RZLHJ4EJR4W3V6IGGXFUQHAMWFPJA, seg_pos={0 0}, upload=13204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NkBjmh5iL2BtckR19qgXQ4wnfvnbpA9erSuMCTEijwX7ZdHGL): protocol: write tcp 139.59.144.22:44486->104.36.82.37:4673: use of closed network connection; write tcp 139.59.144.22:44486->104.36.82.37:4673: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44486->104.36.82.37:4673: use of closed network connection Aug 7 22:30:33.634174413 (12h50m55.378231287s) [seg_pos={0 0}, upload=13204]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 22:30:33.634256322 (12h50m55.378313188s) [upload=13204]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 22:30:33.634262633 (12h50m55.378319509s) [upload=13204]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:30:34.036796742 (12h50m55.7808536s) [upload=13204]: Done uploading segments... Aug 7 22:30:34.036819686 (12h50m55.780876545s) [upload=13204]: Upload finished. err: <nil> Duration: 3.330666244s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 19 out of 60 is failed (threshold=3) [....X.X...........................XXXXXXXXXXXX.XXXX.X.......] duration: 2.909606343s FAIL (2025-08-07 22:34:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.910411525s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 17 out of 60 is failed (threshold=3) [X................................X.XXXXXXXXXXXXXXX..........] duration: 867.873378ms FAIL (2025-08-07 22:34:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 4 (12h38m8.493437894s) [upload=13019]: Uploading segments... Aug 7 22:17:46.749423663 (12h38m8.493480532s) [upload=13019]: Got next segment. Inline: true Aug 7 22:17:46.749428712 (12h38m8.493485573s) [upload=13019]: Waiting for error group managing segments... Aug 7 22:17:46.749471140 (12h38m8.49352799s) [upload=13019]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:17:46.749476259 (12h38m8.493533119s) [upload=13019]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:17:46.749479952 (12h38m8.493536802s) [upload=13019]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:18:46.471549080 (12h39m8.215605941s) [upload=13019]: Done uploading segments... Aug 7 22:18:46.471571547 (12h39m8.215628407s) [upload=13019]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:18:46.904761993 (12h39m8.648818857s) [upload=13033]: Starting upload Aug 7 22:18:46.904856076 (12h39m8.648912937s) [upload=13033]: Done starting upload Aug 7 22:18:46.904960146 (12h39m8.649017s) [upload=13033]: Uploading segments... Aug 7 22:18:46.905047245 (12h39m8.649104114s) [upload=13033]: Got next segment. Inline: true Aug 7 22:18:46.905056198 (12h39m8.649113053s) [upload=13033]: Waiting for error group managing segments... Aug 7 22:18:46.905116687 (12h39m8.649173538s) [upload=13033]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:18:46.905122460 (12h39m8.649179309s) [upload=13033]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:18:46.905126367 (12h39m8.649183281s) [upload=13033]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:19:46.626010888 (12h40m8.370067739s) [upload=13033]: Done uploading segments... Aug 7 22:19:46.626032620 (12h40m8.370089471s) [upload=13033]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:19:59.643579230 (12h40m21.387636081s) [upload=13050]: Starting upload Aug 7 22:19:59.643668904 (12h40m21.387725771s) [upload=13050]: Done starting upload Aug 7 22:19:59.643785033 (12h40m21.38784188s) [upload=13050]: Uploading segments... Aug 7 22:19:59.643889787 (12h40m21.387946639s) [upload=13050]: Got next segment. Inline: true Aug 7 22:19:59.643897379 (12h40m21.387954229s) [upload=13050]: Waiting for error group managing segments... Aug 7 22:19:59.643957450 (12h40m21.388014313s) [upload=13050]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:19:59.643962932 (12h40m21.388019793s) [upload=13050]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:19:59.643966291 (12h40m21.388023151s) [upload=13050]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:20:59.369875577 (12h41m21.113932442s) [upload=13050]: Done uploading segments... Aug 7 22:20:59.369905671 (12h41m21.113962518s) [upload=13050]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:20:59.787948456 (12h41m21.532005324s) [upload=13064]: Starting upload Aug 7 22:20:59.788068842 (12h41m21.53212571s) [upload=13064]: Done starting upload Aug 7 22:20:59.788168587 (12h41m21.532225457s) [upload=13064]: Uploading segments... Aug 7 22:20:59.788311286 (12h41m21.532368151s) [upload=13064]: Got next segment. Inline: true Aug 7 22:20:59.788318759 (12h41m21.532375633s) [upload=13064]: Waiting for error group managing segments... Aug 7 22:20:59.788388410 (12h41m21.532445276s) [upload=13064]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:20:59.788394823 (12h41m21.532451689s) [upload=13064]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:20:59.788398689 (12h41m21.532455554s) [upload=13064]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:21:59.519361969 (12h42m21.263418825s) [upload=13064]: Done uploading segments... Aug 7 22:21:59.519387607 (12h42m21.263444468s) [upload=13064]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:21:59.945997169 (12h42m21.69005402s) [upload=13075]: Starting upload Aug 7 22:21:59.946071844 (12h42m21.690129359s) [upload=13075]: Done starting upload Aug 7 22:21:59.946178905 (12h42m21.690235754s) [upload=13075]: Uploading segments... Aug 7 22:21:59.946230067 (12h42m21.690286936s) [upload=13075]: Got next segment. Inline: true Aug 7 22:21:59.946235138 (12h42m21.690292002s) [upload=13075]: Waiting for error group managing segments... Aug 7 22:21:59.946295164 (12h42m21.690352024s) [upload=13075]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:21:59.946300535 (12h42m21.690357396s) [upload=13075]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:21:59.946303920 (12h42m21.69036078s) [upload=13075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:22:59.677780729 (12h43m21.421837589s) [upload=13075]: Done uploading segments... Aug 7 22:22:59.677798007 (12h43m21.421854868s) [upload=13075]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:23:00.094834123 (12h43m21.838890994s) [upload=13088]: Starting upload Aug 7 22:23:00.094984258 (12h43m21.83904116s) [upload=13088]: Done starting upload Aug 7 22:23:00.095111897 (12h43m21.839168758s) [upload=13088]: Uploading segments... Aug 7 22:23:00.095477827 (12h43m21.839534697s) [upload=13088]: Got next segment. Inline: true Aug 7 22:23:00.095487112 (12h43m21.839543982s) [upload=13088]: Waiting for error group managing segments... Aug 7 22:23:00.095559737 (12h43m21.839616605s) [upload=13088]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:23:00.095567952 (12h43m21.839624822s) [upload=13088]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:23:00.095571963 (12h43m21.839628824s) [upload=13088]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:23:59.816074411 (12h44m21.560131259s) [upload=13088]: Done uploading segments... Aug 7 22:23:59.816094923 (12h44m21.560162185s) [upload=13088]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:24:00.243449247 (12h44m21.987506098s) [upload=13101]: Starting upload Aug 7 22:24:00.243529153 (12h44m21.987586015s) [upload=13101]: Done starting upload Aug 7 22:24:00.243603428 (12h44m21.987660281s) [upload=13101]: Uploading segments... Aug 7 22:24:00.243664741 (12h44m21.98772161s) [upload=13101]: Got next segment. Inline: true Aug 7 22:24:00.243670702 (12h44m21.987727562s) [upload=13101]: Waiting for error group managing segments... Aug 7 22:24:00.243730265 (12h44m21.987787124s) [upload=13101]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:24:00.243736084 (12h44m21.987792946s) [upload=13101]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:24:00.243739669 (12h44m21.987796531s) [upload=13101]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:24:59.965587620 (12h45m21.709644469s) [upload=13101]: Done uploading segments... Aug 7 22:24:59.965609247 (12h45m21.709666095s) [upload=13101]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:25:00.417995504 (12h45m22.162052369s) [upload=13115]: Starting upload Aug 7 22:25:00.418148792 (12h45m22.162205658s) [upload=13115]: Done starting upload Aug 7 22:25:00.418313994 (12h45m22.162370855s) [upload=13115]: Uploading segments... Aug 7 22:25:00.418427015 (12h45m22.162483886s) [upload=13115]: Got next segment. Inline: true Aug 7 22:25:00.418453990 (12h45m22.162510859s) [upload=13115]: Waiting for error group managing segments... Aug 7 22:25:00.418542094 (12h45m22.16259896s) [upload=13115]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:25:00.418549113 (12h45m22.162605982s) [upload=13115]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:25:00.418555718 (12h45m22.162612588s) [upload=13115]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:26:00.143481421 (12h46m21.887538319s) [upload=13115]: Done uploading segments... Aug 7 22:26:00.143508788 (12h46m21.88756568s) [upload=13115]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:26:00.572183011 (12h46m22.316239858s) [upload=13133]: Starting upload Aug 7 22:26:00.572257919 (12h46m22.316314778s) [upload=13133]: Done starting upload Aug 7 22:26:00.572347104 (12h46m22.316403949s) [upload=13133]: Uploading segments... Aug 7 22:26:00.572414569 (12h46m22.316471429s) [upload=13133]: Got next segment. Inline: true Aug 7 22:26:00.572419354 (12h46m22.316476214s) [upload=13133]: Waiting for error group managing segments... Aug 7 22:26:00.572457012 (12h46m22.316513858s) [upload=13133]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:26:00.572461785 (12h46m22.316518631s) [upload=13133]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:26:00.572465017 (12h46m22.316521878s) [upload=13133]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:27:00.300711265 (12h47m22.044768125s) [upload=13133]: Done uploading segments... Aug 7 22:27:00.300733422 (12h47m22.044790282s) [upload=13133]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:27:00.736249379 (12h47m22.48030624s) [upload=13147]: Starting upload Aug 7 22:27:00.736365836 (12h47m22.480422697s) [upload=13147]: Done starting upload Aug 7 22:27:00.736456413 (12h47m22.480513264s) [upload=13147]: Uploading segments... Aug 7 22:27:00.736517869 (12h47m22.480574739s) [upload=13147]: Got next segment. Inline: true Aug 7 22:27:00.736526521 (12h47m22.480583386s) [upload=13147]: Waiting for error group managing segments... Aug 7 22:27:00.736600220 (12h47m22.480657069s) [upload=13147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:27:00.736605984 (12h47m22.480662844s) [upload=13147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:27:00.736609354 (12h47m22.480666214s) [upload=13147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:28:00.459012227 (12h48m22.203069105s) [upload=13147]: Done uploading segments... Aug 7 22:28:00.459039460 (12h48m22.203096312s) [upload=13147]: Upload finished. err: metaclient: context deadline exceeded Duration: 867.967405ms core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [......................................X.....................] duration: 103.648093ms remote: "tcp:185.244.226.2:443" target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt FAIL (2025-08-07 22:33:57 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 103.764604ms edge linksharing bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 17 out of 60 is failed (threshold=3) [.............................XXXXXXXXXXXX.XXXX....X.........] duration: 866.849625ms FAIL (2025-08-07 22:33:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== [upload=13077]: Starting upload Aug 7 22:22:03.530157956 (12h42m25.27421483s) [upload=13077]: Done starting upload Aug 7 22:22:03.530254340 (12h42m25.274311199s) [upload=13077]: Uploading segments... Aug 7 22:22:03.530319521 (12h42m25.274376398s) [upload=13077]: Got next segment. Inline: true Aug 7 22:22:03.530326839 (12h42m25.27438371s) [upload=13077]: Waiting for error group managing segments... Aug 7 22:22:03.530387033 (12h42m25.274443903s) [upload=13077]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:22:03.530393764 (12h42m25.274450635s) [upload=13077]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:22:03.530398068 (12h42m25.274454934s) [upload=13077]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:22:03.668101987 (12h42m25.412158856s) [upload=13077]: Done uploading segments... Aug 7 22:22:03.668146181 (12h42m25.412203031s) [upload=13077]: Upload finished. err: <nil> Aug 7 22:22:50.781175997 (12h43m12.525232844s) [upload=13085]: Starting upload Aug 7 22:22:50.781238492 (12h43m12.525295338s) [upload=13085]: Done starting upload Aug 7 22:22:50.781295832 (12h43m12.525352678s) [upload=13085]: Uploading segments... Aug 7 22:22:50.781339974 (12h43m12.525396844s) [upload=13085]: Got next segment. Inline: true Aug 7 22:22:50.781344619 (12h43m12.525401479s) [upload=13085]: Waiting for error group managing segments... Aug 7 22:22:50.781382529 (12h43m12.525439375s) [upload=13085]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:22:50.781387267 (12h43m12.525444113s) [upload=13085]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:22:50.781389841 (12h43m12.525446704s) [upload=13085]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:23:20.513298723 (12h43m42.257355587s) [upload=13085]: Done uploading segments... Aug 7 22:23:20.513325654 (12h43m42.25738252s) [upload=13085]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:23:51.254516692 (12h44m12.998573555s) [upload=13099]: Starting upload Aug 7 22:23:51.254606307 (12h44m12.998663169s) [upload=13099]: Done starting upload Aug 7 22:23:51.254669521 (12h44m12.998726371s) [upload=13099]: Uploading segments... Aug 7 22:23:51.254709863 (12h44m12.998766733s) [upload=13099]: Got next segment. Inline: true Aug 7 22:23:51.254715030 (12h44m12.99877189s) [upload=13099]: Waiting for error group managing segments... Aug 7 22:23:51.254768832 (12h44m12.998825693s) [upload=13099]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:23:51.254774395 (12h44m12.998831255s) [upload=13099]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:23:51.254778260 (12h44m12.998835121s) [upload=13099]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:24:20.980586635 (12h44m42.724643486s) [upload=13099]: Done uploading segments... Aug 7 22:24:20.980611487 (12h44m42.724668337s) [upload=13099]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:24:22.809871679 (12h44m44.55392853s) [upload=13108]: Starting upload Aug 7 22:24:22.809959685 (12h44m44.554016545s) [upload=13108]: Done starting upload Aug 7 22:24:22.810026016 (12h44m44.55408288s) [upload=13108]: Uploading segments... Aug 7 22:24:22.810084984 (12h44m44.554141856s) [upload=13108]: Got next segment. Inline: true Aug 7 22:24:22.810090937 (12h44m44.554147807s) [upload=13108]: Waiting for error group managing segments... Aug 7 22:24:22.810180251 (12h44m44.554237121s) [upload=13108]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:24:22.810187976 (12h44m44.554244835s) [upload=13108]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:24:22.810192661 (12h44m44.55424953s) [upload=13108]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:24:52.535275009 (12h45m14.279331864s) [upload=13108]: Done uploading segments... Aug 7 22:24:52.535303649 (12h45m14.279360515s) [upload=13108]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:24:52.960601883 (12h45m14.70465875s) [upload=13112]: Starting upload Aug 7 22:24:52.960708121 (12h45m14.704764984s) [upload=13112]: Done starting upload Aug 7 22:24:52.960845573 (12h45m14.704902419s) [upload=13112]: Uploading segments... Aug 7 22:24:52.960931513 (12h45m14.704988384s) [upload=13112]: Got next segment. Inline: true Aug 7 22:24:52.960939376 (12h45m14.704996243s) [upload=13112]: Waiting for error group managing segments... Aug 7 22:24:52.961019433 (12h45m14.705076313s) [upload=13112]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:24:52.961026755 (12h45m14.705083622s) [upload=13112]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:24:52.961030905 (12h45m14.705087769s) [upload=13112]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:25:22.687567107 (12h45m44.431623957s) [upload=13112]: Done uploading segments... Aug 7 22:25:22.687590061 (12h45m44.431646912s) [upload=13112]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:25:23.131013864 (12h45m44.875070712s) [upload=13122]: Starting upload Aug 7 22:25:23.131088619 (12h45m44.875145482s) [upload=13122]: Done starting upload Aug 7 22:25:23.131275088 (12h45m44.875331941s) [upload=13122]: Uploading segments... Aug 7 22:25:23.131376992 (12h45m44.875433843s) [upload=13122]: Got next segment. Inline: true Aug 7 22:25:23.131382965 (12h45m44.875439816s) [upload=13122]: Waiting for error group managing segments... Aug 7 22:25:23.131434882 (12h45m44.875491733s) [upload=13122]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:25:23.131440120 (12h45m44.875496982s) [upload=13122]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:25:23.131443668 (12h45m44.875500519s) [upload=13122]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:25:23.272670449 (12h45m45.016727299s) [upload=13122]: Done uploading segments... Aug 7 22:25:23.272702241 (12h45m45.016759091s) [upload=13122]: Upload finished. err: <nil> Aug 7 22:26:25.811198326 (12h46m47.555255193s) [upload=13140]: Starting upload Aug 7 22:26:25.811285415 (12h46m47.555342279s) [upload=13140]: Done starting upload Aug 7 22:26:25.811369732 (12h46m47.555426582s) [upload=13140]: Uploading segments... Aug 7 22:26:25.811467114 (12h46m47.55552398s) [upload=13140]: Got next segment. Inline: true Aug 7 22:26:25.811473701 (12h46m47.555530552s) [upload=13140]: Waiting for error group managing segments... Aug 7 22:26:25.811539126 (12h46m47.555595979s) [upload=13140]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:26:25.811545282 (12h46m47.555602146s) [upload=13140]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:26:25.811548939 (12h46m47.555605802s) [upload=13140]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:26:25.968679249 (12h46m47.712736113s) [upload=13140]: Done uploading segments... Aug 7 22:26:25.968709074 (12h46m47.712765928s) [upload=13140]: Upload finished. err: <nil> Aug 7 22:27:27.799853559 (12h47m49.543910409s) [upload=13154]: Starting upload Aug 7 22:27:27.799998197 (12h47m49.544055055s) [upload=13154]: Done starting upload Aug 7 22:27:27.800097878 (12h47m49.544154732s) [upload=13154]: Uploading segments... Aug 7 22:27:27.800202613 (12h47m49.544259463s) [upload=13154]: Got next segment. Inline: true Aug 7 22:27:27.800221436 (12h47m49.544278297s) [upload=13154]: Waiting for error group managing segments... Aug 7 22:27:27.800305006 (12h47m49.544361867s) [upload=13154]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:27:27.800310790 (12h47m49.54436765s) [upload=13154]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:27:27.800314678 (12h47m49.544371539s) [upload=13154]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:27:27.950105257 (12h47m49.694162117s) [upload=13154]: Done uploading segments... Aug 7 22:27:27.950124372 (12h47m49.694181233s) [upload=13154]: Upload finished. err: <nil> Aug 7 22:27:59.550898410 (12h48m21.29495526s) [upload=13165]: Starting upload Aug 7 22:27:59.550970829 (12h48m21.295027691s) [upload=13165]: Done starting upload Aug 7 22:27:59.551049380 (12h48m21.295106232s) [upload=13165]: Uploading segments... Aug 7 22:27:59.551283044 (12h48m21.295339906s) [upload=13165]: Got next segment. Inline: true Aug 7 22:27:59.551293209 (12h48m21.295350069s) [upload=13165]: Waiting for error group managing segments... Aug 7 22:27:59.551359587 (12h48m21.295416438s) [upload=13165]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:27:59.551365951 (12h48m21.295422812s) [upload=13165]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:27:59.551369453 (12h48m21.295426314s) [upload=13165]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:27:59.691525895 (12h48m21.435582778s) [upload=13165]: Done uploading segments... Aug 7 22:27:59.691551086 (12h48m21.435607933s) [upload=13165]: Upload finished. err: <nil> Aug 7 22:29:06.342665932 (12h49m28.086722791s) [upload=13183]: Starting upload Aug 7 22:29:06.342756156 (12h49m28.086813016s) [upload=13183]: Done starting upload Aug 7 22:29:06.342875471 (12h49m28.086932321s) [upload=13183]: Uploading segments... Aug 7 22:29:06.343020027 (12h49m28.087076894s) [upload=13183]: Got next segment. Inline: true Aug 7 22:29:06.343029056 (12h49m28.087085926s) [upload=13183]: Waiting for error group managing segments... Aug 7 22:29:06.343276816 (12h49m28.087333721s) [upload=13183]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:29:06.343287054 (12h49m28.087343919s) [upload=13183]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:29:06.343290895 (12h49m28.08734776s) [upload=13183]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:29:36.070785151 (12h49m57.814842003s) [upload=13183]: Done uploading segments... Aug 7 22:29:36.070807148 (12h49m57.814863999s) [upload=13183]: Upload finished. err: metaclient: context deadline exceeded Duration: 866.881181ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 18 out of 60 is failed (threshold=3) [..................................XXXXXXXXXXXXXXXXXX........] duration: 2.521777704s FAIL (2025-08-07 22:33:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.521832078s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 19 out of 60 is failed (threshold=3) [..............................XXXXXXXXXXXXXXXX.XXX..........] duration: 1.023862063s FAIL (2025-08-07 22:33:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.02388803s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [..X...................................XXXXXXXXXXXXX..X......] duration: 2.96080718s FAIL (2025-08-07 22:33:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== 7 22:27:44.139472223 (12h48m5.883529083s) [upload=13160]: Got next segment. Inline: false Aug 7 22:27:44.139508151 (12h48m5.883565011s) [upload=13160]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:27:44.139513399 (12h48m5.883570259s) [upload=13160]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:27:44.245907064 (12h48m5.989963927s) [seg_pos={0 0}, upload=13160]: Begin upload segment... Aug 7 22:27:44.246039735 (12h48m5.990096597s) [noise=true , address=104.36.82.10:4079 , piece_id=2ADIUNAB5FPKLXMW46HBB75SJLYOKA4N3TYS6J72WZDKLQTSP23A, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.246596012 (12h48m5.990652882s) [noise=true , address=104.36.82.4:4801 , piece_id=U6XCDALXGOTJHRL2P4V7DOPIJJF6VUKB5RIJYOP3WXFTC6HQKQXQ, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.247006251 (12h48m5.991063115s) [noise=true , address=77.90.31.12:4590 , piece_id=IYZTGAPBVXXQNQVDNR4NZRPDNNIBETBQ26DZUZ2SODA2D2VQUXTA, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.247197178 (12h48m5.991254041s) [seg_pos={0 0}, upload=13160]: Done begin upload segment. Aug 7 22:27:44.247225152 (12h48m5.991282018s) [noise=true , address=104.36.82.5:4837 , piece_id=XDA3KTWSH5OJPHK4IFOPINMJAR6VJLL7B7WNO3MV2VT3HQCGJ2KA, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.247751444 (12h48m5.99180831s) [noise=true , address=104.36.82.6:4876 , piece_id=45DZFNAUCX6EJNPT4WBVLEIO2GZROYIF2NQ254LU32T6AE7P4ZRA, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.247956514 (12h48m5.992013382s) [noise=true , address=104.36.82.11:4115 , piece_id=B2C72IS3FGY7DT4SXLUG5FYHCN5ECAMCAIVT2ZNVCXML2RVCQZFQ, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.248499391 (12h48m5.992556252s) [noise=true , address=45.91.116.12:4225 , piece_id=N7R42AWZKTHBBPZY24OVKSBRTPGVBLJ3HBYB2ONRZ56F6YZK2XWQ, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.248606386 (12h48m5.992663249s) [noise=true , address=77.90.31.13:4628 , piece_id=FSARBSWO4VFKOYHINODNYG7BC4JEH6KX5Z3IOYIJ4DGQ3DCZNDJA, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.249110151 (12h48m5.99316702s) [noise=true , address=45.91.116.9:4261 , piece_id=ZESRB2XAQ4WPIB2HQKYLZKJDXDE65NXMRX4M5JE3EKRHROCT4JKA, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.249195913 (12h48m5.993252781s) [noise=true , address=104.36.82.13:4185 , piece_id=QI54Y53UTBR7JEQ4NPD5R37EC47364LEUWETTCGI3WQUKSHUTP5A, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.249587895 (12h48m5.993644759s) [noise=true , address=45.91.116.8:4047 , piece_id=LDURVPEWVV7OEPVY4L4ZER4H2EDUPJS45N55Y57VZUTLSTKGXSRA, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.249756005 (12h48m5.993812872s) [noise=true , address=77.90.31.11:4553 , piece_id=ZDJ5BW2XPNHEBNB6FBVWAH66T6SVKAUY3724YY6HZ25VBLKYFTIQ, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.250801847 (12h48m5.99485871s) [noise=true , address=77.90.31.10:4517 , piece_id=QHV63D3QBJ5GVDWNHGWJTLXQNHGK4PZ4OCT4UWCEQKHMYBDL26WA, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.251094602 (12h48m5.995151463s) [noise=true , address=77.90.31.18:4806 , piece_id=ZJLVNSXETRBXYNHIDLN7TSQU6OQVWTCOQFJLPWAMCPY2JHBTHVQA, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.251408354 (12h48m5.995465217s) [noise=true , address=104.36.82.10:4078 , piece_id=UCDTQMJTYXAMJXGRPKBGTJZ4OCF4P2BSCR6WR3KSXCK6S5PWWZ7A, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.251693979 (12h48m5.995750839s) [noise=true , address=45.91.116.5:4297 , piece_id=QEQZU67YDQXNW7ACIRVXJKX2HIKMZMAP23TCAMTB2RDK3RFKLLRQ, seg_pos={0 0}, upload=13160]: Uploading piece... Aug 7 22:27:44.928589281 (12h48m6.672646155s) [upload=13160]: Next returned nil segment Aug 7 22:27:44.928610126 (12h48m6.672666995s) [upload=13160]: Waiting for error group managing segments... Aug 7 22:27:45.055730069 (12h48m6.799786923s) [noise=true , address=104.36.82.11:4115 , piece_id=B2C72IS3FGY7DT4SXLUG5FYHCN5ECAMCAIVT2ZNVCXML2RVCQZFQ, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.058230571 (12h48m6.802287421s) [noise=true , address=104.36.82.4:4801 , piece_id=U6XCDALXGOTJHRL2P4V7DOPIJJF6VUKB5RIJYOP3WXFTC6HQKQXQ, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.072176193 (12h48m6.816233043s) [noise=true , address=104.36.82.6:4876 , piece_id=45DZFNAUCX6EJNPT4WBVLEIO2GZROYIF2NQ254LU32T6AE7P4ZRA, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.076144433 (12h48m6.82020128s) [noise=true , address=104.36.82.5:4837 , piece_id=XDA3KTWSH5OJPHK4IFOPINMJAR6VJLL7B7WNO3MV2VT3HQCGJ2KA, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.083969611 (12h48m6.828026457s) [noise=true , address=104.36.82.13:4185 , piece_id=QI54Y53UTBR7JEQ4NPD5R37EC47364LEUWETTCGI3WQUKSHUTP5A, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.090499988 (12h48m6.834556849s) [noise=true , address=104.36.82.10:4078 , piece_id=UCDTQMJTYXAMJXGRPKBGTJZ4OCF4P2BSCR6WR3KSXCK6S5PWWZ7A, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.129675895 (12h48m6.873732757s) [noise=true , address=45.91.116.12:4225 , piece_id=N7R42AWZKTHBBPZY24OVKSBRTPGVBLJ3HBYB2ONRZ56F6YZK2XWQ, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.148504731 (12h48m6.892561581s) [noise=true , address=77.90.31.10:4517 , piece_id=QHV63D3QBJ5GVDWNHGWJTLXQNHGK4PZ4OCT4UWCEQKHMYBDL26WA, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.183435281 (12h48m6.927492128s) [noise=true , address=104.36.82.10:4079 , piece_id=2ADIUNAB5FPKLXMW46HBB75SJLYOKA4N3TYS6J72WZDKLQTSP23A, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.188056276 (12h48m6.932113137s) [noise=true , address=77.90.31.12:4590 , piece_id=IYZTGAPBVXXQNQVDNR4NZRPDNNIBETBQ26DZUZ2SODA2D2VQUXTA, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.189109860 (12h48m6.93316672s) [noise=true , address=77.90.31.13:4628 , piece_id=FSARBSWO4VFKOYHINODNYG7BC4JEH6KX5Z3IOYIJ4DGQ3DCZNDJA, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.191395584 (12h48m6.93545243s) [noise=true , address=45.91.116.9:4261 , piece_id=ZESRB2XAQ4WPIB2HQKYLZKJDXDE65NXMRX4M5JE3EKRHROCT4JKA, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.212167167 (12h48m6.956224014s) [noise=true , address=77.90.31.11:4553 , piece_id=ZDJ5BW2XPNHEBNB6FBVWAH66T6SVKAUY3724YY6HZ25VBLKYFTIQ, seg_pos={0 0}, upload=13160]: Done uploading piece. err: <nil> Aug 7 22:27:45.212191124 (12h48m6.956247983s) [seg_pos={0 0}, upload=13160]: Segment reached optimal threshold of 13 pieces. Aug 7 22:27:45.212404510 (12h48m6.956461359s) [noise=true , address=45.91.116.8:4047 , piece_id=LDURVPEWVV7OEPVY4L4ZER4H2EDUPJS45N55Y57VZUTLSTKGXSRA, seg_pos={0 0}, upload=13160]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: context canceled; context canceled Aug 7 22:27:45.212444310 (12h48m6.956501156s) [noise=true , address=77.90.31.18:4806 , piece_id=ZJLVNSXETRBXYNHIDLN7TSQU6OQVWTCOQFJLPWAMCPY2JHBTHVQA, seg_pos={0 0}, upload=13160]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: write tcp 139.59.144.22:44926->77.90.31.18:4806: use of closed network connection; write tcp 139.59.144.22:44926->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44926->77.90.31.18:4806: use of closed network connection Aug 7 22:27:45.212507508 (12h48m6.956564355s) [noise=true , address=45.91.116.5:4297 , piece_id=QEQZU67YDQXNW7ACIRVXJKX2HIKMZMAP23TCAMTB2RDK3RFKLLRQ, seg_pos={0 0}, upload=13160]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 139.59.144.22:41424->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:41424->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41424->45.91.116.5:4297: use of closed network connection Aug 7 22:27:45.212525428 (12h48m6.956582288s) [seg_pos={0 0}, upload=13160]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 22:27:45.212587349 (12h48m6.956644209s) [upload=13160]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 22:27:45.212591472 (12h48m6.956648332s) [upload=13160]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:27:45.467451329 (12h48m7.21150818s) [upload=13160]: Done uploading segments... Aug 7 22:27:45.467470231 (12h48m7.211527081s) [upload=13160]: Upload finished. err: <nil> Aug 7 22:28:46.300540067 (12h49m8.044596928s) [upload=13178]: Starting upload Aug 7 22:28:46.300629117 (12h49m8.044685979s) [upload=13178]: Done starting upload Aug 7 22:28:46.300797752 (12h49m8.044854603s) [upload=13178]: Uploading segments... Aug 7 22:28:46.301190738 (12h49m8.045247598s) [upload=13178]: Got next segment. Inline: false Aug 7 22:28:46.301268000 (12h49m8.04532486s) [upload=13178]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:28:46.301275318 (12h49m8.045332178s) [upload=13178]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 22:29:16.009299446 (12h49m37.753356315s) [upload=13178]: Next segment err: context deadline exceeded Aug 7 22:29:16.009371909 (12h49m37.753428759s) [upload=13178]: Done uploading segments... Aug 7 22:29:16.009392178 (12h49m37.753449035s) [upload=13178]: Upload finished. err: context deadline exceeded Duration: 2.960840869s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [........X...X..........................X...............X....] duration: 837.727239ms FAIL (2025-08-07 22:33:47 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== d=13120]: Done uploading segments... Aug 7 22:25:13.463635961 (12h45m35.207692812s) [upload=13120]: Upload finished. err: <nil> Aug 7 22:25:36.676221958 (12h45m58.42027881s) [upload=13126]: Starting upload Aug 7 22:25:36.676280024 (12h45m58.420336886s) [upload=13126]: Done starting upload Aug 7 22:25:36.676348826 (12h45m58.420405681s) [upload=13126]: Uploading segments... Aug 7 22:25:36.676415761 (12h45m58.42047263s) [upload=13126]: Got next segment. Inline: true Aug 7 22:25:36.676421620 (12h45m58.420478481s) [upload=13126]: Waiting for error group managing segments... Aug 7 22:25:36.676463453 (12h45m58.420520304s) [upload=13126]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:25:36.676469083 (12h45m58.420525944s) [upload=13126]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:25:36.676472693 (12h45m58.420529553s) [upload=13126]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:25:36.846956085 (12h45m58.591012935s) [upload=13126]: Done uploading segments... Aug 7 22:25:36.846976507 (12h45m58.591033359s) [upload=13126]: Upload finished. err: <nil> Aug 7 22:26:18.836000763 (12h46m40.580057613s) [upload=13137]: Starting upload Aug 7 22:26:18.836038438 (12h46m40.580095288s) [upload=13137]: Done starting upload Aug 7 22:26:18.836098090 (12h46m40.58015494s) [upload=13137]: Uploading segments... Aug 7 22:26:18.836143460 (12h46m40.580200331s) [upload=13137]: Got next segment. Inline: true Aug 7 22:26:18.836148461 (12h46m40.580205309s) [upload=13137]: Waiting for error group managing segments... Aug 7 22:26:18.836191879 (12h46m40.580248725s) [upload=13137]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:26:18.836197487 (12h46m40.580254348s) [upload=13137]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:26:18.836205083 (12h46m40.580261941s) [upload=13137]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:26:18.978690589 (12h46m40.722747448s) [upload=13137]: Done uploading segments... Aug 7 22:26:18.978706599 (12h46m40.722763445s) [upload=13137]: Upload finished. err: <nil> Aug 7 22:27:06.613274662 (12h47m28.357331513s) [upload=13150]: Starting upload Aug 7 22:27:06.613358594 (12h47m28.357415443s) [upload=13150]: Done starting upload Aug 7 22:27:06.613451706 (12h47m28.357508565s) [upload=13150]: Uploading segments... Aug 7 22:27:06.613532713 (12h47m28.357589583s) [upload=13150]: Got next segment. Inline: true Aug 7 22:27:06.613540195 (12h47m28.357597165s) [upload=13150]: Waiting for error group managing segments... Aug 7 22:27:06.613608323 (12h47m28.357665185s) [upload=13150]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:27:06.613616998 (12h47m28.35767387s) [upload=13150]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:27:06.613621123 (12h47m28.357677991s) [upload=13150]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:27:06.756270932 (12h47m28.500327782s) [upload=13150]: Done uploading segments... Aug 7 22:27:06.756289996 (12h47m28.500346845s) [upload=13150]: Upload finished. err: <nil> Aug 7 22:27:39.275964882 (12h48m1.020021742s) [upload=13159]: Starting upload Aug 7 22:27:39.276031028 (12h48m1.020087874s) [upload=13159]: Done starting upload Aug 7 22:27:39.276100015 (12h48m1.020156863s) [upload=13159]: Uploading segments... Aug 7 22:27:39.276162849 (12h48m1.020219721s) [upload=13159]: Got next segment. Inline: true Aug 7 22:27:39.276168125 (12h48m1.020224985s) [upload=13159]: Waiting for error group managing segments... Aug 7 22:27:39.276216009 (12h48m1.020272869s) [upload=13159]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:27:39.276220617 (12h48m1.020277478s) [upload=13159]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:27:39.276223278 (12h48m1.020280137s) [upload=13159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:27:39.491372003 (12h48m1.235428853s) [upload=13159]: Done uploading segments... Aug 7 22:27:39.491394004 (12h48m1.235450857s) [upload=13159]: Upload finished. err: <nil> Aug 7 22:28:03.413359947 (12h48m25.157416798s) [upload=13167]: Starting upload Aug 7 22:28:03.413424697 (12h48m25.157481559s) [upload=13167]: Done starting upload Aug 7 22:28:03.413546352 (12h48m25.157603207s) [upload=13167]: Uploading segments... Aug 7 22:28:03.413695999 (12h48m25.157752868s) [upload=13167]: Got next segment. Inline: true Aug 7 22:28:03.413705607 (12h48m25.157762476s) [upload=13167]: Waiting for error group managing segments... Aug 7 22:28:03.413788849 (12h48m25.157845716s) [upload=13167]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:28:03.413816554 (12h48m25.157873423s) [upload=13167]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:28:03.413820681 (12h48m25.157877544s) [upload=13167]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:28:03.561945313 (12h48m25.306002181s) [upload=13167]: Done uploading segments... Aug 7 22:28:03.561970962 (12h48m25.306027825s) [upload=13167]: Upload finished. err: <nil> Aug 7 22:28:25.535904284 (12h48m47.279961133s) [upload=13172]: Starting upload Aug 7 22:28:25.535961986 (12h48m47.280018836s) [upload=13172]: Done starting upload Aug 7 22:28:25.536032757 (12h48m47.280089607s) [upload=13172]: Uploading segments... Aug 7 22:28:25.536074131 (12h48m47.280131001s) [upload=13172]: Got next segment. Inline: true Aug 7 22:28:25.536089717 (12h48m47.280146578s) [upload=13172]: Waiting for error group managing segments... Aug 7 22:28:25.536134532 (12h48m47.280191382s) [upload=13172]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:28:25.536151821 (12h48m47.280208671s) [upload=13172]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:28:25.536166195 (12h48m47.280223058s) [upload=13172]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:28:26.345427619 (12h48m48.089484469s) [upload=13172]: Done uploading segments... Aug 7 22:28:26.345447034 (12h48m48.089503895s) [upload=13172]: Upload finished. err: <nil> Aug 7 22:28:49.552061521 (12h49m11.296118389s) [upload=13179]: Starting upload Aug 7 22:28:49.552134904 (12h49m11.296191767s) [upload=13179]: Done starting upload Aug 7 22:28:49.552208829 (12h49m11.296265676s) [upload=13179]: Uploading segments... Aug 7 22:28:49.552262194 (12h49m11.296319062s) [upload=13179]: Got next segment. Inline: true Aug 7 22:28:49.552268137 (12h49m11.296324998s) [upload=13179]: Waiting for error group managing segments... Aug 7 22:28:49.552316820 (12h49m11.296373667s) [upload=13179]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:28:49.552323169 (12h49m11.29638003s) [upload=13179]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:28:49.552327059 (12h49m11.29638392s) [upload=13179]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:28:49.693951093 (12h49m11.438007952s) [upload=13179]: Done uploading segments... Aug 7 22:28:49.693988901 (12h49m11.438045762s) [upload=13179]: Upload finished. err: <nil> Aug 7 22:29:09.820328663 (12h49m31.564385513s) [upload=13184]: Starting upload Aug 7 22:29:09.820384335 (12h49m31.564441196s) [upload=13184]: Done starting upload Aug 7 22:29:09.820445932 (12h49m31.564502786s) [upload=13184]: Uploading segments... Aug 7 22:29:09.820487651 (12h49m31.564544517s) [upload=13184]: Got next segment. Inline: true Aug 7 22:29:09.820492611 (12h49m31.564549472s) [upload=13184]: Waiting for error group managing segments... Aug 7 22:29:09.820534051 (12h49m31.564590901s) [upload=13184]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:29:09.820539062 (12h49m31.564595922s) [upload=13184]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:29:09.820542725 (12h49m31.564599585s) [upload=13184]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:29:10.028668214 (12h49m31.772725062s) [upload=13184]: Done uploading segments... Aug 7 22:29:10.028692776 (12h49m31.772749623s) [upload=13184]: Upload finished. err: <nil> Aug 7 22:30:21.625872621 (12h50m43.369929471s) [upload=13201]: Starting upload Aug 7 22:30:21.625976134 (12h50m43.370033002s) [upload=13201]: Done starting upload Aug 7 22:30:21.626054127 (12h50m43.370110973s) [upload=13201]: Uploading segments... Aug 7 22:30:21.626095581 (12h50m43.370152447s) [upload=13201]: Got next segment. Inline: true Aug 7 22:30:21.626101049 (12h50m43.370157915s) [upload=13201]: Waiting for error group managing segments... Aug 7 22:30:21.626141797 (12h50m43.370198644s) [upload=13201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:30:21.626147151 (12h50m43.370204015s) [upload=13201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:30:21.626149721 (12h50m43.370206581s) [upload=13201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:30:21.760255105 (12h50m43.504311956s) [upload=13201]: Done uploading segments... Aug 7 22:30:21.760273763 (12h50m43.504330614s) [upload=13201]: Upload finished. err: <nil> Aug 7 22:30:47.060687315 (12h51m8.804744175s) [upload=13211]: Starting upload Aug 7 22:30:47.060743560 (12h51m8.80480042s) [upload=13211]: Done starting upload Aug 7 22:30:47.060819066 (12h51m8.804875915s) [upload=13211]: Uploading segments... Aug 7 22:30:47.060872995 (12h51m8.804929861s) [upload=13211]: Got next segment. Inline: true Aug 7 22:30:47.060878710 (12h51m8.80493557s) [upload=13211]: Waiting for error group managing segments... Aug 7 22:30:47.060932110 (12h51m8.804988961s) [upload=13211]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:30:47.060937671 (12h51m8.80499453s) [upload=13211]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:30:47.060940905 (12h51m8.804997769s) [upload=13211]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:30:48.162928472 (12h51m9.906985333s) [upload=13211]: Done uploading segments... Aug 7 22:30:48.162948603 (12h51m9.907005457s) [upload=13211]: Upload finished. err: <nil> Duration: 837.759885ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 2 out of 60 is failed (threshold=3) [.....................................X............X.........] duration: 108.344736ms remote: "tcp:185.244.226.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 22:33:46 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 108.368197ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 17 out of 60 is failed (threshold=3) [..............................XXXXXXXXXXXX.XXXX....X........] duration: 856.620937ms FAIL (2025-08-07 22:33:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== [upload=13077]: Starting upload Aug 7 22:22:03.530157956 (12h42m25.27421483s) [upload=13077]: Done starting upload Aug 7 22:22:03.530254340 (12h42m25.274311199s) [upload=13077]: Uploading segments... Aug 7 22:22:03.530319521 (12h42m25.274376398s) [upload=13077]: Got next segment. Inline: true Aug 7 22:22:03.530326839 (12h42m25.27438371s) [upload=13077]: Waiting for error group managing segments... Aug 7 22:22:03.530387033 (12h42m25.274443903s) [upload=13077]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:22:03.530393764 (12h42m25.274450635s) [upload=13077]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:22:03.530398068 (12h42m25.274454934s) [upload=13077]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:22:03.668101987 (12h42m25.412158856s) [upload=13077]: Done uploading segments... Aug 7 22:22:03.668146181 (12h42m25.412203031s) [upload=13077]: Upload finished. err: <nil> Aug 7 22:22:50.781175997 (12h43m12.525232844s) [upload=13085]: Starting upload Aug 7 22:22:50.781238492 (12h43m12.525295338s) [upload=13085]: Done starting upload Aug 7 22:22:50.781295832 (12h43m12.525352678s) [upload=13085]: Uploading segments... Aug 7 22:22:50.781339974 (12h43m12.525396844s) [upload=13085]: Got next segment. Inline: true Aug 7 22:22:50.781344619 (12h43m12.525401479s) [upload=13085]: Waiting for error group managing segments... Aug 7 22:22:50.781382529 (12h43m12.525439375s) [upload=13085]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:22:50.781387267 (12h43m12.525444113s) [upload=13085]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:22:50.781389841 (12h43m12.525446704s) [upload=13085]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:23:20.513298723 (12h43m42.257355587s) [upload=13085]: Done uploading segments... Aug 7 22:23:20.513325654 (12h43m42.25738252s) [upload=13085]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:23:51.254516692 (12h44m12.998573555s) [upload=13099]: Starting upload Aug 7 22:23:51.254606307 (12h44m12.998663169s) [upload=13099]: Done starting upload Aug 7 22:23:51.254669521 (12h44m12.998726371s) [upload=13099]: Uploading segments... Aug 7 22:23:51.254709863 (12h44m12.998766733s) [upload=13099]: Got next segment. Inline: true Aug 7 22:23:51.254715030 (12h44m12.99877189s) [upload=13099]: Waiting for error group managing segments... Aug 7 22:23:51.254768832 (12h44m12.998825693s) [upload=13099]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:23:51.254774395 (12h44m12.998831255s) [upload=13099]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:23:51.254778260 (12h44m12.998835121s) [upload=13099]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:24:20.980586635 (12h44m42.724643486s) [upload=13099]: Done uploading segments... Aug 7 22:24:20.980611487 (12h44m42.724668337s) [upload=13099]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:24:22.809871679 (12h44m44.55392853s) [upload=13108]: Starting upload Aug 7 22:24:22.809959685 (12h44m44.554016545s) [upload=13108]: Done starting upload Aug 7 22:24:22.810026016 (12h44m44.55408288s) [upload=13108]: Uploading segments... Aug 7 22:24:22.810084984 (12h44m44.554141856s) [upload=13108]: Got next segment. Inline: true Aug 7 22:24:22.810090937 (12h44m44.554147807s) [upload=13108]: Waiting for error group managing segments... Aug 7 22:24:22.810180251 (12h44m44.554237121s) [upload=13108]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:24:22.810187976 (12h44m44.554244835s) [upload=13108]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:24:22.810192661 (12h44m44.55424953s) [upload=13108]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:24:52.535275009 (12h45m14.279331864s) [upload=13108]: Done uploading segments... Aug 7 22:24:52.535303649 (12h45m14.279360515s) [upload=13108]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:24:52.960601883 (12h45m14.70465875s) [upload=13112]: Starting upload Aug 7 22:24:52.960708121 (12h45m14.704764984s) [upload=13112]: Done starting upload Aug 7 22:24:52.960845573 (12h45m14.704902419s) [upload=13112]: Uploading segments... Aug 7 22:24:52.960931513 (12h45m14.704988384s) [upload=13112]: Got next segment. Inline: true Aug 7 22:24:52.960939376 (12h45m14.704996243s) [upload=13112]: Waiting for error group managing segments... Aug 7 22:24:52.961019433 (12h45m14.705076313s) [upload=13112]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:24:52.961026755 (12h45m14.705083622s) [upload=13112]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:24:52.961030905 (12h45m14.705087769s) [upload=13112]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:25:22.687567107 (12h45m44.431623957s) [upload=13112]: Done uploading segments... Aug 7 22:25:22.687590061 (12h45m44.431646912s) [upload=13112]: Upload finished. err: metaclient: context deadline exceeded Aug 7 22:25:23.131013864 (12h45m44.875070712s) [upload=13122]: Starting upload Aug 7 22:25:23.131088619 (12h45m44.875145482s) [upload=13122]: Done starting upload Aug 7 22:25:23.131275088 (12h45m44.875331941s) [upload=13122]: Uploading segments... Aug 7 22:25:23.131376992 (12h45m44.875433843s) [upload=13122]: Got next segment. Inline: true Aug 7 22:25:23.131382965 (12h45m44.875439816s) [upload=13122]: Waiting for error group managing segments... Aug 7 22:25:23.131434882 (12h45m44.875491733s) [upload=13122]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:25:23.131440120 (12h45m44.875496982s) [upload=13122]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:25:23.131443668 (12h45m44.875500519s) [upload=13122]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:25:23.272670449 (12h45m45.016727299s) [upload=13122]: Done uploading segments... Aug 7 22:25:23.272702241 (12h45m45.016759091s) [upload=13122]: Upload finished. err: <nil> Aug 7 22:26:25.811198326 (12h46m47.555255193s) [upload=13140]: Starting upload Aug 7 22:26:25.811285415 (12h46m47.555342279s) [upload=13140]: Done starting upload Aug 7 22:26:25.811369732 (12h46m47.555426582s) [upload=13140]: Uploading segments... Aug 7 22:26:25.811467114 (12h46m47.55552398s) [upload=13140]: Got next segment. Inline: true Aug 7 22:26:25.811473701 (12h46m47.555530552s) [upload=13140]: Waiting for error group managing segments... Aug 7 22:26:25.811539126 (12h46m47.555595979s) [upload=13140]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:26:25.811545282 (12h46m47.555602146s) [upload=13140]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:26:25.811548939 (12h46m47.555605802s) [upload=13140]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:26:25.968679249 (12h46m47.712736113s) [upload=13140]: Done uploading segments... Aug 7 22:26:25.968709074 (12h46m47.712765928s) [upload=13140]: Upload finished. err: <nil> Aug 7 22:27:27.799853559 (12h47m49.543910409s) [upload=13154]: Starting upload Aug 7 22:27:27.799998197 (12h47m49.544055055s) [upload=13154]: Done starting upload Aug 7 22:27:27.800097878 (12h47m49.544154732s) [upload=13154]: Uploading segments... Aug 7 22:27:27.800202613 (12h47m49.544259463s) [upload=13154]: Got next segment. Inline: true Aug 7 22:27:27.800221436 (12h47m49.544278297s) [upload=13154]: Waiting for error group managing segments... Aug 7 22:27:27.800305006 (12h47m49.544361867s) [upload=13154]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:27:27.800310790 (12h47m49.54436765s) [upload=13154]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:27:27.800314678 (12h47m49.544371539s) [upload=13154]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:27:27.950105257 (12h47m49.694162117s) [upload=13154]: Done uploading segments... Aug 7 22:27:27.950124372 (12h47m49.694181233s) [upload=13154]: Upload finished. err: <nil> Aug 7 22:27:59.550898410 (12h48m21.29495526s) [upload=13165]: Starting upload Aug 7 22:27:59.550970829 (12h48m21.295027691s) [upload=13165]: Done starting upload Aug 7 22:27:59.551049380 (12h48m21.295106232s) [upload=13165]: Uploading segments... Aug 7 22:27:59.551283044 (12h48m21.295339906s) [upload=13165]: Got next segment. Inline: true Aug 7 22:27:59.551293209 (12h48m21.295350069s) [upload=13165]: Waiting for error group managing segments... Aug 7 22:27:59.551359587 (12h48m21.295416438s) [upload=13165]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:27:59.551365951 (12h48m21.295422812s) [upload=13165]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:27:59.551369453 (12h48m21.295426314s) [upload=13165]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:27:59.691525895 (12h48m21.435582778s) [upload=13165]: Done uploading segments... Aug 7 22:27:59.691551086 (12h48m21.435607933s) [upload=13165]: Upload finished. err: <nil> Aug 7 22:29:06.342665932 (12h49m28.086722791s) [upload=13183]: Starting upload Aug 7 22:29:06.342756156 (12h49m28.086813016s) [upload=13183]: Done starting upload Aug 7 22:29:06.342875471 (12h49m28.086932321s) [upload=13183]: Uploading segments... Aug 7 22:29:06.343020027 (12h49m28.087076894s) [upload=13183]: Got next segment. Inline: true Aug 7 22:29:06.343029056 (12h49m28.087085926s) [upload=13183]: Waiting for error group managing segments... Aug 7 22:29:06.343276816 (12h49m28.087333721s) [upload=13183]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:29:06.343287054 (12h49m28.087343919s) [upload=13183]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:29:06.343290895 (12h49m28.08734776s) [upload=13183]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:29:36.070785151 (12h49m57.814842003s) [upload=13183]: Done uploading segments... Aug 7 22:29:36.070807148 (12h49m57.814863999s) [upload=13183]: Upload finished. err: metaclient: context deadline exceeded Duration: 856.654978ms core uplink us-select-1

Orbiter Check History

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)