Orbiter History

Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure

Orbiter Check History

Showing 50 of 173460 reports (Page 405 of 3470)

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 3.220495574s FAIL (2025-09-16 00:19:23 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 3.220524981s edge gateway us-select-1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 359.331258ms FAIL (2025-09-16 00:19:05 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout Duration: 359.367734ms edge authservice http us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 2 out of 60 is failed (threshold=3) [..XX........................................................] duration: 331.222052ms target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-09-16 00:18:36 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": dial tcp 45.38.24.2:443: i/o timeout Duration: 331.255963ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt", ExpectedStatus: 308, ExpectedRedirect: "https://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt"} failure (antiblip window 3/60): (http.us_select_1.linksharing) antiblip: 2 out of 60 is failed (threshold=3) [.......XX...................................................] duration: 212.960162ms target: http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt FAIL (2025-09-16 00:18:32 UTC) orbiter warning: Get "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt": dial tcp 45.38.24.2:80: i/o timeout Duration: 213.003659ms edge linksharing us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 1.179852992s FAIL (2025-09-16 00:18:30 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 1.179881074s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 2.77663822s FAIL (2025-09-16 00:18:27 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 2.776663989s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..........XX................................................] duration: 3.037323648s FAIL (2025-09-16 00:18:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.037360657s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.554538137s FAIL (2025-09-16 00:18:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 15 23:40:26.070475751 (417h32m46.562908204s) [upload=367243]: Starting upload Sep 15 23:40:26.070566142 (417h32m46.562998574s) [upload=367243]: Done starting upload Sep 15 23:40:26.070793734 (417h32m46.563226168s) [upload=367243]: Uploading segments... Sep 15 23:40:26.071016487 (417h32m46.563448921s) [upload=367243]: Got next segment. Inline: false Sep 15 23:40:26.071063323 (417h32m46.563495775s) [upload=367243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 15 23:40:26.071076339 (417h32m46.563508792s) [upload=367243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 15 23:40:26.184942850 (417h32m46.677375304s) [seg_pos={0 0}, upload=367243]: Begin upload segment... Sep 15 23:40:26.185028069 (417h32m46.677460517s) [seg_pos={0 0}, upload=367243]: Done begin upload segment. Sep 15 23:40:26.185068989 (417h32m46.677501441s) [noise=true , address=45.91.116.10:4083 , piece_id=6QPRCJSSBIIVFNULJGE2BSONQMHL4ZSBXMEXMISXVV2KCVHJBX3A, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.185724240 (417h32m46.678156692s) [noise=true , address=77.90.31.10:4518 , piece_id=CCDYRFIWKABBJPOCBWPIGEQQYSHNBLKEOQOPXQE5OFSGT6VXCEGQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.186057058 (417h32m46.678489511s) [noise=true , address=45.91.116.8:4048 , piece_id=BYJVXDVFY52FDQYJ7NWQCQ3KASG5ZLSTO66DNYZGT3SPYGXC2YVA, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.186314850 (417h32m46.678747303s) [noise=true , address=45.91.116.7:4009 , piece_id=FJUXJA3SCEHKZQ3TF7XAHBONPENSHA2EZM5VPOUWT5AF5FFAWBXA, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.186491297 (417h32m46.678923748s) [noise=true , address=145.79.227.15:4873 , piece_id=WDMWNGRHW3BXXT54GZOIVGD3EICENKQ2PHNV3LMXWZIBNCOUM2VQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.186897946 (417h32m46.679330395s) [noise=true , address=77.90.31.12:4592 , piece_id=CFBKUEALXCLOEQLPUVBOK4RHCFXFJV5XZHEIPOTFBZIKGLKXYP2A, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.186916513 (417h32m46.679348962s) [noise=true , address=145.79.227.11:4730 , piece_id=JNN7RLZF3OTHAFVIYPFUTOEDREQ3CNN5LXIUFGAD4XNKQEUQIWRQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.187216141 (417h32m46.679648585s) [noise=true , address=45.91.116.8:4047 , piece_id=FTL6SUC6QJGQL6FSE7QVDQOOOBMA6HO2URKVQBQD5WW6ON5LM6CA, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.187410059 (417h32m46.67984251s) [noise=true , address=45.91.116.6:4973 , piece_id=6CEAU6FYIJ3OFB5GR57SVP45CJHKTYCFQIL5YPYQ6CGUCKB53CVA, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.187548554 (417h32m46.679981007s) [noise=true , address=77.90.31.10:4520 , piece_id=RZNYFFC4AVVFRKDRKVAVAPX2JNUNIMOCWCVG3L3LETHT4HZEF4CQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.187891239 (417h32m46.680323691s) [noise=true , address=45.91.116.6:4974 , piece_id=WJL5MLTVRLGWJLYDYOLNXZ2EFOEVOVSMFUYWHKOA3TTWZLKWQUSA, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.187930010 (417h32m46.680362462s) [noise=true , address=77.90.31.21:4879 , piece_id=WQDIRDNI4CFMT75KZ3TNQY5O6K5UGNTEUQUD3KM6PGLXAB2KJRVQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.188300671 (417h32m46.680733122s) [noise=true , address=45.91.116.11:4118 , piece_id=BBDOSLC5DCETT7MVATBJH6MGBEJ3J5GV7I5EU3X7F4Y7L7A7N3WQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.188597680 (417h32m46.681030122s) [noise=true , address=77.90.31.17:4771 , piece_id=OOUSAWV2GEVAIQ4K7C3EWFX6SO3QM7O4WFY6QVXI5FB4OZ2KGYUQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.188939904 (417h32m46.681372353s) [noise=true , address=145.79.227.4:4185 , piece_id=IXUXFHQSDIBJ3K3LR26ERNKEA2QXZVU5QH3GKZ65NOKWVBBNRB6A, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.189187119 (417h32m46.681619564s) [noise=true , address=77.90.31.17:4772 , piece_id=76ORLKXHG3WEWRSLLOHQW3RRGVU5GYG3IEJGHNMWLMPZYS7MSFUQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.808170182 (417h32m47.300602626s) [upload=367243]: Next returned nil segment Sep 15 23:40:26.808216420 (417h32m47.300648865s) [upload=367243]: Waiting for error group managing segments... Sep 15 23:40:26.946199863 (417h32m47.438632293s) [noise=true , address=45.91.116.11:4118 , piece_id=BBDOSLC5DCETT7MVATBJH6MGBEJ3J5GV7I5EU3X7F4Y7L7A7N3WQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:26.946330956 (417h32m47.438763403s) [noise=true , address=45.91.116.7:4009 , piece_id=FJUXJA3SCEHKZQ3TF7XAHBONPENSHA2EZM5VPOUWT5AF5FFAWBXA, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:26.958202290 (417h32m47.450634723s) [noise=true , address=45.91.116.6:4973 , piece_id=6CEAU6FYIJ3OFB5GR57SVP45CJHKTYCFQIL5YPYQ6CGUCKB53CVA, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:26.961831650 (417h32m47.454264082s) [noise=true , address=45.91.116.10:4083 , piece_id=6QPRCJSSBIIVFNULJGE2BSONQMHL4ZSBXMEXMISXVV2KCVHJBX3A, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:26.962849577 (417h32m47.455282006s) [noise=true , address=45.91.116.6:4974 , piece_id=WJL5MLTVRLGWJLYDYOLNXZ2EFOEVOVSMFUYWHKOA3TTWZLKWQUSA, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:26.969890777 (417h32m47.46232321s) [noise=true , address=45.91.116.8:4048 , piece_id=BYJVXDVFY52FDQYJ7NWQCQ3KASG5ZLSTO66DNYZGT3SPYGXC2YVA, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:27.005327018 (417h32m47.497759469s) [noise=true , address=45.91.116.8:4047 , piece_id=FTL6SUC6QJGQL6FSE7QVDQOOOBMA6HO2URKVQBQD5WW6ON5LM6CA, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:27.050024747 (417h32m47.542457189s) [noise=true , address=145.79.227.15:4873 , piece_id=WDMWNGRHW3BXXT54GZOIVGD3EICENKQ2PHNV3LMXWZIBNCOUM2VQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:27.065442548 (417h32m47.557874977s) [noise=true , address=145.79.227.11:4730 , piece_id=JNN7RLZF3OTHAFVIYPFUTOEDREQ3CNN5LXIUFGAD4XNKQEUQIWRQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:27.166046867 (417h32m47.658479304s) [noise=true , address=145.79.227.4:4185 , piece_id=IXUXFHQSDIBJ3K3LR26ERNKEA2QXZVU5QH3GKZ65NOKWVBBNRB6A, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:55.784307815 (417h33m16.276740265s) [noise=true , address=77.90.31.10:4518 , piece_id=CCDYRFIWKABBJPOCBWPIGEQQYSHNBLKEOQOPXQE5OFSGT6VXCEGQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 139.59.144.22:51944->77.90.31.10:4518: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784436862 (417h33m16.276869296s) [noise=true , address=77.90.31.10:4520 , piece_id=RZNYFFC4AVVFRKDRKVAVAPX2JNUNIMOCWCVG3L3LETHT4HZEF4CQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784539779 (417h33m16.276972222s) [noise=true , address=77.90.31.17:4772 , piece_id=76ORLKXHG3WEWRSLLOHQW3RRGVU5GYG3IEJGHNMWLMPZYS7MSFUQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784719697 (417h33m16.277152133s) [noise=true , address=77.90.31.12:4592 , piece_id=CFBKUEALXCLOEQLPUVBOK4RHCFXFJV5XZHEIPOTFBZIKGLKXYP2A, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784747558 (417h33m16.27718s) [noise=true , address=77.90.31.17:4771 , piece_id=OOUSAWV2GEVAIQ4K7C3EWFX6SO3QM7O4WFY6QVXI5FB4OZ2KGYUQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:52324->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784785329 (417h33m16.277217765s) [noise=true , address=77.90.31.21:4879 , piece_id=WQDIRDNI4CFMT75KZ3TNQY5O6K5UGNTEUQUD3KM6PGLXAB2KJRVQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784818442 (417h33m16.277250887s) [seg_pos={0 0}, upload=367243]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 15 23:40:55.900501127 (417h33m16.392933562s) [upload=367243]: Done uploading segments... Sep 15 23:40:55.900528269 (417h33m16.392960711s) [upload=367243]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.554570417s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 2 out of 60 is failed (threshold=3) [...XX.......................................................] duration: 403.317748ms target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-09-16 00:18:12 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": dial tcp 45.38.24.2:443: i/o timeout Duration: 403.439995ms edge gateway vhost us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 1.282136215s FAIL (2025-09-16 00:18:12 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 1.282165403s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 2.782271441s FAIL (2025-09-16 00:17:55 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 2.782373499s edge gateway us-select-1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 343.216669ms FAIL (2025-09-16 00:17:53 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout Duration: 343.245871ms edge authservice http us-select-1
checkers.HTTP{"GET" "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt", ExpectedStatus: 308, ExpectedRedirect: "https://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt"} failure (antiblip window 3/60): (http.us_select_1.linksharing) antiblip: 2 out of 60 is failed (threshold=3) [........XX..................................................] duration: 217.327712ms target: http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt FAIL (2025-09-16 00:17:51 UTC) orbiter warning: Get "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt": dial tcp 45.38.24.2:80: i/o timeout Duration: 217.35311ms edge linksharing us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 1.107154433s FAIL (2025-09-16 00:17:50 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 1.107214209s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 2.515310513s FAIL (2025-09-16 00:17:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 15 23:40:26.070475751 (417h32m46.562908204s) [upload=367243]: Starting upload Sep 15 23:40:26.070566142 (417h32m46.562998574s) [upload=367243]: Done starting upload Sep 15 23:40:26.070793734 (417h32m46.563226168s) [upload=367243]: Uploading segments... Sep 15 23:40:26.071016487 (417h32m46.563448921s) [upload=367243]: Got next segment. Inline: false Sep 15 23:40:26.071063323 (417h32m46.563495775s) [upload=367243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 15 23:40:26.071076339 (417h32m46.563508792s) [upload=367243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 15 23:40:26.184942850 (417h32m46.677375304s) [seg_pos={0 0}, upload=367243]: Begin upload segment... Sep 15 23:40:26.185028069 (417h32m46.677460517s) [seg_pos={0 0}, upload=367243]: Done begin upload segment. Sep 15 23:40:26.185068989 (417h32m46.677501441s) [noise=true , address=45.91.116.10:4083 , piece_id=6QPRCJSSBIIVFNULJGE2BSONQMHL4ZSBXMEXMISXVV2KCVHJBX3A, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.185724240 (417h32m46.678156692s) [noise=true , address=77.90.31.10:4518 , piece_id=CCDYRFIWKABBJPOCBWPIGEQQYSHNBLKEOQOPXQE5OFSGT6VXCEGQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.186057058 (417h32m46.678489511s) [noise=true , address=45.91.116.8:4048 , piece_id=BYJVXDVFY52FDQYJ7NWQCQ3KASG5ZLSTO66DNYZGT3SPYGXC2YVA, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.186314850 (417h32m46.678747303s) [noise=true , address=45.91.116.7:4009 , piece_id=FJUXJA3SCEHKZQ3TF7XAHBONPENSHA2EZM5VPOUWT5AF5FFAWBXA, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.186491297 (417h32m46.678923748s) [noise=true , address=145.79.227.15:4873 , piece_id=WDMWNGRHW3BXXT54GZOIVGD3EICENKQ2PHNV3LMXWZIBNCOUM2VQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.186897946 (417h32m46.679330395s) [noise=true , address=77.90.31.12:4592 , piece_id=CFBKUEALXCLOEQLPUVBOK4RHCFXFJV5XZHEIPOTFBZIKGLKXYP2A, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.186916513 (417h32m46.679348962s) [noise=true , address=145.79.227.11:4730 , piece_id=JNN7RLZF3OTHAFVIYPFUTOEDREQ3CNN5LXIUFGAD4XNKQEUQIWRQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.187216141 (417h32m46.679648585s) [noise=true , address=45.91.116.8:4047 , piece_id=FTL6SUC6QJGQL6FSE7QVDQOOOBMA6HO2URKVQBQD5WW6ON5LM6CA, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.187410059 (417h32m46.67984251s) [noise=true , address=45.91.116.6:4973 , piece_id=6CEAU6FYIJ3OFB5GR57SVP45CJHKTYCFQIL5YPYQ6CGUCKB53CVA, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.187548554 (417h32m46.679981007s) [noise=true , address=77.90.31.10:4520 , piece_id=RZNYFFC4AVVFRKDRKVAVAPX2JNUNIMOCWCVG3L3LETHT4HZEF4CQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.187891239 (417h32m46.680323691s) [noise=true , address=45.91.116.6:4974 , piece_id=WJL5MLTVRLGWJLYDYOLNXZ2EFOEVOVSMFUYWHKOA3TTWZLKWQUSA, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.187930010 (417h32m46.680362462s) [noise=true , address=77.90.31.21:4879 , piece_id=WQDIRDNI4CFMT75KZ3TNQY5O6K5UGNTEUQUD3KM6PGLXAB2KJRVQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.188300671 (417h32m46.680733122s) [noise=true , address=45.91.116.11:4118 , piece_id=BBDOSLC5DCETT7MVATBJH6MGBEJ3J5GV7I5EU3X7F4Y7L7A7N3WQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.188597680 (417h32m46.681030122s) [noise=true , address=77.90.31.17:4771 , piece_id=OOUSAWV2GEVAIQ4K7C3EWFX6SO3QM7O4WFY6QVXI5FB4OZ2KGYUQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.188939904 (417h32m46.681372353s) [noise=true , address=145.79.227.4:4185 , piece_id=IXUXFHQSDIBJ3K3LR26ERNKEA2QXZVU5QH3GKZ65NOKWVBBNRB6A, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.189187119 (417h32m46.681619564s) [noise=true , address=77.90.31.17:4772 , piece_id=76ORLKXHG3WEWRSLLOHQW3RRGVU5GYG3IEJGHNMWLMPZYS7MSFUQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.808170182 (417h32m47.300602626s) [upload=367243]: Next returned nil segment Sep 15 23:40:26.808216420 (417h32m47.300648865s) [upload=367243]: Waiting for error group managing segments... Sep 15 23:40:26.946199863 (417h32m47.438632293s) [noise=true , address=45.91.116.11:4118 , piece_id=BBDOSLC5DCETT7MVATBJH6MGBEJ3J5GV7I5EU3X7F4Y7L7A7N3WQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:26.946330956 (417h32m47.438763403s) [noise=true , address=45.91.116.7:4009 , piece_id=FJUXJA3SCEHKZQ3TF7XAHBONPENSHA2EZM5VPOUWT5AF5FFAWBXA, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:26.958202290 (417h32m47.450634723s) [noise=true , address=45.91.116.6:4973 , piece_id=6CEAU6FYIJ3OFB5GR57SVP45CJHKTYCFQIL5YPYQ6CGUCKB53CVA, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:26.961831650 (417h32m47.454264082s) [noise=true , address=45.91.116.10:4083 , piece_id=6QPRCJSSBIIVFNULJGE2BSONQMHL4ZSBXMEXMISXVV2KCVHJBX3A, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:26.962849577 (417h32m47.455282006s) [noise=true , address=45.91.116.6:4974 , piece_id=WJL5MLTVRLGWJLYDYOLNXZ2EFOEVOVSMFUYWHKOA3TTWZLKWQUSA, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:26.969890777 (417h32m47.46232321s) [noise=true , address=45.91.116.8:4048 , piece_id=BYJVXDVFY52FDQYJ7NWQCQ3KASG5ZLSTO66DNYZGT3SPYGXC2YVA, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:27.005327018 (417h32m47.497759469s) [noise=true , address=45.91.116.8:4047 , piece_id=FTL6SUC6QJGQL6FSE7QVDQOOOBMA6HO2URKVQBQD5WW6ON5LM6CA, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:27.050024747 (417h32m47.542457189s) [noise=true , address=145.79.227.15:4873 , piece_id=WDMWNGRHW3BXXT54GZOIVGD3EICENKQ2PHNV3LMXWZIBNCOUM2VQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:27.065442548 (417h32m47.557874977s) [noise=true , address=145.79.227.11:4730 , piece_id=JNN7RLZF3OTHAFVIYPFUTOEDREQ3CNN5LXIUFGAD4XNKQEUQIWRQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:27.166046867 (417h32m47.658479304s) [noise=true , address=145.79.227.4:4185 , piece_id=IXUXFHQSDIBJ3K3LR26ERNKEA2QXZVU5QH3GKZ65NOKWVBBNRB6A, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:55.784307815 (417h33m16.276740265s) [noise=true , address=77.90.31.10:4518 , piece_id=CCDYRFIWKABBJPOCBWPIGEQQYSHNBLKEOQOPXQE5OFSGT6VXCEGQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 139.59.144.22:51944->77.90.31.10:4518: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784436862 (417h33m16.276869296s) [noise=true , address=77.90.31.10:4520 , piece_id=RZNYFFC4AVVFRKDRKVAVAPX2JNUNIMOCWCVG3L3LETHT4HZEF4CQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784539779 (417h33m16.276972222s) [noise=true , address=77.90.31.17:4772 , piece_id=76ORLKXHG3WEWRSLLOHQW3RRGVU5GYG3IEJGHNMWLMPZYS7MSFUQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784719697 (417h33m16.277152133s) [noise=true , address=77.90.31.12:4592 , piece_id=CFBKUEALXCLOEQLPUVBOK4RHCFXFJV5XZHEIPOTFBZIKGLKXYP2A, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784747558 (417h33m16.27718s) [noise=true , address=77.90.31.17:4771 , piece_id=OOUSAWV2GEVAIQ4K7C3EWFX6SO3QM7O4WFY6QVXI5FB4OZ2KGYUQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:52324->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784785329 (417h33m16.277217765s) [noise=true , address=77.90.31.21:4879 , piece_id=WQDIRDNI4CFMT75KZ3TNQY5O6K5UGNTEUQUD3KM6PGLXAB2KJRVQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784818442 (417h33m16.277250887s) [seg_pos={0 0}, upload=367243]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 15 23:40:55.900501127 (417h33m16.392933562s) [upload=367243]: Done uploading segments... Sep 15 23:40:55.900528269 (417h33m16.392960711s) [upload=367243]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.515346835s core uplink us-select-1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 360.208583ms FAIL (2025-09-16 00:17:34 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout Duration: 360.325775ms edge authservice http us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 2 out of 60 is failed (threshold=3) [....XX......................................................] duration: 323.522333ms target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-09-16 00:17:30 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": dial tcp 45.38.24.2:443: i/o timeout Duration: 323.551626ms edge gateway vhost us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...........XX...............................................] duration: 3.165104131s FAIL (2025-09-16 00:17:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.165195845s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [XXX.........................................................] duration: 2.890130641s FAIL (2025-09-16 00:17:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 15 23:40:26.070475751 (417h32m46.562908204s) [upload=367243]: Starting upload Sep 15 23:40:26.070566142 (417h32m46.562998574s) [upload=367243]: Done starting upload Sep 15 23:40:26.070793734 (417h32m46.563226168s) [upload=367243]: Uploading segments... Sep 15 23:40:26.071016487 (417h32m46.563448921s) [upload=367243]: Got next segment. Inline: false Sep 15 23:40:26.071063323 (417h32m46.563495775s) [upload=367243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 15 23:40:26.071076339 (417h32m46.563508792s) [upload=367243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 15 23:40:26.184942850 (417h32m46.677375304s) [seg_pos={0 0}, upload=367243]: Begin upload segment... Sep 15 23:40:26.185028069 (417h32m46.677460517s) [seg_pos={0 0}, upload=367243]: Done begin upload segment. Sep 15 23:40:26.185068989 (417h32m46.677501441s) [noise=true , address=45.91.116.10:4083 , piece_id=6QPRCJSSBIIVFNULJGE2BSONQMHL4ZSBXMEXMISXVV2KCVHJBX3A, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.185724240 (417h32m46.678156692s) [noise=true , address=77.90.31.10:4518 , piece_id=CCDYRFIWKABBJPOCBWPIGEQQYSHNBLKEOQOPXQE5OFSGT6VXCEGQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.186057058 (417h32m46.678489511s) [noise=true , address=45.91.116.8:4048 , piece_id=BYJVXDVFY52FDQYJ7NWQCQ3KASG5ZLSTO66DNYZGT3SPYGXC2YVA, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.186314850 (417h32m46.678747303s) [noise=true , address=45.91.116.7:4009 , piece_id=FJUXJA3SCEHKZQ3TF7XAHBONPENSHA2EZM5VPOUWT5AF5FFAWBXA, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.186491297 (417h32m46.678923748s) [noise=true , address=145.79.227.15:4873 , piece_id=WDMWNGRHW3BXXT54GZOIVGD3EICENKQ2PHNV3LMXWZIBNCOUM2VQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.186897946 (417h32m46.679330395s) [noise=true , address=77.90.31.12:4592 , piece_id=CFBKUEALXCLOEQLPUVBOK4RHCFXFJV5XZHEIPOTFBZIKGLKXYP2A, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.186916513 (417h32m46.679348962s) [noise=true , address=145.79.227.11:4730 , piece_id=JNN7RLZF3OTHAFVIYPFUTOEDREQ3CNN5LXIUFGAD4XNKQEUQIWRQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.187216141 (417h32m46.679648585s) [noise=true , address=45.91.116.8:4047 , piece_id=FTL6SUC6QJGQL6FSE7QVDQOOOBMA6HO2URKVQBQD5WW6ON5LM6CA, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.187410059 (417h32m46.67984251s) [noise=true , address=45.91.116.6:4973 , piece_id=6CEAU6FYIJ3OFB5GR57SVP45CJHKTYCFQIL5YPYQ6CGUCKB53CVA, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.187548554 (417h32m46.679981007s) [noise=true , address=77.90.31.10:4520 , piece_id=RZNYFFC4AVVFRKDRKVAVAPX2JNUNIMOCWCVG3L3LETHT4HZEF4CQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.187891239 (417h32m46.680323691s) [noise=true , address=45.91.116.6:4974 , piece_id=WJL5MLTVRLGWJLYDYOLNXZ2EFOEVOVSMFUYWHKOA3TTWZLKWQUSA, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.187930010 (417h32m46.680362462s) [noise=true , address=77.90.31.21:4879 , piece_id=WQDIRDNI4CFMT75KZ3TNQY5O6K5UGNTEUQUD3KM6PGLXAB2KJRVQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.188300671 (417h32m46.680733122s) [noise=true , address=45.91.116.11:4118 , piece_id=BBDOSLC5DCETT7MVATBJH6MGBEJ3J5GV7I5EU3X7F4Y7L7A7N3WQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.188597680 (417h32m46.681030122s) [noise=true , address=77.90.31.17:4771 , piece_id=OOUSAWV2GEVAIQ4K7C3EWFX6SO3QM7O4WFY6QVXI5FB4OZ2KGYUQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.188939904 (417h32m46.681372353s) [noise=true , address=145.79.227.4:4185 , piece_id=IXUXFHQSDIBJ3K3LR26ERNKEA2QXZVU5QH3GKZ65NOKWVBBNRB6A, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.189187119 (417h32m46.681619564s) [noise=true , address=77.90.31.17:4772 , piece_id=76ORLKXHG3WEWRSLLOHQW3RRGVU5GYG3IEJGHNMWLMPZYS7MSFUQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.808170182 (417h32m47.300602626s) [upload=367243]: Next returned nil segment Sep 15 23:40:26.808216420 (417h32m47.300648865s) [upload=367243]: Waiting for error group managing segments... Sep 15 23:40:26.946199863 (417h32m47.438632293s) [noise=true , address=45.91.116.11:4118 , piece_id=BBDOSLC5DCETT7MVATBJH6MGBEJ3J5GV7I5EU3X7F4Y7L7A7N3WQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:26.946330956 (417h32m47.438763403s) [noise=true , address=45.91.116.7:4009 , piece_id=FJUXJA3SCEHKZQ3TF7XAHBONPENSHA2EZM5VPOUWT5AF5FFAWBXA, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:26.958202290 (417h32m47.450634723s) [noise=true , address=45.91.116.6:4973 , piece_id=6CEAU6FYIJ3OFB5GR57SVP45CJHKTYCFQIL5YPYQ6CGUCKB53CVA, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:26.961831650 (417h32m47.454264082s) [noise=true , address=45.91.116.10:4083 , piece_id=6QPRCJSSBIIVFNULJGE2BSONQMHL4ZSBXMEXMISXVV2KCVHJBX3A, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:26.962849577 (417h32m47.455282006s) [noise=true , address=45.91.116.6:4974 , piece_id=WJL5MLTVRLGWJLYDYOLNXZ2EFOEVOVSMFUYWHKOA3TTWZLKWQUSA, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:26.969890777 (417h32m47.46232321s) [noise=true , address=45.91.116.8:4048 , piece_id=BYJVXDVFY52FDQYJ7NWQCQ3KASG5ZLSTO66DNYZGT3SPYGXC2YVA, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:27.005327018 (417h32m47.497759469s) [noise=true , address=45.91.116.8:4047 , piece_id=FTL6SUC6QJGQL6FSE7QVDQOOOBMA6HO2URKVQBQD5WW6ON5LM6CA, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:27.050024747 (417h32m47.542457189s) [noise=true , address=145.79.227.15:4873 , piece_id=WDMWNGRHW3BXXT54GZOIVGD3EICENKQ2PHNV3LMXWZIBNCOUM2VQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:27.065442548 (417h32m47.557874977s) [noise=true , address=145.79.227.11:4730 , piece_id=JNN7RLZF3OTHAFVIYPFUTOEDREQ3CNN5LXIUFGAD4XNKQEUQIWRQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:27.166046867 (417h32m47.658479304s) [noise=true , address=145.79.227.4:4185 , piece_id=IXUXFHQSDIBJ3K3LR26ERNKEA2QXZVU5QH3GKZ65NOKWVBBNRB6A, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:55.784307815 (417h33m16.276740265s) [noise=true , address=77.90.31.10:4518 , piece_id=CCDYRFIWKABBJPOCBWPIGEQQYSHNBLKEOQOPXQE5OFSGT6VXCEGQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 139.59.144.22:51944->77.90.31.10:4518: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784436862 (417h33m16.276869296s) [noise=true , address=77.90.31.10:4520 , piece_id=RZNYFFC4AVVFRKDRKVAVAPX2JNUNIMOCWCVG3L3LETHT4HZEF4CQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784539779 (417h33m16.276972222s) [noise=true , address=77.90.31.17:4772 , piece_id=76ORLKXHG3WEWRSLLOHQW3RRGVU5GYG3IEJGHNMWLMPZYS7MSFUQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784719697 (417h33m16.277152133s) [noise=true , address=77.90.31.12:4592 , piece_id=CFBKUEALXCLOEQLPUVBOK4RHCFXFJV5XZHEIPOTFBZIKGLKXYP2A, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784747558 (417h33m16.27718s) [noise=true , address=77.90.31.17:4771 , piece_id=OOUSAWV2GEVAIQ4K7C3EWFX6SO3QM7O4WFY6QVXI5FB4OZ2KGYUQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:52324->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784785329 (417h33m16.277217765s) [noise=true , address=77.90.31.21:4879 , piece_id=WQDIRDNI4CFMT75KZ3TNQY5O6K5UGNTEUQUD3KM6PGLXAB2KJRVQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784818442 (417h33m16.277250887s) [seg_pos={0 0}, upload=367243]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 15 23:40:55.900501127 (417h33m16.392933562s) [upload=367243]: Done uploading segments... Sep 15 23:40:55.900528269 (417h33m16.392960711s) [upload=367243]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.890166311s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [............XX..............................................] duration: 2.700009565s FAIL (2025-09-16 00:16:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.700038951s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 2 out of 60 is failed (threshold=3) [.....XX.....................................................] duration: 318.97575ms target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-09-16 00:16:51 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": dial tcp 45.38.24.2:443: i/o timeout Duration: 319.073053ms edge gateway vhost us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 1.248268334s FAIL (2025-09-16 00:16:46 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 1.24829952s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.XXX........................................................] duration: 3.173950384s FAIL (2025-09-16 00:16:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 15 23:40:26.070475751 (417h32m46.562908204s) [upload=367243]: Starting upload Sep 15 23:40:26.070566142 (417h32m46.562998574s) [upload=367243]: Done starting upload Sep 15 23:40:26.070793734 (417h32m46.563226168s) [upload=367243]: Uploading segments... Sep 15 23:40:26.071016487 (417h32m46.563448921s) [upload=367243]: Got next segment. Inline: false Sep 15 23:40:26.071063323 (417h32m46.563495775s) [upload=367243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 15 23:40:26.071076339 (417h32m46.563508792s) [upload=367243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 15 23:40:26.184942850 (417h32m46.677375304s) [seg_pos={0 0}, upload=367243]: Begin upload segment... Sep 15 23:40:26.185028069 (417h32m46.677460517s) [seg_pos={0 0}, upload=367243]: Done begin upload segment. Sep 15 23:40:26.185068989 (417h32m46.677501441s) [noise=true , address=45.91.116.10:4083 , piece_id=6QPRCJSSBIIVFNULJGE2BSONQMHL4ZSBXMEXMISXVV2KCVHJBX3A, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.185724240 (417h32m46.678156692s) [noise=true , address=77.90.31.10:4518 , piece_id=CCDYRFIWKABBJPOCBWPIGEQQYSHNBLKEOQOPXQE5OFSGT6VXCEGQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.186057058 (417h32m46.678489511s) [noise=true , address=45.91.116.8:4048 , piece_id=BYJVXDVFY52FDQYJ7NWQCQ3KASG5ZLSTO66DNYZGT3SPYGXC2YVA, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.186314850 (417h32m46.678747303s) [noise=true , address=45.91.116.7:4009 , piece_id=FJUXJA3SCEHKZQ3TF7XAHBONPENSHA2EZM5VPOUWT5AF5FFAWBXA, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.186491297 (417h32m46.678923748s) [noise=true , address=145.79.227.15:4873 , piece_id=WDMWNGRHW3BXXT54GZOIVGD3EICENKQ2PHNV3LMXWZIBNCOUM2VQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.186897946 (417h32m46.679330395s) [noise=true , address=77.90.31.12:4592 , piece_id=CFBKUEALXCLOEQLPUVBOK4RHCFXFJV5XZHEIPOTFBZIKGLKXYP2A, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.186916513 (417h32m46.679348962s) [noise=true , address=145.79.227.11:4730 , piece_id=JNN7RLZF3OTHAFVIYPFUTOEDREQ3CNN5LXIUFGAD4XNKQEUQIWRQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.187216141 (417h32m46.679648585s) [noise=true , address=45.91.116.8:4047 , piece_id=FTL6SUC6QJGQL6FSE7QVDQOOOBMA6HO2URKVQBQD5WW6ON5LM6CA, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.187410059 (417h32m46.67984251s) [noise=true , address=45.91.116.6:4973 , piece_id=6CEAU6FYIJ3OFB5GR57SVP45CJHKTYCFQIL5YPYQ6CGUCKB53CVA, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.187548554 (417h32m46.679981007s) [noise=true , address=77.90.31.10:4520 , piece_id=RZNYFFC4AVVFRKDRKVAVAPX2JNUNIMOCWCVG3L3LETHT4HZEF4CQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.187891239 (417h32m46.680323691s) [noise=true , address=45.91.116.6:4974 , piece_id=WJL5MLTVRLGWJLYDYOLNXZ2EFOEVOVSMFUYWHKOA3TTWZLKWQUSA, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.187930010 (417h32m46.680362462s) [noise=true , address=77.90.31.21:4879 , piece_id=WQDIRDNI4CFMT75KZ3TNQY5O6K5UGNTEUQUD3KM6PGLXAB2KJRVQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.188300671 (417h32m46.680733122s) [noise=true , address=45.91.116.11:4118 , piece_id=BBDOSLC5DCETT7MVATBJH6MGBEJ3J5GV7I5EU3X7F4Y7L7A7N3WQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.188597680 (417h32m46.681030122s) [noise=true , address=77.90.31.17:4771 , piece_id=OOUSAWV2GEVAIQ4K7C3EWFX6SO3QM7O4WFY6QVXI5FB4OZ2KGYUQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.188939904 (417h32m46.681372353s) [noise=true , address=145.79.227.4:4185 , piece_id=IXUXFHQSDIBJ3K3LR26ERNKEA2QXZVU5QH3GKZ65NOKWVBBNRB6A, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.189187119 (417h32m46.681619564s) [noise=true , address=77.90.31.17:4772 , piece_id=76ORLKXHG3WEWRSLLOHQW3RRGVU5GYG3IEJGHNMWLMPZYS7MSFUQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.808170182 (417h32m47.300602626s) [upload=367243]: Next returned nil segment Sep 15 23:40:26.808216420 (417h32m47.300648865s) [upload=367243]: Waiting for error group managing segments... Sep 15 23:40:26.946199863 (417h32m47.438632293s) [noise=true , address=45.91.116.11:4118 , piece_id=BBDOSLC5DCETT7MVATBJH6MGBEJ3J5GV7I5EU3X7F4Y7L7A7N3WQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:26.946330956 (417h32m47.438763403s) [noise=true , address=45.91.116.7:4009 , piece_id=FJUXJA3SCEHKZQ3TF7XAHBONPENSHA2EZM5VPOUWT5AF5FFAWBXA, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:26.958202290 (417h32m47.450634723s) [noise=true , address=45.91.116.6:4973 , piece_id=6CEAU6FYIJ3OFB5GR57SVP45CJHKTYCFQIL5YPYQ6CGUCKB53CVA, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:26.961831650 (417h32m47.454264082s) [noise=true , address=45.91.116.10:4083 , piece_id=6QPRCJSSBIIVFNULJGE2BSONQMHL4ZSBXMEXMISXVV2KCVHJBX3A, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:26.962849577 (417h32m47.455282006s) [noise=true , address=45.91.116.6:4974 , piece_id=WJL5MLTVRLGWJLYDYOLNXZ2EFOEVOVSMFUYWHKOA3TTWZLKWQUSA, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:26.969890777 (417h32m47.46232321s) [noise=true , address=45.91.116.8:4048 , piece_id=BYJVXDVFY52FDQYJ7NWQCQ3KASG5ZLSTO66DNYZGT3SPYGXC2YVA, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:27.005327018 (417h32m47.497759469s) [noise=true , address=45.91.116.8:4047 , piece_id=FTL6SUC6QJGQL6FSE7QVDQOOOBMA6HO2URKVQBQD5WW6ON5LM6CA, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:27.050024747 (417h32m47.542457189s) [noise=true , address=145.79.227.15:4873 , piece_id=WDMWNGRHW3BXXT54GZOIVGD3EICENKQ2PHNV3LMXWZIBNCOUM2VQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:27.065442548 (417h32m47.557874977s) [noise=true , address=145.79.227.11:4730 , piece_id=JNN7RLZF3OTHAFVIYPFUTOEDREQ3CNN5LXIUFGAD4XNKQEUQIWRQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:27.166046867 (417h32m47.658479304s) [noise=true , address=145.79.227.4:4185 , piece_id=IXUXFHQSDIBJ3K3LR26ERNKEA2QXZVU5QH3GKZ65NOKWVBBNRB6A, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:55.784307815 (417h33m16.276740265s) [noise=true , address=77.90.31.10:4518 , piece_id=CCDYRFIWKABBJPOCBWPIGEQQYSHNBLKEOQOPXQE5OFSGT6VXCEGQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 139.59.144.22:51944->77.90.31.10:4518: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784436862 (417h33m16.276869296s) [noise=true , address=77.90.31.10:4520 , piece_id=RZNYFFC4AVVFRKDRKVAVAPX2JNUNIMOCWCVG3L3LETHT4HZEF4CQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784539779 (417h33m16.276972222s) [noise=true , address=77.90.31.17:4772 , piece_id=76ORLKXHG3WEWRSLLOHQW3RRGVU5GYG3IEJGHNMWLMPZYS7MSFUQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784719697 (417h33m16.277152133s) [noise=true , address=77.90.31.12:4592 , piece_id=CFBKUEALXCLOEQLPUVBOK4RHCFXFJV5XZHEIPOTFBZIKGLKXYP2A, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784747558 (417h33m16.27718s) [noise=true , address=77.90.31.17:4771 , piece_id=OOUSAWV2GEVAIQ4K7C3EWFX6SO3QM7O4WFY6QVXI5FB4OZ2KGYUQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:52324->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784785329 (417h33m16.277217765s) [noise=true , address=77.90.31.21:4879 , piece_id=WQDIRDNI4CFMT75KZ3TNQY5O6K5UGNTEUQUD3KM6PGLXAB2KJRVQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784818442 (417h33m16.277250887s) [seg_pos={0 0}, upload=367243]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 15 23:40:55.900501127 (417h33m16.392933562s) [upload=367243]: Done uploading segments... Sep 15 23:40:55.900528269 (417h33m16.392960711s) [upload=367243]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 3.173987353s core uplink us-select-1
checkers.HTTP{"GET" "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt", ExpectedStatus: 308, ExpectedRedirect: "https://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt"} failure (antiblip window 3/60): (http.us_select_1.linksharing) antiblip: 2 out of 60 is failed (threshold=3) [.........XX.................................................] duration: 244.278455ms target: http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt FAIL (2025-09-16 00:16:41 UTC) orbiter warning: Get "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt": dial tcp 45.38.24.2:80: i/o timeout Duration: 244.301577ms edge linksharing us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 2.8272058s FAIL (2025-09-16 00:16:41 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 2.827231951s edge gateway us-select-1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 361.607381ms FAIL (2025-09-16 00:16:35 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout Duration: 361.640018ms edge authservice http us-select-1
checkers.HTTP{"GET" "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt", ExpectedStatus: 308, ExpectedRedirect: "https://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt"} failure (antiblip window 3/60): (http.us_select_1.linksharing) antiblip: 2 out of 60 is failed (threshold=3) [..........XX................................................] duration: 213.121119ms target: http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt FAIL (2025-09-16 00:16:24 UTC) orbiter warning: Get "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt": dial tcp 45.38.24.2:80: i/o timeout Duration: 213.153993ms edge linksharing us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 2 out of 60 is failed (threshold=3) [......XX....................................................] duration: 324.837033ms target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-09-16 00:16:18 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": dial tcp 45.38.24.2:443: i/o timeout Duration: 324.94053ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..XXX.......................................................] duration: 2.544341597s FAIL (2025-09-16 00:15:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 15 23:40:26.070475751 (417h32m46.562908204s) [upload=367243]: Starting upload Sep 15 23:40:26.070566142 (417h32m46.562998574s) [upload=367243]: Done starting upload Sep 15 23:40:26.070793734 (417h32m46.563226168s) [upload=367243]: Uploading segments... Sep 15 23:40:26.071016487 (417h32m46.563448921s) [upload=367243]: Got next segment. Inline: false Sep 15 23:40:26.071063323 (417h32m46.563495775s) [upload=367243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 15 23:40:26.071076339 (417h32m46.563508792s) [upload=367243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 15 23:40:26.184942850 (417h32m46.677375304s) [seg_pos={0 0}, upload=367243]: Begin upload segment... Sep 15 23:40:26.185028069 (417h32m46.677460517s) [seg_pos={0 0}, upload=367243]: Done begin upload segment. Sep 15 23:40:26.185068989 (417h32m46.677501441s) [noise=true , address=45.91.116.10:4083 , piece_id=6QPRCJSSBIIVFNULJGE2BSONQMHL4ZSBXMEXMISXVV2KCVHJBX3A, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.185724240 (417h32m46.678156692s) [noise=true , address=77.90.31.10:4518 , piece_id=CCDYRFIWKABBJPOCBWPIGEQQYSHNBLKEOQOPXQE5OFSGT6VXCEGQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.186057058 (417h32m46.678489511s) [noise=true , address=45.91.116.8:4048 , piece_id=BYJVXDVFY52FDQYJ7NWQCQ3KASG5ZLSTO66DNYZGT3SPYGXC2YVA, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.186314850 (417h32m46.678747303s) [noise=true , address=45.91.116.7:4009 , piece_id=FJUXJA3SCEHKZQ3TF7XAHBONPENSHA2EZM5VPOUWT5AF5FFAWBXA, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.186491297 (417h32m46.678923748s) [noise=true , address=145.79.227.15:4873 , piece_id=WDMWNGRHW3BXXT54GZOIVGD3EICENKQ2PHNV3LMXWZIBNCOUM2VQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.186897946 (417h32m46.679330395s) [noise=true , address=77.90.31.12:4592 , piece_id=CFBKUEALXCLOEQLPUVBOK4RHCFXFJV5XZHEIPOTFBZIKGLKXYP2A, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.186916513 (417h32m46.679348962s) [noise=true , address=145.79.227.11:4730 , piece_id=JNN7RLZF3OTHAFVIYPFUTOEDREQ3CNN5LXIUFGAD4XNKQEUQIWRQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.187216141 (417h32m46.679648585s) [noise=true , address=45.91.116.8:4047 , piece_id=FTL6SUC6QJGQL6FSE7QVDQOOOBMA6HO2URKVQBQD5WW6ON5LM6CA, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.187410059 (417h32m46.67984251s) [noise=true , address=45.91.116.6:4973 , piece_id=6CEAU6FYIJ3OFB5GR57SVP45CJHKTYCFQIL5YPYQ6CGUCKB53CVA, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.187548554 (417h32m46.679981007s) [noise=true , address=77.90.31.10:4520 , piece_id=RZNYFFC4AVVFRKDRKVAVAPX2JNUNIMOCWCVG3L3LETHT4HZEF4CQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.187891239 (417h32m46.680323691s) [noise=true , address=45.91.116.6:4974 , piece_id=WJL5MLTVRLGWJLYDYOLNXZ2EFOEVOVSMFUYWHKOA3TTWZLKWQUSA, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.187930010 (417h32m46.680362462s) [noise=true , address=77.90.31.21:4879 , piece_id=WQDIRDNI4CFMT75KZ3TNQY5O6K5UGNTEUQUD3KM6PGLXAB2KJRVQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.188300671 (417h32m46.680733122s) [noise=true , address=45.91.116.11:4118 , piece_id=BBDOSLC5DCETT7MVATBJH6MGBEJ3J5GV7I5EU3X7F4Y7L7A7N3WQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.188597680 (417h32m46.681030122s) [noise=true , address=77.90.31.17:4771 , piece_id=OOUSAWV2GEVAIQ4K7C3EWFX6SO3QM7O4WFY6QVXI5FB4OZ2KGYUQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.188939904 (417h32m46.681372353s) [noise=true , address=145.79.227.4:4185 , piece_id=IXUXFHQSDIBJ3K3LR26ERNKEA2QXZVU5QH3GKZ65NOKWVBBNRB6A, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.189187119 (417h32m46.681619564s) [noise=true , address=77.90.31.17:4772 , piece_id=76ORLKXHG3WEWRSLLOHQW3RRGVU5GYG3IEJGHNMWLMPZYS7MSFUQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.808170182 (417h32m47.300602626s) [upload=367243]: Next returned nil segment Sep 15 23:40:26.808216420 (417h32m47.300648865s) [upload=367243]: Waiting for error group managing segments... Sep 15 23:40:26.946199863 (417h32m47.438632293s) [noise=true , address=45.91.116.11:4118 , piece_id=BBDOSLC5DCETT7MVATBJH6MGBEJ3J5GV7I5EU3X7F4Y7L7A7N3WQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:26.946330956 (417h32m47.438763403s) [noise=true , address=45.91.116.7:4009 , piece_id=FJUXJA3SCEHKZQ3TF7XAHBONPENSHA2EZM5VPOUWT5AF5FFAWBXA, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:26.958202290 (417h32m47.450634723s) [noise=true , address=45.91.116.6:4973 , piece_id=6CEAU6FYIJ3OFB5GR57SVP45CJHKTYCFQIL5YPYQ6CGUCKB53CVA, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:26.961831650 (417h32m47.454264082s) [noise=true , address=45.91.116.10:4083 , piece_id=6QPRCJSSBIIVFNULJGE2BSONQMHL4ZSBXMEXMISXVV2KCVHJBX3A, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:26.962849577 (417h32m47.455282006s) [noise=true , address=45.91.116.6:4974 , piece_id=WJL5MLTVRLGWJLYDYOLNXZ2EFOEVOVSMFUYWHKOA3TTWZLKWQUSA, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:26.969890777 (417h32m47.46232321s) [noise=true , address=45.91.116.8:4048 , piece_id=BYJVXDVFY52FDQYJ7NWQCQ3KASG5ZLSTO66DNYZGT3SPYGXC2YVA, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:27.005327018 (417h32m47.497759469s) [noise=true , address=45.91.116.8:4047 , piece_id=FTL6SUC6QJGQL6FSE7QVDQOOOBMA6HO2URKVQBQD5WW6ON5LM6CA, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:27.050024747 (417h32m47.542457189s) [noise=true , address=145.79.227.15:4873 , piece_id=WDMWNGRHW3BXXT54GZOIVGD3EICENKQ2PHNV3LMXWZIBNCOUM2VQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:27.065442548 (417h32m47.557874977s) [noise=true , address=145.79.227.11:4730 , piece_id=JNN7RLZF3OTHAFVIYPFUTOEDREQ3CNN5LXIUFGAD4XNKQEUQIWRQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:27.166046867 (417h32m47.658479304s) [noise=true , address=145.79.227.4:4185 , piece_id=IXUXFHQSDIBJ3K3LR26ERNKEA2QXZVU5QH3GKZ65NOKWVBBNRB6A, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:55.784307815 (417h33m16.276740265s) [noise=true , address=77.90.31.10:4518 , piece_id=CCDYRFIWKABBJPOCBWPIGEQQYSHNBLKEOQOPXQE5OFSGT6VXCEGQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 139.59.144.22:51944->77.90.31.10:4518: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784436862 (417h33m16.276869296s) [noise=true , address=77.90.31.10:4520 , piece_id=RZNYFFC4AVVFRKDRKVAVAPX2JNUNIMOCWCVG3L3LETHT4HZEF4CQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784539779 (417h33m16.276972222s) [noise=true , address=77.90.31.17:4772 , piece_id=76ORLKXHG3WEWRSLLOHQW3RRGVU5GYG3IEJGHNMWLMPZYS7MSFUQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784719697 (417h33m16.277152133s) [noise=true , address=77.90.31.12:4592 , piece_id=CFBKUEALXCLOEQLPUVBOK4RHCFXFJV5XZHEIPOTFBZIKGLKXYP2A, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784747558 (417h33m16.27718s) [noise=true , address=77.90.31.17:4771 , piece_id=OOUSAWV2GEVAIQ4K7C3EWFX6SO3QM7O4WFY6QVXI5FB4OZ2KGYUQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:52324->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784785329 (417h33m16.277217765s) [noise=true , address=77.90.31.21:4879 , piece_id=WQDIRDNI4CFMT75KZ3TNQY5O6K5UGNTEUQUD3KM6PGLXAB2KJRVQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784818442 (417h33m16.277250887s) [seg_pos={0 0}, upload=367243]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 15 23:40:55.900501127 (417h33m16.392933562s) [upload=367243]: Done uploading segments... Sep 15 23:40:55.900528269 (417h33m16.392960711s) [upload=367243]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.544373749s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 2.869888403s FAIL (2025-09-16 00:15:58 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 2.869911519s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.............XX.............................................] duration: 3.21228966s FAIL (2025-09-16 00:15:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.212328676s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 1.379935149s FAIL (2025-09-16 00:15:51 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 1.379982153s edge gateway us-select-1
checkers.HTTP{"GET" "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt", ExpectedStatus: 308, ExpectedRedirect: "https://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt"} failure (antiblip window 3/60): (http.us_select_1.linksharing) antiblip: 2 out of 60 is failed (threshold=3) [...........XX...............................................] duration: 204.432922ms target: http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt FAIL (2025-09-16 00:15:48 UTC) orbiter warning: Get "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt": dial tcp 45.38.24.2:80: i/o timeout Duration: 204.461076ms edge linksharing us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...XXX......................................................] duration: 2.575220263s FAIL (2025-09-16 00:15:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 15 23:40:26.070475751 (417h32m46.562908204s) [upload=367243]: Starting upload Sep 15 23:40:26.070566142 (417h32m46.562998574s) [upload=367243]: Done starting upload Sep 15 23:40:26.070793734 (417h32m46.563226168s) [upload=367243]: Uploading segments... Sep 15 23:40:26.071016487 (417h32m46.563448921s) [upload=367243]: Got next segment. Inline: false Sep 15 23:40:26.071063323 (417h32m46.563495775s) [upload=367243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 15 23:40:26.071076339 (417h32m46.563508792s) [upload=367243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 15 23:40:26.184942850 (417h32m46.677375304s) [seg_pos={0 0}, upload=367243]: Begin upload segment... Sep 15 23:40:26.185028069 (417h32m46.677460517s) [seg_pos={0 0}, upload=367243]: Done begin upload segment. Sep 15 23:40:26.185068989 (417h32m46.677501441s) [noise=true , address=45.91.116.10:4083 , piece_id=6QPRCJSSBIIVFNULJGE2BSONQMHL4ZSBXMEXMISXVV2KCVHJBX3A, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.185724240 (417h32m46.678156692s) [noise=true , address=77.90.31.10:4518 , piece_id=CCDYRFIWKABBJPOCBWPIGEQQYSHNBLKEOQOPXQE5OFSGT6VXCEGQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.186057058 (417h32m46.678489511s) [noise=true , address=45.91.116.8:4048 , piece_id=BYJVXDVFY52FDQYJ7NWQCQ3KASG5ZLSTO66DNYZGT3SPYGXC2YVA, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.186314850 (417h32m46.678747303s) [noise=true , address=45.91.116.7:4009 , piece_id=FJUXJA3SCEHKZQ3TF7XAHBONPENSHA2EZM5VPOUWT5AF5FFAWBXA, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.186491297 (417h32m46.678923748s) [noise=true , address=145.79.227.15:4873 , piece_id=WDMWNGRHW3BXXT54GZOIVGD3EICENKQ2PHNV3LMXWZIBNCOUM2VQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.186897946 (417h32m46.679330395s) [noise=true , address=77.90.31.12:4592 , piece_id=CFBKUEALXCLOEQLPUVBOK4RHCFXFJV5XZHEIPOTFBZIKGLKXYP2A, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.186916513 (417h32m46.679348962s) [noise=true , address=145.79.227.11:4730 , piece_id=JNN7RLZF3OTHAFVIYPFUTOEDREQ3CNN5LXIUFGAD4XNKQEUQIWRQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.187216141 (417h32m46.679648585s) [noise=true , address=45.91.116.8:4047 , piece_id=FTL6SUC6QJGQL6FSE7QVDQOOOBMA6HO2URKVQBQD5WW6ON5LM6CA, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.187410059 (417h32m46.67984251s) [noise=true , address=45.91.116.6:4973 , piece_id=6CEAU6FYIJ3OFB5GR57SVP45CJHKTYCFQIL5YPYQ6CGUCKB53CVA, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.187548554 (417h32m46.679981007s) [noise=true , address=77.90.31.10:4520 , piece_id=RZNYFFC4AVVFRKDRKVAVAPX2JNUNIMOCWCVG3L3LETHT4HZEF4CQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.187891239 (417h32m46.680323691s) [noise=true , address=45.91.116.6:4974 , piece_id=WJL5MLTVRLGWJLYDYOLNXZ2EFOEVOVSMFUYWHKOA3TTWZLKWQUSA, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.187930010 (417h32m46.680362462s) [noise=true , address=77.90.31.21:4879 , piece_id=WQDIRDNI4CFMT75KZ3TNQY5O6K5UGNTEUQUD3KM6PGLXAB2KJRVQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.188300671 (417h32m46.680733122s) [noise=true , address=45.91.116.11:4118 , piece_id=BBDOSLC5DCETT7MVATBJH6MGBEJ3J5GV7I5EU3X7F4Y7L7A7N3WQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.188597680 (417h32m46.681030122s) [noise=true , address=77.90.31.17:4771 , piece_id=OOUSAWV2GEVAIQ4K7C3EWFX6SO3QM7O4WFY6QVXI5FB4OZ2KGYUQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.188939904 (417h32m46.681372353s) [noise=true , address=145.79.227.4:4185 , piece_id=IXUXFHQSDIBJ3K3LR26ERNKEA2QXZVU5QH3GKZ65NOKWVBBNRB6A, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.189187119 (417h32m46.681619564s) [noise=true , address=77.90.31.17:4772 , piece_id=76ORLKXHG3WEWRSLLOHQW3RRGVU5GYG3IEJGHNMWLMPZYS7MSFUQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.808170182 (417h32m47.300602626s) [upload=367243]: Next returned nil segment Sep 15 23:40:26.808216420 (417h32m47.300648865s) [upload=367243]: Waiting for error group managing segments... Sep 15 23:40:26.946199863 (417h32m47.438632293s) [noise=true , address=45.91.116.11:4118 , piece_id=BBDOSLC5DCETT7MVATBJH6MGBEJ3J5GV7I5EU3X7F4Y7L7A7N3WQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:26.946330956 (417h32m47.438763403s) [noise=true , address=45.91.116.7:4009 , piece_id=FJUXJA3SCEHKZQ3TF7XAHBONPENSHA2EZM5VPOUWT5AF5FFAWBXA, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:26.958202290 (417h32m47.450634723s) [noise=true , address=45.91.116.6:4973 , piece_id=6CEAU6FYIJ3OFB5GR57SVP45CJHKTYCFQIL5YPYQ6CGUCKB53CVA, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:26.961831650 (417h32m47.454264082s) [noise=true , address=45.91.116.10:4083 , piece_id=6QPRCJSSBIIVFNULJGE2BSONQMHL4ZSBXMEXMISXVV2KCVHJBX3A, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:26.962849577 (417h32m47.455282006s) [noise=true , address=45.91.116.6:4974 , piece_id=WJL5MLTVRLGWJLYDYOLNXZ2EFOEVOVSMFUYWHKOA3TTWZLKWQUSA, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:26.969890777 (417h32m47.46232321s) [noise=true , address=45.91.116.8:4048 , piece_id=BYJVXDVFY52FDQYJ7NWQCQ3KASG5ZLSTO66DNYZGT3SPYGXC2YVA, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:27.005327018 (417h32m47.497759469s) [noise=true , address=45.91.116.8:4047 , piece_id=FTL6SUC6QJGQL6FSE7QVDQOOOBMA6HO2URKVQBQD5WW6ON5LM6CA, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:27.050024747 (417h32m47.542457189s) [noise=true , address=145.79.227.15:4873 , piece_id=WDMWNGRHW3BXXT54GZOIVGD3EICENKQ2PHNV3LMXWZIBNCOUM2VQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:27.065442548 (417h32m47.557874977s) [noise=true , address=145.79.227.11:4730 , piece_id=JNN7RLZF3OTHAFVIYPFUTOEDREQ3CNN5LXIUFGAD4XNKQEUQIWRQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:27.166046867 (417h32m47.658479304s) [noise=true , address=145.79.227.4:4185 , piece_id=IXUXFHQSDIBJ3K3LR26ERNKEA2QXZVU5QH3GKZ65NOKWVBBNRB6A, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:55.784307815 (417h33m16.276740265s) [noise=true , address=77.90.31.10:4518 , piece_id=CCDYRFIWKABBJPOCBWPIGEQQYSHNBLKEOQOPXQE5OFSGT6VXCEGQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 139.59.144.22:51944->77.90.31.10:4518: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784436862 (417h33m16.276869296s) [noise=true , address=77.90.31.10:4520 , piece_id=RZNYFFC4AVVFRKDRKVAVAPX2JNUNIMOCWCVG3L3LETHT4HZEF4CQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784539779 (417h33m16.276972222s) [noise=true , address=77.90.31.17:4772 , piece_id=76ORLKXHG3WEWRSLLOHQW3RRGVU5GYG3IEJGHNMWLMPZYS7MSFUQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784719697 (417h33m16.277152133s) [noise=true , address=77.90.31.12:4592 , piece_id=CFBKUEALXCLOEQLPUVBOK4RHCFXFJV5XZHEIPOTFBZIKGLKXYP2A, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784747558 (417h33m16.27718s) [noise=true , address=77.90.31.17:4771 , piece_id=OOUSAWV2GEVAIQ4K7C3EWFX6SO3QM7O4WFY6QVXI5FB4OZ2KGYUQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:52324->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784785329 (417h33m16.277217765s) [noise=true , address=77.90.31.21:4879 , piece_id=WQDIRDNI4CFMT75KZ3TNQY5O6K5UGNTEUQUD3KM6PGLXAB2KJRVQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784818442 (417h33m16.277250887s) [seg_pos={0 0}, upload=367243]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 15 23:40:55.900501127 (417h33m16.392933562s) [upload=367243]: Done uploading segments... Sep 15 23:40:55.900528269 (417h33m16.392960711s) [upload=367243]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.57525964s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 2 out of 60 is failed (threshold=3) [.......XX...................................................] duration: 519.239122ms target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-09-16 00:15:25 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": dial tcp 45.38.24.2:443: i/o timeout Duration: 519.265866ms edge gateway vhost us-select-1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 372.12173ms FAIL (2025-09-16 00:15:22 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout Duration: 372.15848ms edge authservice http us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..............XX............................................] duration: 3.138507129s FAIL (2025-09-16 00:15:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.138535506s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....XXX.....................................................] duration: 2.649884909s FAIL (2025-09-16 00:15:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 15 23:40:26.070475751 (417h32m46.562908204s) [upload=367243]: Starting upload Sep 15 23:40:26.070566142 (417h32m46.562998574s) [upload=367243]: Done starting upload Sep 15 23:40:26.070793734 (417h32m46.563226168s) [upload=367243]: Uploading segments... Sep 15 23:40:26.071016487 (417h32m46.563448921s) [upload=367243]: Got next segment. Inline: false Sep 15 23:40:26.071063323 (417h32m46.563495775s) [upload=367243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 15 23:40:26.071076339 (417h32m46.563508792s) [upload=367243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 15 23:40:26.184942850 (417h32m46.677375304s) [seg_pos={0 0}, upload=367243]: Begin upload segment... Sep 15 23:40:26.185028069 (417h32m46.677460517s) [seg_pos={0 0}, upload=367243]: Done begin upload segment. Sep 15 23:40:26.185068989 (417h32m46.677501441s) [noise=true , address=45.91.116.10:4083 , piece_id=6QPRCJSSBIIVFNULJGE2BSONQMHL4ZSBXMEXMISXVV2KCVHJBX3A, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.185724240 (417h32m46.678156692s) [noise=true , address=77.90.31.10:4518 , piece_id=CCDYRFIWKABBJPOCBWPIGEQQYSHNBLKEOQOPXQE5OFSGT6VXCEGQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.186057058 (417h32m46.678489511s) [noise=true , address=45.91.116.8:4048 , piece_id=BYJVXDVFY52FDQYJ7NWQCQ3KASG5ZLSTO66DNYZGT3SPYGXC2YVA, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.186314850 (417h32m46.678747303s) [noise=true , address=45.91.116.7:4009 , piece_id=FJUXJA3SCEHKZQ3TF7XAHBONPENSHA2EZM5VPOUWT5AF5FFAWBXA, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.186491297 (417h32m46.678923748s) [noise=true , address=145.79.227.15:4873 , piece_id=WDMWNGRHW3BXXT54GZOIVGD3EICENKQ2PHNV3LMXWZIBNCOUM2VQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.186897946 (417h32m46.679330395s) [noise=true , address=77.90.31.12:4592 , piece_id=CFBKUEALXCLOEQLPUVBOK4RHCFXFJV5XZHEIPOTFBZIKGLKXYP2A, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.186916513 (417h32m46.679348962s) [noise=true , address=145.79.227.11:4730 , piece_id=JNN7RLZF3OTHAFVIYPFUTOEDREQ3CNN5LXIUFGAD4XNKQEUQIWRQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.187216141 (417h32m46.679648585s) [noise=true , address=45.91.116.8:4047 , piece_id=FTL6SUC6QJGQL6FSE7QVDQOOOBMA6HO2URKVQBQD5WW6ON5LM6CA, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.187410059 (417h32m46.67984251s) [noise=true , address=45.91.116.6:4973 , piece_id=6CEAU6FYIJ3OFB5GR57SVP45CJHKTYCFQIL5YPYQ6CGUCKB53CVA, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.187548554 (417h32m46.679981007s) [noise=true , address=77.90.31.10:4520 , piece_id=RZNYFFC4AVVFRKDRKVAVAPX2JNUNIMOCWCVG3L3LETHT4HZEF4CQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.187891239 (417h32m46.680323691s) [noise=true , address=45.91.116.6:4974 , piece_id=WJL5MLTVRLGWJLYDYOLNXZ2EFOEVOVSMFUYWHKOA3TTWZLKWQUSA, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.187930010 (417h32m46.680362462s) [noise=true , address=77.90.31.21:4879 , piece_id=WQDIRDNI4CFMT75KZ3TNQY5O6K5UGNTEUQUD3KM6PGLXAB2KJRVQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.188300671 (417h32m46.680733122s) [noise=true , address=45.91.116.11:4118 , piece_id=BBDOSLC5DCETT7MVATBJH6MGBEJ3J5GV7I5EU3X7F4Y7L7A7N3WQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.188597680 (417h32m46.681030122s) [noise=true , address=77.90.31.17:4771 , piece_id=OOUSAWV2GEVAIQ4K7C3EWFX6SO3QM7O4WFY6QVXI5FB4OZ2KGYUQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.188939904 (417h32m46.681372353s) [noise=true , address=145.79.227.4:4185 , piece_id=IXUXFHQSDIBJ3K3LR26ERNKEA2QXZVU5QH3GKZ65NOKWVBBNRB6A, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.189187119 (417h32m46.681619564s) [noise=true , address=77.90.31.17:4772 , piece_id=76ORLKXHG3WEWRSLLOHQW3RRGVU5GYG3IEJGHNMWLMPZYS7MSFUQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.808170182 (417h32m47.300602626s) [upload=367243]: Next returned nil segment Sep 15 23:40:26.808216420 (417h32m47.300648865s) [upload=367243]: Waiting for error group managing segments... Sep 15 23:40:26.946199863 (417h32m47.438632293s) [noise=true , address=45.91.116.11:4118 , piece_id=BBDOSLC5DCETT7MVATBJH6MGBEJ3J5GV7I5EU3X7F4Y7L7A7N3WQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:26.946330956 (417h32m47.438763403s) [noise=true , address=45.91.116.7:4009 , piece_id=FJUXJA3SCEHKZQ3TF7XAHBONPENSHA2EZM5VPOUWT5AF5FFAWBXA, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:26.958202290 (417h32m47.450634723s) [noise=true , address=45.91.116.6:4973 , piece_id=6CEAU6FYIJ3OFB5GR57SVP45CJHKTYCFQIL5YPYQ6CGUCKB53CVA, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:26.961831650 (417h32m47.454264082s) [noise=true , address=45.91.116.10:4083 , piece_id=6QPRCJSSBIIVFNULJGE2BSONQMHL4ZSBXMEXMISXVV2KCVHJBX3A, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:26.962849577 (417h32m47.455282006s) [noise=true , address=45.91.116.6:4974 , piece_id=WJL5MLTVRLGWJLYDYOLNXZ2EFOEVOVSMFUYWHKOA3TTWZLKWQUSA, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:26.969890777 (417h32m47.46232321s) [noise=true , address=45.91.116.8:4048 , piece_id=BYJVXDVFY52FDQYJ7NWQCQ3KASG5ZLSTO66DNYZGT3SPYGXC2YVA, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:27.005327018 (417h32m47.497759469s) [noise=true , address=45.91.116.8:4047 , piece_id=FTL6SUC6QJGQL6FSE7QVDQOOOBMA6HO2URKVQBQD5WW6ON5LM6CA, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:27.050024747 (417h32m47.542457189s) [noise=true , address=145.79.227.15:4873 , piece_id=WDMWNGRHW3BXXT54GZOIVGD3EICENKQ2PHNV3LMXWZIBNCOUM2VQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:27.065442548 (417h32m47.557874977s) [noise=true , address=145.79.227.11:4730 , piece_id=JNN7RLZF3OTHAFVIYPFUTOEDREQ3CNN5LXIUFGAD4XNKQEUQIWRQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:27.166046867 (417h32m47.658479304s) [noise=true , address=145.79.227.4:4185 , piece_id=IXUXFHQSDIBJ3K3LR26ERNKEA2QXZVU5QH3GKZ65NOKWVBBNRB6A, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:55.784307815 (417h33m16.276740265s) [noise=true , address=77.90.31.10:4518 , piece_id=CCDYRFIWKABBJPOCBWPIGEQQYSHNBLKEOQOPXQE5OFSGT6VXCEGQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 139.59.144.22:51944->77.90.31.10:4518: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784436862 (417h33m16.276869296s) [noise=true , address=77.90.31.10:4520 , piece_id=RZNYFFC4AVVFRKDRKVAVAPX2JNUNIMOCWCVG3L3LETHT4HZEF4CQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784539779 (417h33m16.276972222s) [noise=true , address=77.90.31.17:4772 , piece_id=76ORLKXHG3WEWRSLLOHQW3RRGVU5GYG3IEJGHNMWLMPZYS7MSFUQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784719697 (417h33m16.277152133s) [noise=true , address=77.90.31.12:4592 , piece_id=CFBKUEALXCLOEQLPUVBOK4RHCFXFJV5XZHEIPOTFBZIKGLKXYP2A, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784747558 (417h33m16.27718s) [noise=true , address=77.90.31.17:4771 , piece_id=OOUSAWV2GEVAIQ4K7C3EWFX6SO3QM7O4WFY6QVXI5FB4OZ2KGYUQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:52324->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784785329 (417h33m16.277217765s) [noise=true , address=77.90.31.21:4879 , piece_id=WQDIRDNI4CFMT75KZ3TNQY5O6K5UGNTEUQUD3KM6PGLXAB2KJRVQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784818442 (417h33m16.277250887s) [seg_pos={0 0}, upload=367243]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 15 23:40:55.900501127 (417h33m16.392933562s) [upload=367243]: Done uploading segments... Sep 15 23:40:55.900528269 (417h33m16.392960711s) [upload=367243]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.649950347s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 3.179590259s FAIL (2025-09-16 00:14:58 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 3.179625503s edge gateway us-select-1
checkers.HTTP{"GET" "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt", ExpectedStatus: 308, ExpectedRedirect: "https://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt"} failure (antiblip window 3/60): (http.us_select_1.linksharing) antiblip: 2 out of 60 is failed (threshold=3) [............XX..............................................] duration: 205.48597ms target: http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt FAIL (2025-09-16 00:14:51 UTC) orbiter warning: Get "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt": dial tcp 45.38.24.2:80: i/o timeout Duration: 205.519939ms edge linksharing us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 2 out of 60 is failed (threshold=3) [........XX..................................................] duration: 303.718628ms target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-09-16 00:14:50 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": dial tcp 45.38.24.2:443: i/o timeout Duration: 303.884129ms edge gateway vhost us-select-1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 361.705446ms FAIL (2025-09-16 00:14:49 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout Duration: 361.742597ms edge authservice http us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...............XX...........................................] duration: 2.901526254s FAIL (2025-09-16 00:14:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.901563675s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 1.796804177s FAIL (2025-09-16 00:14:42 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 1.796834528s edge gateway us-select-1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 359.800724ms FAIL (2025-09-16 00:14:28 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout Duration: 359.832198ms edge authservice http us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [................XX..........................................] duration: 2.882649s FAIL (2025-09-16 00:14:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.882694292s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 2.786211259s FAIL (2025-09-16 00:14:24 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 2.786239776s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....XXX....................................................] duration: 2.605031012s FAIL (2025-09-16 00:14:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 15 23:40:26.070475751 (417h32m46.562908204s) [upload=367243]: Starting upload Sep 15 23:40:26.070566142 (417h32m46.562998574s) [upload=367243]: Done starting upload Sep 15 23:40:26.070793734 (417h32m46.563226168s) [upload=367243]: Uploading segments... Sep 15 23:40:26.071016487 (417h32m46.563448921s) [upload=367243]: Got next segment. Inline: false Sep 15 23:40:26.071063323 (417h32m46.563495775s) [upload=367243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 15 23:40:26.071076339 (417h32m46.563508792s) [upload=367243]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 15 23:40:26.184942850 (417h32m46.677375304s) [seg_pos={0 0}, upload=367243]: Begin upload segment... Sep 15 23:40:26.185028069 (417h32m46.677460517s) [seg_pos={0 0}, upload=367243]: Done begin upload segment. Sep 15 23:40:26.185068989 (417h32m46.677501441s) [noise=true , address=45.91.116.10:4083 , piece_id=6QPRCJSSBIIVFNULJGE2BSONQMHL4ZSBXMEXMISXVV2KCVHJBX3A, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.185724240 (417h32m46.678156692s) [noise=true , address=77.90.31.10:4518 , piece_id=CCDYRFIWKABBJPOCBWPIGEQQYSHNBLKEOQOPXQE5OFSGT6VXCEGQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.186057058 (417h32m46.678489511s) [noise=true , address=45.91.116.8:4048 , piece_id=BYJVXDVFY52FDQYJ7NWQCQ3KASG5ZLSTO66DNYZGT3SPYGXC2YVA, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.186314850 (417h32m46.678747303s) [noise=true , address=45.91.116.7:4009 , piece_id=FJUXJA3SCEHKZQ3TF7XAHBONPENSHA2EZM5VPOUWT5AF5FFAWBXA, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.186491297 (417h32m46.678923748s) [noise=true , address=145.79.227.15:4873 , piece_id=WDMWNGRHW3BXXT54GZOIVGD3EICENKQ2PHNV3LMXWZIBNCOUM2VQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.186897946 (417h32m46.679330395s) [noise=true , address=77.90.31.12:4592 , piece_id=CFBKUEALXCLOEQLPUVBOK4RHCFXFJV5XZHEIPOTFBZIKGLKXYP2A, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.186916513 (417h32m46.679348962s) [noise=true , address=145.79.227.11:4730 , piece_id=JNN7RLZF3OTHAFVIYPFUTOEDREQ3CNN5LXIUFGAD4XNKQEUQIWRQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.187216141 (417h32m46.679648585s) [noise=true , address=45.91.116.8:4047 , piece_id=FTL6SUC6QJGQL6FSE7QVDQOOOBMA6HO2URKVQBQD5WW6ON5LM6CA, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.187410059 (417h32m46.67984251s) [noise=true , address=45.91.116.6:4973 , piece_id=6CEAU6FYIJ3OFB5GR57SVP45CJHKTYCFQIL5YPYQ6CGUCKB53CVA, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.187548554 (417h32m46.679981007s) [noise=true , address=77.90.31.10:4520 , piece_id=RZNYFFC4AVVFRKDRKVAVAPX2JNUNIMOCWCVG3L3LETHT4HZEF4CQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.187891239 (417h32m46.680323691s) [noise=true , address=45.91.116.6:4974 , piece_id=WJL5MLTVRLGWJLYDYOLNXZ2EFOEVOVSMFUYWHKOA3TTWZLKWQUSA, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.187930010 (417h32m46.680362462s) [noise=true , address=77.90.31.21:4879 , piece_id=WQDIRDNI4CFMT75KZ3TNQY5O6K5UGNTEUQUD3KM6PGLXAB2KJRVQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.188300671 (417h32m46.680733122s) [noise=true , address=45.91.116.11:4118 , piece_id=BBDOSLC5DCETT7MVATBJH6MGBEJ3J5GV7I5EU3X7F4Y7L7A7N3WQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.188597680 (417h32m46.681030122s) [noise=true , address=77.90.31.17:4771 , piece_id=OOUSAWV2GEVAIQ4K7C3EWFX6SO3QM7O4WFY6QVXI5FB4OZ2KGYUQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.188939904 (417h32m46.681372353s) [noise=true , address=145.79.227.4:4185 , piece_id=IXUXFHQSDIBJ3K3LR26ERNKEA2QXZVU5QH3GKZ65NOKWVBBNRB6A, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.189187119 (417h32m46.681619564s) [noise=true , address=77.90.31.17:4772 , piece_id=76ORLKXHG3WEWRSLLOHQW3RRGVU5GYG3IEJGHNMWLMPZYS7MSFUQ, seg_pos={0 0}, upload=367243]: Uploading piece... Sep 15 23:40:26.808170182 (417h32m47.300602626s) [upload=367243]: Next returned nil segment Sep 15 23:40:26.808216420 (417h32m47.300648865s) [upload=367243]: Waiting for error group managing segments... Sep 15 23:40:26.946199863 (417h32m47.438632293s) [noise=true , address=45.91.116.11:4118 , piece_id=BBDOSLC5DCETT7MVATBJH6MGBEJ3J5GV7I5EU3X7F4Y7L7A7N3WQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:26.946330956 (417h32m47.438763403s) [noise=true , address=45.91.116.7:4009 , piece_id=FJUXJA3SCEHKZQ3TF7XAHBONPENSHA2EZM5VPOUWT5AF5FFAWBXA, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:26.958202290 (417h32m47.450634723s) [noise=true , address=45.91.116.6:4973 , piece_id=6CEAU6FYIJ3OFB5GR57SVP45CJHKTYCFQIL5YPYQ6CGUCKB53CVA, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:26.961831650 (417h32m47.454264082s) [noise=true , address=45.91.116.10:4083 , piece_id=6QPRCJSSBIIVFNULJGE2BSONQMHL4ZSBXMEXMISXVV2KCVHJBX3A, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:26.962849577 (417h32m47.455282006s) [noise=true , address=45.91.116.6:4974 , piece_id=WJL5MLTVRLGWJLYDYOLNXZ2EFOEVOVSMFUYWHKOA3TTWZLKWQUSA, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:26.969890777 (417h32m47.46232321s) [noise=true , address=45.91.116.8:4048 , piece_id=BYJVXDVFY52FDQYJ7NWQCQ3KASG5ZLSTO66DNYZGT3SPYGXC2YVA, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:27.005327018 (417h32m47.497759469s) [noise=true , address=45.91.116.8:4047 , piece_id=FTL6SUC6QJGQL6FSE7QVDQOOOBMA6HO2URKVQBQD5WW6ON5LM6CA, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:27.050024747 (417h32m47.542457189s) [noise=true , address=145.79.227.15:4873 , piece_id=WDMWNGRHW3BXXT54GZOIVGD3EICENKQ2PHNV3LMXWZIBNCOUM2VQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:27.065442548 (417h32m47.557874977s) [noise=true , address=145.79.227.11:4730 , piece_id=JNN7RLZF3OTHAFVIYPFUTOEDREQ3CNN5LXIUFGAD4XNKQEUQIWRQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:27.166046867 (417h32m47.658479304s) [noise=true , address=145.79.227.4:4185 , piece_id=IXUXFHQSDIBJ3K3LR26ERNKEA2QXZVU5QH3GKZ65NOKWVBBNRB6A, seg_pos={0 0}, upload=367243]: Done uploading piece. err: <nil> Sep 15 23:40:55.784307815 (417h33m16.276740265s) [noise=true , address=77.90.31.10:4518 , piece_id=CCDYRFIWKABBJPOCBWPIGEQQYSHNBLKEOQOPXQE5OFSGT6VXCEGQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 139.59.144.22:51944->77.90.31.10:4518: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784436862 (417h33m16.276869296s) [noise=true , address=77.90.31.10:4520 , piece_id=RZNYFFC4AVVFRKDRKVAVAPX2JNUNIMOCWCVG3L3LETHT4HZEF4CQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784539779 (417h33m16.276972222s) [noise=true , address=77.90.31.17:4772 , piece_id=76ORLKXHG3WEWRSLLOHQW3RRGVU5GYG3IEJGHNMWLMPZYS7MSFUQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784719697 (417h33m16.277152133s) [noise=true , address=77.90.31.12:4592 , piece_id=CFBKUEALXCLOEQLPUVBOK4RHCFXFJV5XZHEIPOTFBZIKGLKXYP2A, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784747558 (417h33m16.27718s) [noise=true , address=77.90.31.17:4771 , piece_id=OOUSAWV2GEVAIQ4K7C3EWFX6SO3QM7O4WFY6QVXI5FB4OZ2KGYUQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:52324->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784785329 (417h33m16.277217765s) [noise=true , address=77.90.31.21:4879 , piece_id=WQDIRDNI4CFMT75KZ3TNQY5O6K5UGNTEUQUD3KM6PGLXAB2KJRVQ, seg_pos={0 0}, upload=367243]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 15 23:40:55.784818442 (417h33m16.277250887s) [seg_pos={0 0}, upload=367243]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 15 23:40:55.900501127 (417h33m16.392933562s) [upload=367243]: Done uploading segments... Sep 15 23:40:55.900528269 (417h33m16.392960711s) [upload=367243]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.605077344s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 2 out of 60 is failed (threshold=3) [.........XX.................................................] duration: 310.963142ms target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-09-16 00:14:06 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": dial tcp 45.38.24.2:443: i/o timeout Duration: 310.988121ms edge gateway vhost us-select-1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 336.14708ms FAIL (2025-09-16 00:14:00 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout Duration: 336.18521ms edge authservice http us-select-1

1
...
403
404
405
406
407
...
3470