Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 2.789161181s trace: https://jaeger.dev.storj.io/trace/252d1ffbffb4a9c9 FAIL (2025-12-14 15:52:44 UTC) orbiter warning: read tcp 139.59.144.22:52262->45.38.24.2:443: read: connection reset by peer Duration: 2.789195167s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 2.752549669s trace: https://jaeger.dev.storj.io/trace/252d1ffbffb4a9c9 FAIL (2025-12-14 15:51:32 UTC) orbiter warning: read tcp 139.59.144.22:52262->45.38.24.2:443: read: connection reset by peer Duration: 2.752588766s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 2.800081539s trace: https://jaeger.dev.storj.io/trace/252d1ffbffb4a9c9 FAIL (2025-12-14 15:50:32 UTC) orbiter warning: read tcp 139.59.144.22:52262->45.38.24.2:443: read: connection reset by peer Duration: 2.800108396s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 2.706987472s trace: https://jaeger.dev.storj.io/trace/252d1ffbffb4a9c9 FAIL (2025-12-14 15:49:34 UTC) orbiter warning: read tcp 139.59.144.22:52262->45.38.24.2:443: read: connection reset by peer Duration: 2.707016051s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 2.588253728s trace: https://jaeger.dev.storj.io/trace/252d1ffbffb4a9c9 FAIL (2025-12-14 15:48:45 UTC) orbiter warning: read tcp 139.59.144.22:52262->45.38.24.2:443: read: connection reset by peer Duration: 2.588278134s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 2.766331119s trace: https://jaeger.dev.storj.io/trace/252d1ffbffb4a9c9 FAIL (2025-12-14 15:48:06 UTC) orbiter warning: read tcp 139.59.144.22:52262->45.38.24.2:443: read: connection reset by peer Duration: 2.766717318s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 2.519190117s trace: https://jaeger.dev.storj.io/trace/252d1ffbffb4a9c9 FAIL (2025-12-14 15:47:33 UTC) orbiter warning: read tcp 139.59.144.22:52262->45.38.24.2:443: read: connection reset by peer Duration: 2.519217846s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 2.6339681s trace: https://jaeger.dev.storj.io/trace/252d1ffbffb4a9c9 FAIL (2025-12-14 15:46:27 UTC) orbiter warning: read tcp 139.59.144.22:52262->45.38.24.2:443: read: connection reset by peer Duration: 2.633980778s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 2.603884954s trace: https://jaeger.dev.storj.io/trace/252d1ffbffb4a9c9 FAIL (2025-12-14 15:45:12 UTC) orbiter warning: read tcp 139.59.144.22:52262->45.38.24.2:443: read: connection reset by peer Duration: 2.603913838s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 2.740533693s trace: https://jaeger.dev.storj.io/trace/252d1ffbffb4a9c9 FAIL (2025-12-14 15:44:29 UTC) orbiter warning: read tcp 139.59.144.22:52262->45.38.24.2:443: read: connection reset by peer Duration: 2.740566084s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 2.667695553s trace: https://jaeger.dev.storj.io/trace/252d1ffbffb4a9c9 FAIL (2025-12-14 15:44:11 UTC) orbiter warning: read tcp 139.59.144.22:52262->45.38.24.2:443: read: connection reset by peer Duration: 2.667720941s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 2.943764104s trace: https://jaeger.dev.storj.io/trace/252d1ffbffb4a9c9 FAIL (2025-12-14 15:43:34 UTC) orbiter warning: read tcp 139.59.144.22:52262->45.38.24.2:443: read: connection reset by peer Duration: 2.943793041s edge gateway us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [X..............] duration: 157.79684ms remote: "tcp:[2620:cb:2000::1]:443" response headers: Content-Type=text/plain; charset=UTF-8 Content-Length=15 Cache-Control=private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Expires=Thu, 01 Jan 1970 00:00:01 GMT Cf-Ray=9ade7e5d7aae9963-FRA X-Frame-Options=SAMEORIGIN Date=Sun, 14 Dec 2025 14:46:56 GMT Referrer-Policy=same-origin Proxy-Status=Cloudflare-Proxy;error=http_response_incomplete target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/1244b8592b06e8c8 FAIL (2025-12-14 14:57:50 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "error code: 500" Duration: 157.823043ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.421664361s trace: https://jaeger.dev.storj.io/trace/1751ea226e84ac5d FAIL (2025-12-14 14:57:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 14 14:16:03.322622529 (214h47m43.302975166s) [upload=188771]: Starting upload Dec 14 14:16:03.322698383 (214h47m43.303051027s) [upload=188771]: Done starting upload Dec 14 14:16:03.322957650 (214h47m43.303310288s) [upload=188771]: Uploading segments... Dec 14 14:16:03.323047310 (214h47m43.303399971s) [upload=188771]: Got next segment. Inline: false Dec 14 14:16:03.323095962 (214h47m43.30344861s) [upload=188771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 14 14:16:03.323104243 (214h47m43.303456893s) [upload=188771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 14 14:16:03.446699051 (214h47m43.427051694s) [seg_pos={0 0}, upload=188771]: Begin upload segment... Dec 14 14:16:03.446762560 (214h47m43.427115199s) [seg_pos={0 0}, upload=188771]: Done begin upload segment. Dec 14 14:16:03.446782966 (214h47m43.427135626s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446798773 (214h47m43.427151422s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446883480 (214h47m43.427236128s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.447750216 (214h47m43.428102851s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448003133 (214h47m43.428355855s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448960450 (214h47m43.429313088s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.450289219 (214h47m43.43064186s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451155877 (214h47m43.431508525s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451700169 (214h47m43.432052812s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453101406 (214h47m43.433454055s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453306320 (214h47m43.433658971s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453902924 (214h47m43.43425557s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.454485471 (214h47m43.434838121s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455040478 (214h47m43.435393125s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455598935 (214h47m43.435951585s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455867680 (214h47m43.436220329s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.861047945 (214h47m43.841400591s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:03.871023383 (214h47m43.851376028s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:04.080925197 (214h47m44.061277846s) [upload=188771]: Next returned nil segment Dec 14 14:16:04.080944439 (214h47m44.061297107s) [upload=188771]: Waiting for error group managing segments... Dec 14 14:16:04.209819284 (214h47m44.190171943s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.233005644 (214h47m44.213358285s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.234227859 (214h47m44.214580501s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239681369 (214h47m44.22003402s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239894423 (214h47m44.220247063s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.240454771 (214h47m44.220807417s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.241497930 (214h47m44.221850571s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.242469256 (214h47m44.222821901s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.243873927 (214h47m44.224226573s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.244902987 (214h47m44.225255626s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.246138337 (214h47m44.226490985s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.284989325 (214h47m44.265341963s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.787553216 (214h47m44.767905858s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:33.026981756 (214h48m13.007334401s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Dec 14 14:16:33.027035288 (214h48m13.007387915s) [seg_pos={0 0}, upload=188771]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 14 14:16:33.139454175 (214h48m13.119806811s) [upload=188771]: Done uploading segments... Dec 14 14:16:33.139495796 (214h48m13.119848423s) [upload=188771]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.421695352s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 2.31408567s trace: https://jaeger.dev.storj.io/trace/1751ea226e84ac5d FAIL (2025-12-14 14:57:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 14 14:16:03.322622529 (214h47m43.302975166s) [upload=188771]: Starting upload Dec 14 14:16:03.322698383 (214h47m43.303051027s) [upload=188771]: Done starting upload Dec 14 14:16:03.322957650 (214h47m43.303310288s) [upload=188771]: Uploading segments... Dec 14 14:16:03.323047310 (214h47m43.303399971s) [upload=188771]: Got next segment. Inline: false Dec 14 14:16:03.323095962 (214h47m43.30344861s) [upload=188771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 14 14:16:03.323104243 (214h47m43.303456893s) [upload=188771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 14 14:16:03.446699051 (214h47m43.427051694s) [seg_pos={0 0}, upload=188771]: Begin upload segment... Dec 14 14:16:03.446762560 (214h47m43.427115199s) [seg_pos={0 0}, upload=188771]: Done begin upload segment. Dec 14 14:16:03.446782966 (214h47m43.427135626s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446798773 (214h47m43.427151422s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446883480 (214h47m43.427236128s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.447750216 (214h47m43.428102851s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448003133 (214h47m43.428355855s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448960450 (214h47m43.429313088s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.450289219 (214h47m43.43064186s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451155877 (214h47m43.431508525s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451700169 (214h47m43.432052812s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453101406 (214h47m43.433454055s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453306320 (214h47m43.433658971s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453902924 (214h47m43.43425557s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.454485471 (214h47m43.434838121s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455040478 (214h47m43.435393125s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455598935 (214h47m43.435951585s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455867680 (214h47m43.436220329s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.861047945 (214h47m43.841400591s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:03.871023383 (214h47m43.851376028s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:04.080925197 (214h47m44.061277846s) [upload=188771]: Next returned nil segment Dec 14 14:16:04.080944439 (214h47m44.061297107s) [upload=188771]: Waiting for error group managing segments... Dec 14 14:16:04.209819284 (214h47m44.190171943s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.233005644 (214h47m44.213358285s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.234227859 (214h47m44.214580501s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239681369 (214h47m44.22003402s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239894423 (214h47m44.220247063s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.240454771 (214h47m44.220807417s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.241497930 (214h47m44.221850571s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.242469256 (214h47m44.222821901s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.243873927 (214h47m44.224226573s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.244902987 (214h47m44.225255626s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.246138337 (214h47m44.226490985s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.284989325 (214h47m44.265341963s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.787553216 (214h47m44.767905858s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:33.026981756 (214h48m13.007334401s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Dec 14 14:16:33.027035288 (214h48m13.007387915s) [seg_pos={0 0}, upload=188771]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 14 14:16:33.139454175 (214h48m13.119806811s) [upload=188771]: Done uploading segments... Dec 14 14:16:33.139495796 (214h48m13.119848423s) [upload=188771]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.314113918s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.X.............] duration: 157.743129ms remote: "tcp:[2620:cb:2000::1]:443" response headers: Content-Type=text/plain; charset=UTF-8 Content-Length=15 Cache-Control=private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Expires=Thu, 01 Jan 1970 00:00:01 GMT Cf-Ray=9ade7e5d7aae9963-FRA X-Frame-Options=SAMEORIGIN Date=Sun, 14 Dec 2025 14:46:56 GMT Referrer-Policy=same-origin Proxy-Status=Cloudflare-Proxy;error=http_response_incomplete target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/1244b8592b06e8c8 FAIL (2025-12-14 14:57:07 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "error code: 500" Duration: 157.770463ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 2.497859729s trace: https://jaeger.dev.storj.io/trace/1751ea226e84ac5d FAIL (2025-12-14 14:57:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 14 14:16:03.322622529 (214h47m43.302975166s) [upload=188771]: Starting upload Dec 14 14:16:03.322698383 (214h47m43.303051027s) [upload=188771]: Done starting upload Dec 14 14:16:03.322957650 (214h47m43.303310288s) [upload=188771]: Uploading segments... Dec 14 14:16:03.323047310 (214h47m43.303399971s) [upload=188771]: Got next segment. Inline: false Dec 14 14:16:03.323095962 (214h47m43.30344861s) [upload=188771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 14 14:16:03.323104243 (214h47m43.303456893s) [upload=188771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 14 14:16:03.446699051 (214h47m43.427051694s) [seg_pos={0 0}, upload=188771]: Begin upload segment... Dec 14 14:16:03.446762560 (214h47m43.427115199s) [seg_pos={0 0}, upload=188771]: Done begin upload segment. Dec 14 14:16:03.446782966 (214h47m43.427135626s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446798773 (214h47m43.427151422s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446883480 (214h47m43.427236128s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.447750216 (214h47m43.428102851s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448003133 (214h47m43.428355855s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448960450 (214h47m43.429313088s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.450289219 (214h47m43.43064186s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451155877 (214h47m43.431508525s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451700169 (214h47m43.432052812s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453101406 (214h47m43.433454055s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453306320 (214h47m43.433658971s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453902924 (214h47m43.43425557s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.454485471 (214h47m43.434838121s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455040478 (214h47m43.435393125s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455598935 (214h47m43.435951585s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455867680 (214h47m43.436220329s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.861047945 (214h47m43.841400591s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:03.871023383 (214h47m43.851376028s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:04.080925197 (214h47m44.061277846s) [upload=188771]: Next returned nil segment Dec 14 14:16:04.080944439 (214h47m44.061297107s) [upload=188771]: Waiting for error group managing segments... Dec 14 14:16:04.209819284 (214h47m44.190171943s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.233005644 (214h47m44.213358285s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.234227859 (214h47m44.214580501s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239681369 (214h47m44.22003402s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239894423 (214h47m44.220247063s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.240454771 (214h47m44.220807417s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.241497930 (214h47m44.221850571s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.242469256 (214h47m44.222821901s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.243873927 (214h47m44.224226573s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.244902987 (214h47m44.225255626s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.246138337 (214h47m44.226490985s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.284989325 (214h47m44.265341963s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.787553216 (214h47m44.767905858s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:33.026981756 (214h48m13.007334401s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Dec 14 14:16:33.027035288 (214h48m13.007387915s) [seg_pos={0 0}, upload=188771]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 14 14:16:33.139454175 (214h48m13.119806811s) [upload=188771]: Done uploading segments... Dec 14 14:16:33.139495796 (214h48m13.119848423s) [upload=188771]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.49802047s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [..X............] duration: 165.949578ms remote: "tcp:[2620:cb:2000::1]:443" response headers: Content-Type=text/plain; charset=UTF-8 Content-Length=15 Cache-Control=private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Expires=Thu, 01 Jan 1970 00:00:01 GMT Cf-Ray=9ade7e5d7aae9963-FRA X-Frame-Options=SAMEORIGIN Date=Sun, 14 Dec 2025 14:46:56 GMT Referrer-Policy=same-origin Proxy-Status=Cloudflare-Proxy;error=http_response_incomplete target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/1244b8592b06e8c8 FAIL (2025-12-14 14:56:36 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "error code: 500" Duration: 165.976667ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 2.514394914s trace: https://jaeger.dev.storj.io/trace/1751ea226e84ac5d FAIL (2025-12-14 14:56:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 14 14:16:03.322622529 (214h47m43.302975166s) [upload=188771]: Starting upload Dec 14 14:16:03.322698383 (214h47m43.303051027s) [upload=188771]: Done starting upload Dec 14 14:16:03.322957650 (214h47m43.303310288s) [upload=188771]: Uploading segments... Dec 14 14:16:03.323047310 (214h47m43.303399971s) [upload=188771]: Got next segment. Inline: false Dec 14 14:16:03.323095962 (214h47m43.30344861s) [upload=188771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 14 14:16:03.323104243 (214h47m43.303456893s) [upload=188771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 14 14:16:03.446699051 (214h47m43.427051694s) [seg_pos={0 0}, upload=188771]: Begin upload segment... Dec 14 14:16:03.446762560 (214h47m43.427115199s) [seg_pos={0 0}, upload=188771]: Done begin upload segment. Dec 14 14:16:03.446782966 (214h47m43.427135626s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446798773 (214h47m43.427151422s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446883480 (214h47m43.427236128s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.447750216 (214h47m43.428102851s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448003133 (214h47m43.428355855s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448960450 (214h47m43.429313088s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.450289219 (214h47m43.43064186s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451155877 (214h47m43.431508525s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451700169 (214h47m43.432052812s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453101406 (214h47m43.433454055s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453306320 (214h47m43.433658971s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453902924 (214h47m43.43425557s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.454485471 (214h47m43.434838121s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455040478 (214h47m43.435393125s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455598935 (214h47m43.435951585s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455867680 (214h47m43.436220329s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.861047945 (214h47m43.841400591s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:03.871023383 (214h47m43.851376028s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:04.080925197 (214h47m44.061277846s) [upload=188771]: Next returned nil segment Dec 14 14:16:04.080944439 (214h47m44.061297107s) [upload=188771]: Waiting for error group managing segments... Dec 14 14:16:04.209819284 (214h47m44.190171943s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.233005644 (214h47m44.213358285s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.234227859 (214h47m44.214580501s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239681369 (214h47m44.22003402s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239894423 (214h47m44.220247063s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.240454771 (214h47m44.220807417s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.241497930 (214h47m44.221850571s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.242469256 (214h47m44.222821901s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.243873927 (214h47m44.224226573s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.244902987 (214h47m44.225255626s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.246138337 (214h47m44.226490985s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.284989325 (214h47m44.265341963s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.787553216 (214h47m44.767905858s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:33.026981756 (214h48m13.007334401s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Dec 14 14:16:33.027035288 (214h48m13.007387915s) [seg_pos={0 0}, upload=188771]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 14 14:16:33.139454175 (214h48m13.119806811s) [upload=188771]: Done uploading segments... Dec 14 14:16:33.139495796 (214h48m13.119848423s) [upload=188771]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.514432317s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [...X...........] duration: 154.99629ms remote: "tcp:[2620:cb:2000::1]:443" response headers: Content-Type=text/plain; charset=UTF-8 Content-Length=15 Cache-Control=private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Expires=Thu, 01 Jan 1970 00:00:01 GMT Cf-Ray=9ade7e5d7aae9963-FRA X-Frame-Options=SAMEORIGIN Date=Sun, 14 Dec 2025 14:46:56 GMT Referrer-Policy=same-origin Proxy-Status=Cloudflare-Proxy;error=http_response_incomplete target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/1244b8592b06e8c8 FAIL (2025-12-14 14:55:43 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "error code: 500" Duration: 155.042228ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 2.499586545s trace: https://jaeger.dev.storj.io/trace/1751ea226e84ac5d FAIL (2025-12-14 14:55:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 14 14:16:03.322622529 (214h47m43.302975166s) [upload=188771]: Starting upload Dec 14 14:16:03.322698383 (214h47m43.303051027s) [upload=188771]: Done starting upload Dec 14 14:16:03.322957650 (214h47m43.303310288s) [upload=188771]: Uploading segments... Dec 14 14:16:03.323047310 (214h47m43.303399971s) [upload=188771]: Got next segment. Inline: false Dec 14 14:16:03.323095962 (214h47m43.30344861s) [upload=188771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 14 14:16:03.323104243 (214h47m43.303456893s) [upload=188771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 14 14:16:03.446699051 (214h47m43.427051694s) [seg_pos={0 0}, upload=188771]: Begin upload segment... Dec 14 14:16:03.446762560 (214h47m43.427115199s) [seg_pos={0 0}, upload=188771]: Done begin upload segment. Dec 14 14:16:03.446782966 (214h47m43.427135626s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446798773 (214h47m43.427151422s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446883480 (214h47m43.427236128s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.447750216 (214h47m43.428102851s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448003133 (214h47m43.428355855s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448960450 (214h47m43.429313088s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.450289219 (214h47m43.43064186s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451155877 (214h47m43.431508525s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451700169 (214h47m43.432052812s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453101406 (214h47m43.433454055s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453306320 (214h47m43.433658971s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453902924 (214h47m43.43425557s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.454485471 (214h47m43.434838121s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455040478 (214h47m43.435393125s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455598935 (214h47m43.435951585s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455867680 (214h47m43.436220329s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.861047945 (214h47m43.841400591s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:03.871023383 (214h47m43.851376028s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:04.080925197 (214h47m44.061277846s) [upload=188771]: Next returned nil segment Dec 14 14:16:04.080944439 (214h47m44.061297107s) [upload=188771]: Waiting for error group managing segments... Dec 14 14:16:04.209819284 (214h47m44.190171943s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.233005644 (214h47m44.213358285s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.234227859 (214h47m44.214580501s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239681369 (214h47m44.22003402s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239894423 (214h47m44.220247063s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.240454771 (214h47m44.220807417s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.241497930 (214h47m44.221850571s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.242469256 (214h47m44.222821901s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.243873927 (214h47m44.224226573s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.244902987 (214h47m44.225255626s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.246138337 (214h47m44.226490985s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.284989325 (214h47m44.265341963s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.787553216 (214h47m44.767905858s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:33.026981756 (214h48m13.007334401s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Dec 14 14:16:33.027035288 (214h48m13.007387915s) [seg_pos={0 0}, upload=188771]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 14 14:16:33.139454175 (214h48m13.119806811s) [upload=188771]: Done uploading segments... Dec 14 14:16:33.139495796 (214h48m13.119848423s) [upload=188771]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.499650762s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [....X..........] duration: 168.942428ms remote: "tcp:[2620:cb:2000::1]:443" response headers: Content-Type=text/plain; charset=UTF-8 Content-Length=15 Cache-Control=private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Expires=Thu, 01 Jan 1970 00:00:01 GMT Cf-Ray=9ade7e5d7aae9963-FRA X-Frame-Options=SAMEORIGIN Date=Sun, 14 Dec 2025 14:46:56 GMT Referrer-Policy=same-origin Proxy-Status=Cloudflare-Proxy;error=http_response_incomplete target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/1244b8592b06e8c8 FAIL (2025-12-14 14:55:04 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "error code: 500" Duration: 168.972205ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 2.341424907s trace: https://jaeger.dev.storj.io/trace/1751ea226e84ac5d FAIL (2025-12-14 14:54:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 14 14:16:03.322622529 (214h47m43.302975166s) [upload=188771]: Starting upload Dec 14 14:16:03.322698383 (214h47m43.303051027s) [upload=188771]: Done starting upload Dec 14 14:16:03.322957650 (214h47m43.303310288s) [upload=188771]: Uploading segments... Dec 14 14:16:03.323047310 (214h47m43.303399971s) [upload=188771]: Got next segment. Inline: false Dec 14 14:16:03.323095962 (214h47m43.30344861s) [upload=188771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 14 14:16:03.323104243 (214h47m43.303456893s) [upload=188771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 14 14:16:03.446699051 (214h47m43.427051694s) [seg_pos={0 0}, upload=188771]: Begin upload segment... Dec 14 14:16:03.446762560 (214h47m43.427115199s) [seg_pos={0 0}, upload=188771]: Done begin upload segment. Dec 14 14:16:03.446782966 (214h47m43.427135626s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446798773 (214h47m43.427151422s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446883480 (214h47m43.427236128s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.447750216 (214h47m43.428102851s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448003133 (214h47m43.428355855s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448960450 (214h47m43.429313088s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.450289219 (214h47m43.43064186s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451155877 (214h47m43.431508525s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451700169 (214h47m43.432052812s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453101406 (214h47m43.433454055s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453306320 (214h47m43.433658971s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453902924 (214h47m43.43425557s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.454485471 (214h47m43.434838121s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455040478 (214h47m43.435393125s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455598935 (214h47m43.435951585s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455867680 (214h47m43.436220329s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.861047945 (214h47m43.841400591s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:03.871023383 (214h47m43.851376028s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:04.080925197 (214h47m44.061277846s) [upload=188771]: Next returned nil segment Dec 14 14:16:04.080944439 (214h47m44.061297107s) [upload=188771]: Waiting for error group managing segments... Dec 14 14:16:04.209819284 (214h47m44.190171943s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.233005644 (214h47m44.213358285s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.234227859 (214h47m44.214580501s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239681369 (214h47m44.22003402s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239894423 (214h47m44.220247063s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.240454771 (214h47m44.220807417s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.241497930 (214h47m44.221850571s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.242469256 (214h47m44.222821901s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.243873927 (214h47m44.224226573s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.244902987 (214h47m44.225255626s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.246138337 (214h47m44.226490985s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.284989325 (214h47m44.265341963s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.787553216 (214h47m44.767905858s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:33.026981756 (214h48m13.007334401s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Dec 14 14:16:33.027035288 (214h48m13.007387915s) [seg_pos={0 0}, upload=188771]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 14 14:16:33.139454175 (214h48m13.119806811s) [upload=188771]: Done uploading segments... Dec 14 14:16:33.139495796 (214h48m13.119848423s) [upload=188771]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.34145717s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.....X.........] duration: 165.464735ms remote: "tcp:[2620:cb:2000::1]:443" response headers: Content-Type=text/plain; charset=UTF-8 Content-Length=15 Cache-Control=private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Expires=Thu, 01 Jan 1970 00:00:01 GMT Cf-Ray=9ade7e5d7aae9963-FRA X-Frame-Options=SAMEORIGIN Date=Sun, 14 Dec 2025 14:46:56 GMT Referrer-Policy=same-origin Proxy-Status=Cloudflare-Proxy;error=http_response_incomplete target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/1244b8592b06e8c8 FAIL (2025-12-14 14:54:15 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "error code: 500" Duration: 165.489656ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 2.440098041s trace: https://jaeger.dev.storj.io/trace/1751ea226e84ac5d FAIL (2025-12-14 14:54:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 14 14:16:03.322622529 (214h47m43.302975166s) [upload=188771]: Starting upload Dec 14 14:16:03.322698383 (214h47m43.303051027s) [upload=188771]: Done starting upload Dec 14 14:16:03.322957650 (214h47m43.303310288s) [upload=188771]: Uploading segments... Dec 14 14:16:03.323047310 (214h47m43.303399971s) [upload=188771]: Got next segment. Inline: false Dec 14 14:16:03.323095962 (214h47m43.30344861s) [upload=188771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 14 14:16:03.323104243 (214h47m43.303456893s) [upload=188771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 14 14:16:03.446699051 (214h47m43.427051694s) [seg_pos={0 0}, upload=188771]: Begin upload segment... Dec 14 14:16:03.446762560 (214h47m43.427115199s) [seg_pos={0 0}, upload=188771]: Done begin upload segment. Dec 14 14:16:03.446782966 (214h47m43.427135626s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446798773 (214h47m43.427151422s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446883480 (214h47m43.427236128s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.447750216 (214h47m43.428102851s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448003133 (214h47m43.428355855s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448960450 (214h47m43.429313088s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.450289219 (214h47m43.43064186s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451155877 (214h47m43.431508525s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451700169 (214h47m43.432052812s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453101406 (214h47m43.433454055s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453306320 (214h47m43.433658971s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453902924 (214h47m43.43425557s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.454485471 (214h47m43.434838121s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455040478 (214h47m43.435393125s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455598935 (214h47m43.435951585s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455867680 (214h47m43.436220329s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.861047945 (214h47m43.841400591s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:03.871023383 (214h47m43.851376028s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:04.080925197 (214h47m44.061277846s) [upload=188771]: Next returned nil segment Dec 14 14:16:04.080944439 (214h47m44.061297107s) [upload=188771]: Waiting for error group managing segments... Dec 14 14:16:04.209819284 (214h47m44.190171943s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.233005644 (214h47m44.213358285s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.234227859 (214h47m44.214580501s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239681369 (214h47m44.22003402s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239894423 (214h47m44.220247063s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.240454771 (214h47m44.220807417s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.241497930 (214h47m44.221850571s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.242469256 (214h47m44.222821901s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.243873927 (214h47m44.224226573s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.244902987 (214h47m44.225255626s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.246138337 (214h47m44.226490985s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.284989325 (214h47m44.265341963s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.787553216 (214h47m44.767905858s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:33.026981756 (214h48m13.007334401s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Dec 14 14:16:33.027035288 (214h48m13.007387915s) [seg_pos={0 0}, upload=188771]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 14 14:16:33.139454175 (214h48m13.119806811s) [upload=188771]: Done uploading segments... Dec 14 14:16:33.139495796 (214h48m13.119848423s) [upload=188771]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.440149118s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 2.438828441s trace: https://jaeger.dev.storj.io/trace/1751ea226e84ac5d FAIL (2025-12-14 14:53:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 14 14:16:03.322622529 (214h47m43.302975166s) [upload=188771]: Starting upload Dec 14 14:16:03.322698383 (214h47m43.303051027s) [upload=188771]: Done starting upload Dec 14 14:16:03.322957650 (214h47m43.303310288s) [upload=188771]: Uploading segments... Dec 14 14:16:03.323047310 (214h47m43.303399971s) [upload=188771]: Got next segment. Inline: false Dec 14 14:16:03.323095962 (214h47m43.30344861s) [upload=188771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 14 14:16:03.323104243 (214h47m43.303456893s) [upload=188771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 14 14:16:03.446699051 (214h47m43.427051694s) [seg_pos={0 0}, upload=188771]: Begin upload segment... Dec 14 14:16:03.446762560 (214h47m43.427115199s) [seg_pos={0 0}, upload=188771]: Done begin upload segment. Dec 14 14:16:03.446782966 (214h47m43.427135626s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446798773 (214h47m43.427151422s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446883480 (214h47m43.427236128s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.447750216 (214h47m43.428102851s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448003133 (214h47m43.428355855s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448960450 (214h47m43.429313088s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.450289219 (214h47m43.43064186s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451155877 (214h47m43.431508525s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451700169 (214h47m43.432052812s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453101406 (214h47m43.433454055s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453306320 (214h47m43.433658971s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453902924 (214h47m43.43425557s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.454485471 (214h47m43.434838121s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455040478 (214h47m43.435393125s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455598935 (214h47m43.435951585s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455867680 (214h47m43.436220329s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.861047945 (214h47m43.841400591s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:03.871023383 (214h47m43.851376028s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:04.080925197 (214h47m44.061277846s) [upload=188771]: Next returned nil segment Dec 14 14:16:04.080944439 (214h47m44.061297107s) [upload=188771]: Waiting for error group managing segments... Dec 14 14:16:04.209819284 (214h47m44.190171943s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.233005644 (214h47m44.213358285s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.234227859 (214h47m44.214580501s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239681369 (214h47m44.22003402s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239894423 (214h47m44.220247063s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.240454771 (214h47m44.220807417s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.241497930 (214h47m44.221850571s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.242469256 (214h47m44.222821901s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.243873927 (214h47m44.224226573s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.244902987 (214h47m44.225255626s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.246138337 (214h47m44.226490985s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.284989325 (214h47m44.265341963s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.787553216 (214h47m44.767905858s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:33.026981756 (214h48m13.007334401s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Dec 14 14:16:33.027035288 (214h48m13.007387915s) [seg_pos={0 0}, upload=188771]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 14 14:16:33.139454175 (214h48m13.119806811s) [upload=188771]: Done uploading segments... Dec 14 14:16:33.139495796 (214h48m13.119848423s) [upload=188771]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.438889548s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 2.380098704s trace: https://jaeger.dev.storj.io/trace/1751ea226e84ac5d FAIL (2025-12-14 14:53:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 14 14:16:03.322622529 (214h47m43.302975166s) [upload=188771]: Starting upload Dec 14 14:16:03.322698383 (214h47m43.303051027s) [upload=188771]: Done starting upload Dec 14 14:16:03.322957650 (214h47m43.303310288s) [upload=188771]: Uploading segments... Dec 14 14:16:03.323047310 (214h47m43.303399971s) [upload=188771]: Got next segment. Inline: false Dec 14 14:16:03.323095962 (214h47m43.30344861s) [upload=188771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 14 14:16:03.323104243 (214h47m43.303456893s) [upload=188771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 14 14:16:03.446699051 (214h47m43.427051694s) [seg_pos={0 0}, upload=188771]: Begin upload segment... Dec 14 14:16:03.446762560 (214h47m43.427115199s) [seg_pos={0 0}, upload=188771]: Done begin upload segment. Dec 14 14:16:03.446782966 (214h47m43.427135626s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446798773 (214h47m43.427151422s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446883480 (214h47m43.427236128s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.447750216 (214h47m43.428102851s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448003133 (214h47m43.428355855s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448960450 (214h47m43.429313088s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.450289219 (214h47m43.43064186s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451155877 (214h47m43.431508525s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451700169 (214h47m43.432052812s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453101406 (214h47m43.433454055s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453306320 (214h47m43.433658971s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453902924 (214h47m43.43425557s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.454485471 (214h47m43.434838121s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455040478 (214h47m43.435393125s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455598935 (214h47m43.435951585s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455867680 (214h47m43.436220329s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.861047945 (214h47m43.841400591s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:03.871023383 (214h47m43.851376028s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:04.080925197 (214h47m44.061277846s) [upload=188771]: Next returned nil segment Dec 14 14:16:04.080944439 (214h47m44.061297107s) [upload=188771]: Waiting for error group managing segments... Dec 14 14:16:04.209819284 (214h47m44.190171943s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.233005644 (214h47m44.213358285s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.234227859 (214h47m44.214580501s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239681369 (214h47m44.22003402s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239894423 (214h47m44.220247063s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.240454771 (214h47m44.220807417s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.241497930 (214h47m44.221850571s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.242469256 (214h47m44.222821901s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.243873927 (214h47m44.224226573s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.244902987 (214h47m44.225255626s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.246138337 (214h47m44.226490985s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.284989325 (214h47m44.265341963s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.787553216 (214h47m44.767905858s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:33.026981756 (214h48m13.007334401s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Dec 14 14:16:33.027035288 (214h48m13.007387915s) [seg_pos={0 0}, upload=188771]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 14 14:16:33.139454175 (214h48m13.119806811s) [upload=188771]: Done uploading segments... Dec 14 14:16:33.139495796 (214h48m13.119848423s) [upload=188771]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.380124325s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [......X........] duration: 163.47303ms remote: "tcp:[2620:cb:2000::1]:443" response headers: Content-Type=text/plain; charset=UTF-8 Content-Length=15 Cache-Control=private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Expires=Thu, 01 Jan 1970 00:00:01 GMT Cf-Ray=9ade7e5d7aae9963-FRA X-Frame-Options=SAMEORIGIN Date=Sun, 14 Dec 2025 14:46:56 GMT Referrer-Policy=same-origin Proxy-Status=Cloudflare-Proxy;error=http_response_incomplete target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/1244b8592b06e8c8 FAIL (2025-12-14 14:53:17 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "error code: 500" Duration: 163.498209ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 2.572742634s trace: https://jaeger.dev.storj.io/trace/1751ea226e84ac5d FAIL (2025-12-14 14:52:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 14 14:16:03.322622529 (214h47m43.302975166s) [upload=188771]: Starting upload Dec 14 14:16:03.322698383 (214h47m43.303051027s) [upload=188771]: Done starting upload Dec 14 14:16:03.322957650 (214h47m43.303310288s) [upload=188771]: Uploading segments... Dec 14 14:16:03.323047310 (214h47m43.303399971s) [upload=188771]: Got next segment. Inline: false Dec 14 14:16:03.323095962 (214h47m43.30344861s) [upload=188771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 14 14:16:03.323104243 (214h47m43.303456893s) [upload=188771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 14 14:16:03.446699051 (214h47m43.427051694s) [seg_pos={0 0}, upload=188771]: Begin upload segment... Dec 14 14:16:03.446762560 (214h47m43.427115199s) [seg_pos={0 0}, upload=188771]: Done begin upload segment. Dec 14 14:16:03.446782966 (214h47m43.427135626s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446798773 (214h47m43.427151422s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446883480 (214h47m43.427236128s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.447750216 (214h47m43.428102851s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448003133 (214h47m43.428355855s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448960450 (214h47m43.429313088s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.450289219 (214h47m43.43064186s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451155877 (214h47m43.431508525s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451700169 (214h47m43.432052812s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453101406 (214h47m43.433454055s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453306320 (214h47m43.433658971s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453902924 (214h47m43.43425557s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.454485471 (214h47m43.434838121s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455040478 (214h47m43.435393125s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455598935 (214h47m43.435951585s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455867680 (214h47m43.436220329s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.861047945 (214h47m43.841400591s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:03.871023383 (214h47m43.851376028s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:04.080925197 (214h47m44.061277846s) [upload=188771]: Next returned nil segment Dec 14 14:16:04.080944439 (214h47m44.061297107s) [upload=188771]: Waiting for error group managing segments... Dec 14 14:16:04.209819284 (214h47m44.190171943s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.233005644 (214h47m44.213358285s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.234227859 (214h47m44.214580501s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239681369 (214h47m44.22003402s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239894423 (214h47m44.220247063s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.240454771 (214h47m44.220807417s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.241497930 (214h47m44.221850571s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.242469256 (214h47m44.222821901s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.243873927 (214h47m44.224226573s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.244902987 (214h47m44.225255626s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.246138337 (214h47m44.226490985s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.284989325 (214h47m44.265341963s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.787553216 (214h47m44.767905858s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:33.026981756 (214h48m13.007334401s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Dec 14 14:16:33.027035288 (214h48m13.007387915s) [seg_pos={0 0}, upload=188771]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 14 14:16:33.139454175 (214h48m13.119806811s) [upload=188771]: Done uploading segments... Dec 14 14:16:33.139495796 (214h48m13.119848423s) [upload=188771]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.572767404s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.......X.......] duration: 186.848048ms remote: "tcp:[2620:cb:2000::1]:443" response headers: Content-Type=text/plain; charset=UTF-8 Content-Length=15 Cache-Control=private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Expires=Thu, 01 Jan 1970 00:00:01 GMT Cf-Ray=9ade7e5d7aae9963-FRA X-Frame-Options=SAMEORIGIN Date=Sun, 14 Dec 2025 14:46:56 GMT Referrer-Policy=same-origin Proxy-Status=Cloudflare-Proxy;error=http_response_incomplete target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/1244b8592b06e8c8 FAIL (2025-12-14 14:52:20 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "error code: 500" Duration: 186.878222ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 2.432289735s trace: https://jaeger.dev.storj.io/trace/1751ea226e84ac5d FAIL (2025-12-14 14:51:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 14 14:16:03.322622529 (214h47m43.302975166s) [upload=188771]: Starting upload Dec 14 14:16:03.322698383 (214h47m43.303051027s) [upload=188771]: Done starting upload Dec 14 14:16:03.322957650 (214h47m43.303310288s) [upload=188771]: Uploading segments... Dec 14 14:16:03.323047310 (214h47m43.303399971s) [upload=188771]: Got next segment. Inline: false Dec 14 14:16:03.323095962 (214h47m43.30344861s) [upload=188771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 14 14:16:03.323104243 (214h47m43.303456893s) [upload=188771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 14 14:16:03.446699051 (214h47m43.427051694s) [seg_pos={0 0}, upload=188771]: Begin upload segment... Dec 14 14:16:03.446762560 (214h47m43.427115199s) [seg_pos={0 0}, upload=188771]: Done begin upload segment. Dec 14 14:16:03.446782966 (214h47m43.427135626s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446798773 (214h47m43.427151422s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446883480 (214h47m43.427236128s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.447750216 (214h47m43.428102851s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448003133 (214h47m43.428355855s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448960450 (214h47m43.429313088s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.450289219 (214h47m43.43064186s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451155877 (214h47m43.431508525s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451700169 (214h47m43.432052812s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453101406 (214h47m43.433454055s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453306320 (214h47m43.433658971s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453902924 (214h47m43.43425557s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.454485471 (214h47m43.434838121s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455040478 (214h47m43.435393125s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455598935 (214h47m43.435951585s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455867680 (214h47m43.436220329s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.861047945 (214h47m43.841400591s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:03.871023383 (214h47m43.851376028s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:04.080925197 (214h47m44.061277846s) [upload=188771]: Next returned nil segment Dec 14 14:16:04.080944439 (214h47m44.061297107s) [upload=188771]: Waiting for error group managing segments... Dec 14 14:16:04.209819284 (214h47m44.190171943s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.233005644 (214h47m44.213358285s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.234227859 (214h47m44.214580501s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239681369 (214h47m44.22003402s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239894423 (214h47m44.220247063s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.240454771 (214h47m44.220807417s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.241497930 (214h47m44.221850571s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.242469256 (214h47m44.222821901s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.243873927 (214h47m44.224226573s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.244902987 (214h47m44.225255626s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.246138337 (214h47m44.226490985s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.284989325 (214h47m44.265341963s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.787553216 (214h47m44.767905858s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:33.026981756 (214h48m13.007334401s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Dec 14 14:16:33.027035288 (214h48m13.007387915s) [seg_pos={0 0}, upload=188771]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 14 14:16:33.139454175 (214h48m13.119806811s) [upload=188771]: Done uploading segments... Dec 14 14:16:33.139495796 (214h48m13.119848423s) [upload=188771]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.432315648s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [........X......] duration: 157.419569ms remote: "tcp:[2620:cb:2000::1]:443" response headers: Content-Type=text/plain; charset=UTF-8 Content-Length=15 Cache-Control=private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Expires=Thu, 01 Jan 1970 00:00:01 GMT Cf-Ray=9ade7e5d7aae9963-FRA X-Frame-Options=SAMEORIGIN Date=Sun, 14 Dec 2025 14:46:56 GMT Referrer-Policy=same-origin Proxy-Status=Cloudflare-Proxy;error=http_response_incomplete target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/1244b8592b06e8c8 FAIL (2025-12-14 14:51:28 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "error code: 500" Duration: 157.445602ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 2.388926207s trace: https://jaeger.dev.storj.io/trace/1751ea226e84ac5d FAIL (2025-12-14 14:50:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 14 14:16:03.322622529 (214h47m43.302975166s) [upload=188771]: Starting upload Dec 14 14:16:03.322698383 (214h47m43.303051027s) [upload=188771]: Done starting upload Dec 14 14:16:03.322957650 (214h47m43.303310288s) [upload=188771]: Uploading segments... Dec 14 14:16:03.323047310 (214h47m43.303399971s) [upload=188771]: Got next segment. Inline: false Dec 14 14:16:03.323095962 (214h47m43.30344861s) [upload=188771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 14 14:16:03.323104243 (214h47m43.303456893s) [upload=188771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 14 14:16:03.446699051 (214h47m43.427051694s) [seg_pos={0 0}, upload=188771]: Begin upload segment... Dec 14 14:16:03.446762560 (214h47m43.427115199s) [seg_pos={0 0}, upload=188771]: Done begin upload segment. Dec 14 14:16:03.446782966 (214h47m43.427135626s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446798773 (214h47m43.427151422s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446883480 (214h47m43.427236128s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.447750216 (214h47m43.428102851s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448003133 (214h47m43.428355855s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448960450 (214h47m43.429313088s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.450289219 (214h47m43.43064186s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451155877 (214h47m43.431508525s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451700169 (214h47m43.432052812s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453101406 (214h47m43.433454055s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453306320 (214h47m43.433658971s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453902924 (214h47m43.43425557s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.454485471 (214h47m43.434838121s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455040478 (214h47m43.435393125s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455598935 (214h47m43.435951585s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455867680 (214h47m43.436220329s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.861047945 (214h47m43.841400591s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:03.871023383 (214h47m43.851376028s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:04.080925197 (214h47m44.061277846s) [upload=188771]: Next returned nil segment Dec 14 14:16:04.080944439 (214h47m44.061297107s) [upload=188771]: Waiting for error group managing segments... Dec 14 14:16:04.209819284 (214h47m44.190171943s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.233005644 (214h47m44.213358285s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.234227859 (214h47m44.214580501s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239681369 (214h47m44.22003402s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239894423 (214h47m44.220247063s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.240454771 (214h47m44.220807417s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.241497930 (214h47m44.221850571s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.242469256 (214h47m44.222821901s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.243873927 (214h47m44.224226573s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.244902987 (214h47m44.225255626s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.246138337 (214h47m44.226490985s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.284989325 (214h47m44.265341963s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.787553216 (214h47m44.767905858s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:33.026981756 (214h48m13.007334401s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Dec 14 14:16:33.027035288 (214h48m13.007387915s) [seg_pos={0 0}, upload=188771]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 14 14:16:33.139454175 (214h48m13.119806811s) [upload=188771]: Done uploading segments... Dec 14 14:16:33.139495796 (214h48m13.119848423s) [upload=188771]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.388952461s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.........X.....] duration: 165.165812ms remote: "tcp:[2620:cb:2000::1]:443" response headers: Content-Type=text/plain; charset=UTF-8 Content-Length=15 Cache-Control=private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Expires=Thu, 01 Jan 1970 00:00:01 GMT Cf-Ray=9ade7e5d7aae9963-FRA X-Frame-Options=SAMEORIGIN Date=Sun, 14 Dec 2025 14:46:56 GMT Referrer-Policy=same-origin Proxy-Status=Cloudflare-Proxy;error=http_response_incomplete target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/1244b8592b06e8c8 FAIL (2025-12-14 14:50:36 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "error code: 500" Duration: 165.191352ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 2.259857847s trace: https://jaeger.dev.storj.io/trace/1751ea226e84ac5d FAIL (2025-12-14 14:50:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 14 14:16:03.322622529 (214h47m43.302975166s) [upload=188771]: Starting upload Dec 14 14:16:03.322698383 (214h47m43.303051027s) [upload=188771]: Done starting upload Dec 14 14:16:03.322957650 (214h47m43.303310288s) [upload=188771]: Uploading segments... Dec 14 14:16:03.323047310 (214h47m43.303399971s) [upload=188771]: Got next segment. Inline: false Dec 14 14:16:03.323095962 (214h47m43.30344861s) [upload=188771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 14 14:16:03.323104243 (214h47m43.303456893s) [upload=188771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 14 14:16:03.446699051 (214h47m43.427051694s) [seg_pos={0 0}, upload=188771]: Begin upload segment... Dec 14 14:16:03.446762560 (214h47m43.427115199s) [seg_pos={0 0}, upload=188771]: Done begin upload segment. Dec 14 14:16:03.446782966 (214h47m43.427135626s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446798773 (214h47m43.427151422s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446883480 (214h47m43.427236128s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.447750216 (214h47m43.428102851s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448003133 (214h47m43.428355855s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448960450 (214h47m43.429313088s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.450289219 (214h47m43.43064186s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451155877 (214h47m43.431508525s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451700169 (214h47m43.432052812s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453101406 (214h47m43.433454055s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453306320 (214h47m43.433658971s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453902924 (214h47m43.43425557s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.454485471 (214h47m43.434838121s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455040478 (214h47m43.435393125s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455598935 (214h47m43.435951585s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455867680 (214h47m43.436220329s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.861047945 (214h47m43.841400591s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:03.871023383 (214h47m43.851376028s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:04.080925197 (214h47m44.061277846s) [upload=188771]: Next returned nil segment Dec 14 14:16:04.080944439 (214h47m44.061297107s) [upload=188771]: Waiting for error group managing segments... Dec 14 14:16:04.209819284 (214h47m44.190171943s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.233005644 (214h47m44.213358285s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.234227859 (214h47m44.214580501s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239681369 (214h47m44.22003402s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239894423 (214h47m44.220247063s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.240454771 (214h47m44.220807417s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.241497930 (214h47m44.221850571s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.242469256 (214h47m44.222821901s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.243873927 (214h47m44.224226573s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.244902987 (214h47m44.225255626s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.246138337 (214h47m44.226490985s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.284989325 (214h47m44.265341963s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.787553216 (214h47m44.767905858s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:33.026981756 (214h48m13.007334401s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Dec 14 14:16:33.027035288 (214h48m13.007387915s) [seg_pos={0 0}, upload=188771]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 14 14:16:33.139454175 (214h48m13.119806811s) [upload=188771]: Done uploading segments... Dec 14 14:16:33.139495796 (214h48m13.119848423s) [upload=188771]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.259885615s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 2.324469387s trace: https://jaeger.dev.storj.io/trace/1751ea226e84ac5d FAIL (2025-12-14 14:49:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 14 14:16:03.322622529 (214h47m43.302975166s) [upload=188771]: Starting upload Dec 14 14:16:03.322698383 (214h47m43.303051027s) [upload=188771]: Done starting upload Dec 14 14:16:03.322957650 (214h47m43.303310288s) [upload=188771]: Uploading segments... Dec 14 14:16:03.323047310 (214h47m43.303399971s) [upload=188771]: Got next segment. Inline: false Dec 14 14:16:03.323095962 (214h47m43.30344861s) [upload=188771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 14 14:16:03.323104243 (214h47m43.303456893s) [upload=188771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 14 14:16:03.446699051 (214h47m43.427051694s) [seg_pos={0 0}, upload=188771]: Begin upload segment... Dec 14 14:16:03.446762560 (214h47m43.427115199s) [seg_pos={0 0}, upload=188771]: Done begin upload segment. Dec 14 14:16:03.446782966 (214h47m43.427135626s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446798773 (214h47m43.427151422s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446883480 (214h47m43.427236128s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.447750216 (214h47m43.428102851s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448003133 (214h47m43.428355855s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448960450 (214h47m43.429313088s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.450289219 (214h47m43.43064186s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451155877 (214h47m43.431508525s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451700169 (214h47m43.432052812s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453101406 (214h47m43.433454055s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453306320 (214h47m43.433658971s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453902924 (214h47m43.43425557s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.454485471 (214h47m43.434838121s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455040478 (214h47m43.435393125s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455598935 (214h47m43.435951585s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455867680 (214h47m43.436220329s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.861047945 (214h47m43.841400591s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:03.871023383 (214h47m43.851376028s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:04.080925197 (214h47m44.061277846s) [upload=188771]: Next returned nil segment Dec 14 14:16:04.080944439 (214h47m44.061297107s) [upload=188771]: Waiting for error group managing segments... Dec 14 14:16:04.209819284 (214h47m44.190171943s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.233005644 (214h47m44.213358285s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.234227859 (214h47m44.214580501s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239681369 (214h47m44.22003402s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239894423 (214h47m44.220247063s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.240454771 (214h47m44.220807417s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.241497930 (214h47m44.221850571s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.242469256 (214h47m44.222821901s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.243873927 (214h47m44.224226573s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.244902987 (214h47m44.225255626s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.246138337 (214h47m44.226490985s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.284989325 (214h47m44.265341963s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.787553216 (214h47m44.767905858s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:33.026981756 (214h48m13.007334401s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Dec 14 14:16:33.027035288 (214h48m13.007387915s) [seg_pos={0 0}, upload=188771]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 14 14:16:33.139454175 (214h48m13.119806811s) [upload=188771]: Done uploading segments... Dec 14 14:16:33.139495796 (214h48m13.119848423s) [upload=188771]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.324498418s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [..........X....] duration: 163.901795ms remote: "tcp:[2620:cb:2000::1]:443" response headers: Content-Type=text/plain; charset=UTF-8 Content-Length=15 Cache-Control=private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Expires=Thu, 01 Jan 1970 00:00:01 GMT Cf-Ray=9ade7e5d7aae9963-FRA X-Frame-Options=SAMEORIGIN Date=Sun, 14 Dec 2025 14:46:56 GMT Referrer-Policy=same-origin Proxy-Status=Cloudflare-Proxy;error=http_response_incomplete target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/1244b8592b06e8c8 FAIL (2025-12-14 14:49:50 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "error code: 500" Duration: 163.993605ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [...........X...] duration: 170.604291ms remote: "tcp:[2620:cb:2000::1]:443" response headers: Content-Type=text/plain; charset=UTF-8 Content-Length=15 Cache-Control=private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Expires=Thu, 01 Jan 1970 00:00:01 GMT Cf-Ray=9ade7e5d7aae9963-FRA X-Frame-Options=SAMEORIGIN Date=Sun, 14 Dec 2025 14:46:56 GMT Referrer-Policy=same-origin Proxy-Status=Cloudflare-Proxy;error=http_response_incomplete target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/1244b8592b06e8c8 FAIL (2025-12-14 14:49:30 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "error code: 500" Duration: 170.630994ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 2.28359372s trace: https://jaeger.dev.storj.io/trace/1751ea226e84ac5d FAIL (2025-12-14 14:49:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 14 14:16:03.322622529 (214h47m43.302975166s) [upload=188771]: Starting upload Dec 14 14:16:03.322698383 (214h47m43.303051027s) [upload=188771]: Done starting upload Dec 14 14:16:03.322957650 (214h47m43.303310288s) [upload=188771]: Uploading segments... Dec 14 14:16:03.323047310 (214h47m43.303399971s) [upload=188771]: Got next segment. Inline: false Dec 14 14:16:03.323095962 (214h47m43.30344861s) [upload=188771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 14 14:16:03.323104243 (214h47m43.303456893s) [upload=188771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 14 14:16:03.446699051 (214h47m43.427051694s) [seg_pos={0 0}, upload=188771]: Begin upload segment... Dec 14 14:16:03.446762560 (214h47m43.427115199s) [seg_pos={0 0}, upload=188771]: Done begin upload segment. Dec 14 14:16:03.446782966 (214h47m43.427135626s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446798773 (214h47m43.427151422s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446883480 (214h47m43.427236128s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.447750216 (214h47m43.428102851s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448003133 (214h47m43.428355855s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448960450 (214h47m43.429313088s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.450289219 (214h47m43.43064186s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451155877 (214h47m43.431508525s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451700169 (214h47m43.432052812s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453101406 (214h47m43.433454055s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453306320 (214h47m43.433658971s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453902924 (214h47m43.43425557s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.454485471 (214h47m43.434838121s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455040478 (214h47m43.435393125s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455598935 (214h47m43.435951585s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455867680 (214h47m43.436220329s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.861047945 (214h47m43.841400591s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:03.871023383 (214h47m43.851376028s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:04.080925197 (214h47m44.061277846s) [upload=188771]: Next returned nil segment Dec 14 14:16:04.080944439 (214h47m44.061297107s) [upload=188771]: Waiting for error group managing segments... Dec 14 14:16:04.209819284 (214h47m44.190171943s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.233005644 (214h47m44.213358285s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.234227859 (214h47m44.214580501s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239681369 (214h47m44.22003402s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239894423 (214h47m44.220247063s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.240454771 (214h47m44.220807417s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.241497930 (214h47m44.221850571s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.242469256 (214h47m44.222821901s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.243873927 (214h47m44.224226573s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.244902987 (214h47m44.225255626s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.246138337 (214h47m44.226490985s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.284989325 (214h47m44.265341963s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.787553216 (214h47m44.767905858s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:33.026981756 (214h48m13.007334401s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Dec 14 14:16:33.027035288 (214h48m13.007387915s) [seg_pos={0 0}, upload=188771]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 14 14:16:33.139454175 (214h48m13.119806811s) [upload=188771]: Done uploading segments... Dec 14 14:16:33.139495796 (214h48m13.119848423s) [upload=188771]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.283620814s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [............X..] duration: 155.767401ms remote: "tcp:[2620:cb:2000::1]:443" response headers: Content-Type=text/plain; charset=UTF-8 Content-Length=15 Cache-Control=private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Expires=Thu, 01 Jan 1970 00:00:01 GMT Cf-Ray=9ade7e5d7aae9963-FRA X-Frame-Options=SAMEORIGIN Date=Sun, 14 Dec 2025 14:46:56 GMT Referrer-Policy=same-origin Proxy-Status=Cloudflare-Proxy;error=http_response_incomplete target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/1244b8592b06e8c8 FAIL (2025-12-14 14:48:33 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "error code: 500" Duration: 155.792771ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 2.319924136s trace: https://jaeger.dev.storj.io/trace/1751ea226e84ac5d FAIL (2025-12-14 14:48:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 14 14:16:03.322622529 (214h47m43.302975166s) [upload=188771]: Starting upload Dec 14 14:16:03.322698383 (214h47m43.303051027s) [upload=188771]: Done starting upload Dec 14 14:16:03.322957650 (214h47m43.303310288s) [upload=188771]: Uploading segments... Dec 14 14:16:03.323047310 (214h47m43.303399971s) [upload=188771]: Got next segment. Inline: false Dec 14 14:16:03.323095962 (214h47m43.30344861s) [upload=188771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 14 14:16:03.323104243 (214h47m43.303456893s) [upload=188771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 14 14:16:03.446699051 (214h47m43.427051694s) [seg_pos={0 0}, upload=188771]: Begin upload segment... Dec 14 14:16:03.446762560 (214h47m43.427115199s) [seg_pos={0 0}, upload=188771]: Done begin upload segment. Dec 14 14:16:03.446782966 (214h47m43.427135626s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446798773 (214h47m43.427151422s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446883480 (214h47m43.427236128s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.447750216 (214h47m43.428102851s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448003133 (214h47m43.428355855s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448960450 (214h47m43.429313088s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.450289219 (214h47m43.43064186s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451155877 (214h47m43.431508525s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451700169 (214h47m43.432052812s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453101406 (214h47m43.433454055s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453306320 (214h47m43.433658971s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453902924 (214h47m43.43425557s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.454485471 (214h47m43.434838121s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455040478 (214h47m43.435393125s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455598935 (214h47m43.435951585s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455867680 (214h47m43.436220329s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.861047945 (214h47m43.841400591s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:03.871023383 (214h47m43.851376028s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:04.080925197 (214h47m44.061277846s) [upload=188771]: Next returned nil segment Dec 14 14:16:04.080944439 (214h47m44.061297107s) [upload=188771]: Waiting for error group managing segments... Dec 14 14:16:04.209819284 (214h47m44.190171943s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.233005644 (214h47m44.213358285s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.234227859 (214h47m44.214580501s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239681369 (214h47m44.22003402s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239894423 (214h47m44.220247063s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.240454771 (214h47m44.220807417s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.241497930 (214h47m44.221850571s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.242469256 (214h47m44.222821901s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.243873927 (214h47m44.224226573s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.244902987 (214h47m44.225255626s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.246138337 (214h47m44.226490985s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.284989325 (214h47m44.265341963s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.787553216 (214h47m44.767905858s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:33.026981756 (214h48m13.007334401s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Dec 14 14:16:33.027035288 (214h48m13.007387915s) [seg_pos={0 0}, upload=188771]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 14 14:16:33.139454175 (214h48m13.119806811s) [upload=188771]: Done uploading segments... Dec 14 14:16:33.139495796 (214h48m13.119848423s) [upload=188771]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.319952327s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 2.516328765s trace: https://jaeger.dev.storj.io/trace/1751ea226e84ac5d FAIL (2025-12-14 14:47:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 14 14:16:03.322622529 (214h47m43.302975166s) [upload=188771]: Starting upload Dec 14 14:16:03.322698383 (214h47m43.303051027s) [upload=188771]: Done starting upload Dec 14 14:16:03.322957650 (214h47m43.303310288s) [upload=188771]: Uploading segments... Dec 14 14:16:03.323047310 (214h47m43.303399971s) [upload=188771]: Got next segment. Inline: false Dec 14 14:16:03.323095962 (214h47m43.30344861s) [upload=188771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 14 14:16:03.323104243 (214h47m43.303456893s) [upload=188771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 14 14:16:03.446699051 (214h47m43.427051694s) [seg_pos={0 0}, upload=188771]: Begin upload segment... Dec 14 14:16:03.446762560 (214h47m43.427115199s) [seg_pos={0 0}, upload=188771]: Done begin upload segment. Dec 14 14:16:03.446782966 (214h47m43.427135626s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446798773 (214h47m43.427151422s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446883480 (214h47m43.427236128s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.447750216 (214h47m43.428102851s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448003133 (214h47m43.428355855s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448960450 (214h47m43.429313088s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.450289219 (214h47m43.43064186s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451155877 (214h47m43.431508525s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451700169 (214h47m43.432052812s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453101406 (214h47m43.433454055s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453306320 (214h47m43.433658971s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453902924 (214h47m43.43425557s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.454485471 (214h47m43.434838121s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455040478 (214h47m43.435393125s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455598935 (214h47m43.435951585s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455867680 (214h47m43.436220329s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.861047945 (214h47m43.841400591s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:03.871023383 (214h47m43.851376028s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:04.080925197 (214h47m44.061277846s) [upload=188771]: Next returned nil segment Dec 14 14:16:04.080944439 (214h47m44.061297107s) [upload=188771]: Waiting for error group managing segments... Dec 14 14:16:04.209819284 (214h47m44.190171943s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.233005644 (214h47m44.213358285s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.234227859 (214h47m44.214580501s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239681369 (214h47m44.22003402s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239894423 (214h47m44.220247063s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.240454771 (214h47m44.220807417s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.241497930 (214h47m44.221850571s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.242469256 (214h47m44.222821901s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.243873927 (214h47m44.224226573s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.244902987 (214h47m44.225255626s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.246138337 (214h47m44.226490985s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.284989325 (214h47m44.265341963s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.787553216 (214h47m44.767905858s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:33.026981756 (214h48m13.007334401s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Dec 14 14:16:33.027035288 (214h48m13.007387915s) [seg_pos={0 0}, upload=188771]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 14 14:16:33.139454175 (214h48m13.119806811s) [upload=188771]: Done uploading segments... Dec 14 14:16:33.139495796 (214h48m13.119848423s) [upload=188771]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.516358868s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.............X.] duration: 184.035129ms remote: "tcp:[2620:cb:2000::1]:443" response headers: Content-Type=text/plain; charset=UTF-8 Content-Length=15 Cache-Control=private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Expires=Thu, 01 Jan 1970 00:00:01 GMT Cf-Ray=9ade7e5d7aae9963-FRA X-Frame-Options=SAMEORIGIN Date=Sun, 14 Dec 2025 14:46:56 GMT Referrer-Policy=same-origin Proxy-Status=Cloudflare-Proxy;error=http_response_incomplete target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/1244b8592b06e8c8 FAIL (2025-12-14 14:47:19 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "error code: 500" Duration: 184.081406ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [..............X] duration: 99.550284ms remote: "tcp:[2620:cb:2000::1]:443" response headers: Content-Type=text/plain; charset=UTF-8 Content-Length=15 Cache-Control=private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Expires=Thu, 01 Jan 1970 00:00:01 GMT Cf-Ray=9ade7e5d7aae9963-FRA X-Frame-Options=SAMEORIGIN Date=Sun, 14 Dec 2025 14:46:56 GMT Referrer-Policy=same-origin Proxy-Status=Cloudflare-Proxy;error=http_response_incomplete target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/1244b8592b06e8c8 FAIL (2025-12-14 14:46:56 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "error code: 500" Duration: 99.577018ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 2.4382214s trace: https://jaeger.dev.storj.io/trace/1751ea226e84ac5d FAIL (2025-12-14 14:46:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 14 14:16:03.322622529 (214h47m43.302975166s) [upload=188771]: Starting upload Dec 14 14:16:03.322698383 (214h47m43.303051027s) [upload=188771]: Done starting upload Dec 14 14:16:03.322957650 (214h47m43.303310288s) [upload=188771]: Uploading segments... Dec 14 14:16:03.323047310 (214h47m43.303399971s) [upload=188771]: Got next segment. Inline: false Dec 14 14:16:03.323095962 (214h47m43.30344861s) [upload=188771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 14 14:16:03.323104243 (214h47m43.303456893s) [upload=188771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 14 14:16:03.446699051 (214h47m43.427051694s) [seg_pos={0 0}, upload=188771]: Begin upload segment... Dec 14 14:16:03.446762560 (214h47m43.427115199s) [seg_pos={0 0}, upload=188771]: Done begin upload segment. Dec 14 14:16:03.446782966 (214h47m43.427135626s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446798773 (214h47m43.427151422s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446883480 (214h47m43.427236128s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.447750216 (214h47m43.428102851s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448003133 (214h47m43.428355855s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448960450 (214h47m43.429313088s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.450289219 (214h47m43.43064186s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451155877 (214h47m43.431508525s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451700169 (214h47m43.432052812s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453101406 (214h47m43.433454055s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453306320 (214h47m43.433658971s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453902924 (214h47m43.43425557s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.454485471 (214h47m43.434838121s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455040478 (214h47m43.435393125s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455598935 (214h47m43.435951585s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455867680 (214h47m43.436220329s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.861047945 (214h47m43.841400591s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:03.871023383 (214h47m43.851376028s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:04.080925197 (214h47m44.061277846s) [upload=188771]: Next returned nil segment Dec 14 14:16:04.080944439 (214h47m44.061297107s) [upload=188771]: Waiting for error group managing segments... Dec 14 14:16:04.209819284 (214h47m44.190171943s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.233005644 (214h47m44.213358285s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.234227859 (214h47m44.214580501s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239681369 (214h47m44.22003402s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239894423 (214h47m44.220247063s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.240454771 (214h47m44.220807417s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.241497930 (214h47m44.221850571s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.242469256 (214h47m44.222821901s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.243873927 (214h47m44.224226573s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.244902987 (214h47m44.225255626s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.246138337 (214h47m44.226490985s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.284989325 (214h47m44.265341963s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.787553216 (214h47m44.767905858s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:33.026981756 (214h48m13.007334401s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Dec 14 14:16:33.027035288 (214h48m13.007387915s) [seg_pos={0 0}, upload=188771]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 14 14:16:33.139454175 (214h48m13.119806811s) [upload=188771]: Done uploading segments... Dec 14 14:16:33.139495796 (214h48m13.119848423s) [upload=188771]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.43832894s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 2.420817711s trace: https://jaeger.dev.storj.io/trace/1751ea226e84ac5d FAIL (2025-12-14 14:46:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 14 14:16:03.322622529 (214h47m43.302975166s) [upload=188771]: Starting upload Dec 14 14:16:03.322698383 (214h47m43.303051027s) [upload=188771]: Done starting upload Dec 14 14:16:03.322957650 (214h47m43.303310288s) [upload=188771]: Uploading segments... Dec 14 14:16:03.323047310 (214h47m43.303399971s) [upload=188771]: Got next segment. Inline: false Dec 14 14:16:03.323095962 (214h47m43.30344861s) [upload=188771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 14 14:16:03.323104243 (214h47m43.303456893s) [upload=188771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 14 14:16:03.446699051 (214h47m43.427051694s) [seg_pos={0 0}, upload=188771]: Begin upload segment... Dec 14 14:16:03.446762560 (214h47m43.427115199s) [seg_pos={0 0}, upload=188771]: Done begin upload segment. Dec 14 14:16:03.446782966 (214h47m43.427135626s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446798773 (214h47m43.427151422s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446883480 (214h47m43.427236128s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.447750216 (214h47m43.428102851s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448003133 (214h47m43.428355855s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448960450 (214h47m43.429313088s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.450289219 (214h47m43.43064186s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451155877 (214h47m43.431508525s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451700169 (214h47m43.432052812s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453101406 (214h47m43.433454055s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453306320 (214h47m43.433658971s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453902924 (214h47m43.43425557s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.454485471 (214h47m43.434838121s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455040478 (214h47m43.435393125s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455598935 (214h47m43.435951585s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455867680 (214h47m43.436220329s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.861047945 (214h47m43.841400591s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:03.871023383 (214h47m43.851376028s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:04.080925197 (214h47m44.061277846s) [upload=188771]: Next returned nil segment Dec 14 14:16:04.080944439 (214h47m44.061297107s) [upload=188771]: Waiting for error group managing segments... Dec 14 14:16:04.209819284 (214h47m44.190171943s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.233005644 (214h47m44.213358285s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.234227859 (214h47m44.214580501s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239681369 (214h47m44.22003402s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239894423 (214h47m44.220247063s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.240454771 (214h47m44.220807417s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.241497930 (214h47m44.221850571s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.242469256 (214h47m44.222821901s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.243873927 (214h47m44.224226573s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.244902987 (214h47m44.225255626s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.246138337 (214h47m44.226490985s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.284989325 (214h47m44.265341963s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.787553216 (214h47m44.767905858s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:33.026981756 (214h48m13.007334401s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Dec 14 14:16:33.027035288 (214h48m13.007387915s) [seg_pos={0 0}, upload=188771]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 14 14:16:33.139454175 (214h48m13.119806811s) [upload=188771]: Done uploading segments... Dec 14 14:16:33.139495796 (214h48m13.119848423s) [upload=188771]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.420848188s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 2.3816382s trace: https://jaeger.dev.storj.io/trace/1751ea226e84ac5d FAIL (2025-12-14 14:45:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 14 14:16:03.322622529 (214h47m43.302975166s) [upload=188771]: Starting upload Dec 14 14:16:03.322698383 (214h47m43.303051027s) [upload=188771]: Done starting upload Dec 14 14:16:03.322957650 (214h47m43.303310288s) [upload=188771]: Uploading segments... Dec 14 14:16:03.323047310 (214h47m43.303399971s) [upload=188771]: Got next segment. Inline: false Dec 14 14:16:03.323095962 (214h47m43.30344861s) [upload=188771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 14 14:16:03.323104243 (214h47m43.303456893s) [upload=188771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 14 14:16:03.446699051 (214h47m43.427051694s) [seg_pos={0 0}, upload=188771]: Begin upload segment... Dec 14 14:16:03.446762560 (214h47m43.427115199s) [seg_pos={0 0}, upload=188771]: Done begin upload segment. Dec 14 14:16:03.446782966 (214h47m43.427135626s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446798773 (214h47m43.427151422s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446883480 (214h47m43.427236128s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.447750216 (214h47m43.428102851s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448003133 (214h47m43.428355855s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448960450 (214h47m43.429313088s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.450289219 (214h47m43.43064186s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451155877 (214h47m43.431508525s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451700169 (214h47m43.432052812s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453101406 (214h47m43.433454055s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453306320 (214h47m43.433658971s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453902924 (214h47m43.43425557s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.454485471 (214h47m43.434838121s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455040478 (214h47m43.435393125s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455598935 (214h47m43.435951585s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455867680 (214h47m43.436220329s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.861047945 (214h47m43.841400591s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:03.871023383 (214h47m43.851376028s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:04.080925197 (214h47m44.061277846s) [upload=188771]: Next returned nil segment Dec 14 14:16:04.080944439 (214h47m44.061297107s) [upload=188771]: Waiting for error group managing segments... Dec 14 14:16:04.209819284 (214h47m44.190171943s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.233005644 (214h47m44.213358285s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.234227859 (214h47m44.214580501s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239681369 (214h47m44.22003402s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239894423 (214h47m44.220247063s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.240454771 (214h47m44.220807417s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.241497930 (214h47m44.221850571s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.242469256 (214h47m44.222821901s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.243873927 (214h47m44.224226573s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.244902987 (214h47m44.225255626s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.246138337 (214h47m44.226490985s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.284989325 (214h47m44.265341963s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.787553216 (214h47m44.767905858s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:33.026981756 (214h48m13.007334401s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Dec 14 14:16:33.027035288 (214h48m13.007387915s) [seg_pos={0 0}, upload=188771]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 14 14:16:33.139454175 (214h48m13.119806811s) [upload=188771]: Done uploading segments... Dec 14 14:16:33.139495796 (214h48m13.119848423s) [upload=188771]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.381668039s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 2.346881105s trace: https://jaeger.dev.storj.io/trace/1751ea226e84ac5d FAIL (2025-12-14 14:44:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 14 14:16:03.322622529 (214h47m43.302975166s) [upload=188771]: Starting upload Dec 14 14:16:03.322698383 (214h47m43.303051027s) [upload=188771]: Done starting upload Dec 14 14:16:03.322957650 (214h47m43.303310288s) [upload=188771]: Uploading segments... Dec 14 14:16:03.323047310 (214h47m43.303399971s) [upload=188771]: Got next segment. Inline: false Dec 14 14:16:03.323095962 (214h47m43.30344861s) [upload=188771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 14 14:16:03.323104243 (214h47m43.303456893s) [upload=188771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 14 14:16:03.446699051 (214h47m43.427051694s) [seg_pos={0 0}, upload=188771]: Begin upload segment... Dec 14 14:16:03.446762560 (214h47m43.427115199s) [seg_pos={0 0}, upload=188771]: Done begin upload segment. Dec 14 14:16:03.446782966 (214h47m43.427135626s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446798773 (214h47m43.427151422s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446883480 (214h47m43.427236128s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.447750216 (214h47m43.428102851s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448003133 (214h47m43.428355855s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448960450 (214h47m43.429313088s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.450289219 (214h47m43.43064186s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451155877 (214h47m43.431508525s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451700169 (214h47m43.432052812s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453101406 (214h47m43.433454055s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453306320 (214h47m43.433658971s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453902924 (214h47m43.43425557s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.454485471 (214h47m43.434838121s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455040478 (214h47m43.435393125s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455598935 (214h47m43.435951585s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455867680 (214h47m43.436220329s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.861047945 (214h47m43.841400591s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:03.871023383 (214h47m43.851376028s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:04.080925197 (214h47m44.061277846s) [upload=188771]: Next returned nil segment Dec 14 14:16:04.080944439 (214h47m44.061297107s) [upload=188771]: Waiting for error group managing segments... Dec 14 14:16:04.209819284 (214h47m44.190171943s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.233005644 (214h47m44.213358285s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.234227859 (214h47m44.214580501s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239681369 (214h47m44.22003402s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239894423 (214h47m44.220247063s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.240454771 (214h47m44.220807417s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.241497930 (214h47m44.221850571s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.242469256 (214h47m44.222821901s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.243873927 (214h47m44.224226573s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.244902987 (214h47m44.225255626s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.246138337 (214h47m44.226490985s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.284989325 (214h47m44.265341963s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.787553216 (214h47m44.767905858s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:33.026981756 (214h48m13.007334401s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Dec 14 14:16:33.027035288 (214h48m13.007387915s) [seg_pos={0 0}, upload=188771]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 14 14:16:33.139454175 (214h48m13.119806811s) [upload=188771]: Done uploading segments... Dec 14 14:16:33.139495796 (214h48m13.119848423s) [upload=188771]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.346910755s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 2.331783963s trace: https://jaeger.dev.storj.io/trace/1751ea226e84ac5d FAIL (2025-12-14 14:43:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 14 14:16:03.322622529 (214h47m43.302975166s) [upload=188771]: Starting upload Dec 14 14:16:03.322698383 (214h47m43.303051027s) [upload=188771]: Done starting upload Dec 14 14:16:03.322957650 (214h47m43.303310288s) [upload=188771]: Uploading segments... Dec 14 14:16:03.323047310 (214h47m43.303399971s) [upload=188771]: Got next segment. Inline: false Dec 14 14:16:03.323095962 (214h47m43.30344861s) [upload=188771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 14 14:16:03.323104243 (214h47m43.303456893s) [upload=188771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 14 14:16:03.446699051 (214h47m43.427051694s) [seg_pos={0 0}, upload=188771]: Begin upload segment... Dec 14 14:16:03.446762560 (214h47m43.427115199s) [seg_pos={0 0}, upload=188771]: Done begin upload segment. Dec 14 14:16:03.446782966 (214h47m43.427135626s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446798773 (214h47m43.427151422s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446883480 (214h47m43.427236128s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.447750216 (214h47m43.428102851s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448003133 (214h47m43.428355855s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448960450 (214h47m43.429313088s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.450289219 (214h47m43.43064186s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451155877 (214h47m43.431508525s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451700169 (214h47m43.432052812s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453101406 (214h47m43.433454055s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453306320 (214h47m43.433658971s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453902924 (214h47m43.43425557s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.454485471 (214h47m43.434838121s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455040478 (214h47m43.435393125s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455598935 (214h47m43.435951585s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455867680 (214h47m43.436220329s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.861047945 (214h47m43.841400591s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:03.871023383 (214h47m43.851376028s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:04.080925197 (214h47m44.061277846s) [upload=188771]: Next returned nil segment Dec 14 14:16:04.080944439 (214h47m44.061297107s) [upload=188771]: Waiting for error group managing segments... Dec 14 14:16:04.209819284 (214h47m44.190171943s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.233005644 (214h47m44.213358285s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.234227859 (214h47m44.214580501s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239681369 (214h47m44.22003402s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239894423 (214h47m44.220247063s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.240454771 (214h47m44.220807417s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.241497930 (214h47m44.221850571s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.242469256 (214h47m44.222821901s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.243873927 (214h47m44.224226573s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.244902987 (214h47m44.225255626s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.246138337 (214h47m44.226490985s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.284989325 (214h47m44.265341963s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.787553216 (214h47m44.767905858s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:33.026981756 (214h48m13.007334401s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Dec 14 14:16:33.027035288 (214h48m13.007387915s) [seg_pos={0 0}, upload=188771]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 14 14:16:33.139454175 (214h48m13.119806811s) [upload=188771]: Done uploading segments... Dec 14 14:16:33.139495796 (214h48m13.119848423s) [upload=188771]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.331816542s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 2.508647455s trace: https://jaeger.dev.storj.io/trace/1751ea226e84ac5d FAIL (2025-12-14 14:42:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 14 14:16:03.322622529 (214h47m43.302975166s) [upload=188771]: Starting upload Dec 14 14:16:03.322698383 (214h47m43.303051027s) [upload=188771]: Done starting upload Dec 14 14:16:03.322957650 (214h47m43.303310288s) [upload=188771]: Uploading segments... Dec 14 14:16:03.323047310 (214h47m43.303399971s) [upload=188771]: Got next segment. Inline: false Dec 14 14:16:03.323095962 (214h47m43.30344861s) [upload=188771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 14 14:16:03.323104243 (214h47m43.303456893s) [upload=188771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 14 14:16:03.446699051 (214h47m43.427051694s) [seg_pos={0 0}, upload=188771]: Begin upload segment... Dec 14 14:16:03.446762560 (214h47m43.427115199s) [seg_pos={0 0}, upload=188771]: Done begin upload segment. Dec 14 14:16:03.446782966 (214h47m43.427135626s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446798773 (214h47m43.427151422s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.446883480 (214h47m43.427236128s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.447750216 (214h47m43.428102851s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448003133 (214h47m43.428355855s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.448960450 (214h47m43.429313088s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.450289219 (214h47m43.43064186s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451155877 (214h47m43.431508525s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.451700169 (214h47m43.432052812s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453101406 (214h47m43.433454055s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453306320 (214h47m43.433658971s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.453902924 (214h47m43.43425557s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.454485471 (214h47m43.434838121s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455040478 (214h47m43.435393125s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455598935 (214h47m43.435951585s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.455867680 (214h47m43.436220329s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Uploading piece... Dec 14 14:16:03.861047945 (214h47m43.841400591s) [noise=true , address=77.90.31.12:4589 , piece_id=H4KRSS3JIDMWH3XPAECKFWD5TLV667ID2ZSZ7SQ4JHRV2RS3E2ZQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:03.871023383 (214h47m43.851376028s) [noise=true , address=77.90.31.19:4841 , piece_id=R2PNSFVNSXCIKEKQNTIUOKZ5OPHPK6NU6EE3K54FAGOIG2NE6ENQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:04.080925197 (214h47m44.061277846s) [upload=188771]: Next returned nil segment Dec 14 14:16:04.080944439 (214h47m44.061297107s) [upload=188771]: Waiting for error group managing segments... Dec 14 14:16:04.209819284 (214h47m44.190171943s) [noise=true , address=145.79.227.47:4265 , piece_id=3QHWAYPXPG34D2QAFXMRIXLUBAX3SVTVCLYR3DTJ6WRWEPOU3PKQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.233005644 (214h47m44.213358285s) [noise=true , address=145.79.227.38:4979 , piece_id=ZZXBSNY2YNSNNXTBGCVI2J27VG4DYOWZWHCB3BZG6KCQXWGEZNFQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.234227859 (214h47m44.214580501s) [noise=true , address=145.79.227.43:4158 , piece_id=RLAN2IHF643NXKXXREGGR5BFI5GYDBCD6UF27YJRFP5M7HPUQ4YQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239681369 (214h47m44.22003402s) [noise=true , address=145.79.227.45:4230 , piece_id=HKMA7J4GJPBAS2L3W64H7WG7H3Z2JZTPBN3E5PI4L4WSGSAEIGFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.239894423 (214h47m44.220247063s) [noise=true , address=145.79.227.48:4302 , piece_id=3YPBISDZHUXSROSFRMCJHQDPPCQ3CG3IKQRBXLHOZIYVDOGC7BTQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.240454771 (214h47m44.220807417s) [noise=true , address=145.79.227.51:4411 , piece_id=VCGINQEQMBEB4PUMBQOXB62UZJ7AXEN7SUSJOS45RGTV4SBIKUYQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.241497930 (214h47m44.221850571s) [noise=true , address=145.79.227.26:4655 , piece_id=3T2RNV6KR3LUYI4AY2HJ2ABHQTELURMGS6WSGZW6UQQUDPZW2GQQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.242469256 (214h47m44.222821901s) [noise=true , address=145.79.227.49:4337 , piece_id=JAA6NLOVKMU2JNEE2GENAQMRODTXVEP6FI3ZXNV63FY7LF3RYYFA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.243873927 (214h47m44.224226573s) [noise=true , address=145.79.227.51:4409 , piece_id=JPCGGAZ7GOZ6XBE4RDYV6W6JBDCYHDQ5SWUH3C5N5QC74I2E2OXQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.244902987 (214h47m44.225255626s) [noise=true , address=77.90.31.21:4877 , piece_id=DLV3ZFIPCHKY3VC5HCPPYIFY527EGI7VUXGM5URKQRLI35QFDBLA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.246138337 (214h47m44.226490985s) [noise=true , address=77.90.31.10:4520 , piece_id=UGV5ITABBYJWVGJS3MN2VUU7IAGX4L5D6NLIHSVZANWUMGGEV2OQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.284989325 (214h47m44.265341963s) [noise=true , address=145.79.227.32:4835 , piece_id=AAS74W2QEXQHBTQSLASQZQSDRKB25ONCUJU4Q44J3KTSIIKGUTDQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: <nil> Dec 14 14:16:04.787553216 (214h47m44.767905858s) [noise=true , address=77.90.31.12:4590 , piece_id=X55W4DSDFO6F4R3YS5EKB3DH4FCBZOFLFC2IH2SDD6AMKPXDSZIQ, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000 Dec 14 14:16:33.026981756 (214h48m13.007334401s) [noise=true , address=77.90.31.16:4734 , piece_id=KQCIKRMDXIY5LUPJVINOLAU6TVURI5Z5MOFC7IDRZKILR2JIN2OA, seg_pos={0 0}, upload=188771]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Dec 14 14:16:33.027035288 (214h48m13.007387915s) [seg_pos={0 0}, upload=188771]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 14 14:16:33.139454175 (214h48m13.119806811s) [upload=188771]: Done uploading segments... Dec 14 14:16:33.139495796 (214h48m13.119848423s) [upload=188771]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.508678876s core uplink us-select-1

Orbiter Check History

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)