Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..........................................X.................] duration:
1.689665172s trace:
https://jaeger.dev.storj.io/trace/519089b67516ae45
FAIL (2026-04-11 01:34:35 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.69019118s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...........................................X................] duration:
1.579734155s trace:
https://jaeger.dev.storj.io/trace/519089b67516ae45
FAIL (2026-04-11 01:33:46 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.579826453s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [............................................X...............] duration:
2.211298386s trace:
https://jaeger.dev.storj.io/trace/519089b67516ae45
FAIL (2026-04-11 01:33:17 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 2.211359867s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.............................................X..............] duration:
1.669467003s trace:
https://jaeger.dev.storj.io/trace/519089b67516ae45
FAIL (2026-04-11 01:32:40 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.66961853s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..............................................X.............] duration:
1.604826957s trace:
https://jaeger.dev.storj.io/trace/519089b67516ae45
FAIL (2026-04-11 01:31:57 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.604858557s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...............................................X............] duration:
1.662267404s trace:
https://jaeger.dev.storj.io/trace/519089b67516ae45
FAIL (2026-04-11 01:31:36 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.662491944s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [................................................X...........] duration:
1.807774063s trace:
https://jaeger.dev.storj.io/trace/519089b67516ae45
FAIL (2026-04-11 01:31:13 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.807797386s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.................................................X..........] duration:
2.198472679s trace:
https://jaeger.dev.storj.io/trace/519089b67516ae45
FAIL (2026-04-11 01:30:30 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 2.19858177s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..................................................X.........] duration:
1.743347076s trace:
https://jaeger.dev.storj.io/trace/519089b67516ae45
FAIL (2026-04-11 01:30:00 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.74340225s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...................................................X........] duration:
1.713239022s trace:
https://jaeger.dev.storj.io/trace/519089b67516ae45
FAIL (2026-04-11 01:29:27 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.713282351s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [....................................................X.......] duration:
2.176220201s trace:
https://jaeger.dev.storj.io/trace/519089b67516ae45
FAIL (2026-04-11 01:29:11 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 2.176248677s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.....................................................X......] duration:
1.619781037s trace:
https://jaeger.dev.storj.io/trace/519089b67516ae45
FAIL (2026-04-11 01:28:21 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.619810334s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [......................................................X.....] duration:
2.286236792s trace:
https://jaeger.dev.storj.io/trace/519089b67516ae45
FAIL (2026-04-11 01:27:25 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 2.286273065s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.......................................................X....] duration:
2.157452314s trace:
https://jaeger.dev.storj.io/trace/519089b67516ae45
FAIL (2026-04-11 01:27:00 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 2.157501823s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [........................................................X...] duration:
2.157194718s trace:
https://jaeger.dev.storj.io/trace/519089b67516ae45
FAIL (2026-04-11 01:25:49 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 2.157211652s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.........................................................X..] duration:
1.830220944s trace:
https://jaeger.dev.storj.io/trace/519089b67516ae45
FAIL (2026-04-11 01:25:29 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.830255218s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..........................................................X.] duration:
1.741289064s trace:
https://jaeger.dev.storj.io/trace/519089b67516ae45
FAIL (2026-04-11 01:24:28 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.741319888s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...........................................................X] duration:
5.001164706s trace:
https://jaeger.dev.storj.io/trace/519089b67516ae45
FAIL (2026-04-11 01:23:19 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 5.001187429s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [X...........................................................] duration:
2.121146291s trace:
https://jaeger.dev.storj.io/trace/62f95f2197417e9a
FAIL (2026-04-11 01:19:14 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.121171047s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
3.055636238s trace:
https://jaeger.dev.storj.io/trace/46130f6078efa43c
FAIL (2026-04-11 01:18:45 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 11)
======LOGS======
ad=77069]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Apr 11 00:34:43.794844927 (80h17m54.388956169s) [seg_pos={0 0}, upload=77069]: Begin upload segment...
Apr 11 00:34:43.794911260 (80h17m54.389022502s) [seg_pos={0 0}, upload=77069]: Done begin upload segment.
Apr 11 00:34:43.795033432 (80h17m54.389144662s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.795106843 (80h17m54.389218085s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796325376 (80h17m54.390436621s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796745253 (80h17m54.390856505s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797331690 (80h17m54.39144294s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797656318 (80h17m54.391767558s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798142741 (80h17m54.392253986s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798157151 (80h17m54.392268399s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798678244 (80h17m54.392789496s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799132157 (80h17m54.393243403s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799320325 (80h17m54.393431558s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799544546 (80h17m54.393655788s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799924439 (80h17m54.394035679s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.802551628 (80h17m54.396662878s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.803497747 (80h17m54.39760899s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.804533930 (80h17m54.39864517s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.805383432 (80h17m54.399494676s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.807796955 (80h17m54.401908205s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:44.421299600 (80h17m55.015410843s) [upload=77069]: Next returned nil segment
Apr 11 00:34:44.421345070 (80h17m55.015456316s) [upload=77069]: Waiting for error group managing segments...
Apr 11 00:34:44.582519405 (80h17m55.176630646s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.597152765 (80h17m55.191264015s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.864642755 (80h17m55.458753998s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.872186316 (80h17m55.466297574s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.026075990 (80h17m55.62018723s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.028603622 (80h17m55.622714861s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.035500804 (80h17m55.629612044s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.039638177 (80h17m55.633749417s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.041344727 (80h17m55.63545598s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.402114414 (80h17m55.996225664s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:46.503621244 (80h17m57.097732493s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:35:43.391436872 (80h18m53.985548124s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:127RhN3jMBTt6LXsgym5RNMM4CvjhcSRhgjbRvSpE1TQTgQdJi9, address:145.79.227.9:4381): protocol: rpc: dial tcp 145.79.227.9:4381: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.392202791 (80h18m53.986314041s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1Rj4hUGWvUPuU49QDsHYhibiJhyY5KDGus1rzNeP7KusgyvTsU, address:145.79.227.23:4568): protocol: rpc: dial tcp 145.79.227.23:4568: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393141126 (80h18m53.987252383s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:122z4Fys9DYGB3WnjvjiZ5TSA3h8dXvMFcaFfYdpMtxGT6ZiiGH, address:145.79.227.44:4218): protocol: rpc: dial tcp 145.79.227.44:4218: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393196062 (80h18m53.987307312s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4, address:145.79.227.30:4768): protocol: rpc: dial tcp 145.79.227.30:4768: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393473160 (80h18m53.987584419s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1q6zcreD1FCjZXuYowJz9amfcQtWySL2iyrFLU8BcPzbD8PhPb, address:145.79.227.37:4597): protocol: rpc: dial tcp 145.79.227.37:4597: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393616973 (80h18m53.987728225s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:12EMGZ4hpXmBp4TwemLrbKAVtjBmtexMzEahRTzFw42qQipsAG, address:145.79.227.16:4918): protocol: rpc: dial tcp 145.79.227.16:4918: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.395024614 (80h18m53.989135865s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc, address:145.79.227.33:4882): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:43.395084251 (80h18m53.989195495s) [seg_pos={0 0}, upload=77069]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil>
Apr 11 00:35:43.502941515 (80h18m54.097052761s) [upload=77069]: Done uploading segments...
Apr 11 00:35:43.503021418 (80h18m54.097132663s) [upload=77069]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11)
Duration: 3.055663526s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
2.287040634s trace:
https://jaeger.dev.storj.io/trace/2b156773aff13a05
FAIL (2026-04-11 01:18:27 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 4)
======LOGS======
os={0 0}, upload=77081]: Done begin upload segment.
Apr 11 00:35:36.607753362 (80h18m47.201864604s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609152089 (80h18m47.203263338s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609360181 (80h18m47.203471424s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609753819 (80h18m47.203865081s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610072200 (80h18m47.20418344s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610268942 (80h18m47.20438019s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610574800 (80h18m47.204686042s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610731353 (80h18m47.204842599s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611095332 (80h18m47.205206571s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611192823 (80h18m47.205304062s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611615833 (80h18m47.205727076s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611772854 (80h18m47.2058841s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612002035 (80h18m47.206113277s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612449991 (80h18m47.206561233s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612488077 (80h18m47.206599327s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.614557840 (80h18m47.208669079s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:37.207182182 (80h18m47.801293437s) [upload=77081]: Next returned nil segment
Apr 11 00:35:37.207198012 (80h18m47.801309263s) [upload=77081]: Waiting for error group managing segments...
Apr 11 00:35:37.354583621 (80h18m47.948694872s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.355618116 (80h18m47.949729356s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.363570054 (80h18m47.957681298s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.380605963 (80h18m47.974717212s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:36:06.234067450 (80h19m16.828178701s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1DmQeoQWTnFhQeJFseJoNYzFyo7vZjYKwDtQVJ6pomksWExsTb, address:145.79.227.45:4233): protocol: rpc: dial tcp 145.79.227.45:4233: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234463037 (80h19m16.828574286s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1KrucHjbKjhBZEUGd2p8cmfyCwm6rx4BVwwVhkpgnEv9EQiJRB, address:145.79.227.10:4406): protocol: rpc: dial tcp 145.79.227.10:4406: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234715782 (80h19m16.828827038s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234764656 (80h19m16.828875916s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: rpc: dial tcp 145.79.227.48:4304: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.235062285 (80h19m16.829173536s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: rpc: dial tcp 145.79.227.15:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237352242 (80h19m16.831463499s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, address:145.79.227.7:4297): protocol: rpc: dial tcp 145.79.227.7:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237418191 (80h19m16.831529449s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: rpc: dial tcp 145.79.227.4:4188: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238026103 (80h19m16.832137353s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw, address:145.79.227.25:4617): protocol: rpc: dial tcp 145.79.227.25:4617: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238136590 (80h19m16.832247826s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: rpc: dial tcp 145.79.227.46:4604: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238167799 (80h19m16.832279058s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12t9rGfCQ7bLkNnZzaPVgkE2Y9rXsiFp1XGdeECzv9M1jk8Qqmp, address:145.79.227.19:4081): protocol: rpc: dial tcp 145.79.227.19:4081: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238878716 (80h19m16.832989974s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12MnvtjSKHuNKUcVEjTviEGThRKQxzmGiMu1ofCwkv8BLAhGuzL, address:145.79.227.50:4376): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239664077 (80h19m16.833775331s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:128nLffK44ioiTpjWjAm2vHFTLouLZdw7BijqBAbsdjo99kPYNo, address:145.79.227.5:4224): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239722363 (80h19m16.833833613s) [seg_pos={0 0}, upload=77081]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil>
Apr 11 00:36:06.339820291 (80h19m16.93393154s) [upload=77081]: Done uploading segments...
Apr 11 00:36:06.339893921 (80h19m16.934005173s) [upload=77081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4)
Duration: 2.287067623s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [XX..........................................................] duration:
1.674738075s trace:
https://jaeger.dev.storj.io/trace/62f95f2197417e9a
FAIL (2026-04-11 01:18:25 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.674795066s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
2.949006768s trace:
https://jaeger.dev.storj.io/trace/46130f6078efa43c
FAIL (2026-04-11 01:18:22 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 11)
======LOGS======
ad=77069]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Apr 11 00:34:43.794844927 (80h17m54.388956169s) [seg_pos={0 0}, upload=77069]: Begin upload segment...
Apr 11 00:34:43.794911260 (80h17m54.389022502s) [seg_pos={0 0}, upload=77069]: Done begin upload segment.
Apr 11 00:34:43.795033432 (80h17m54.389144662s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.795106843 (80h17m54.389218085s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796325376 (80h17m54.390436621s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796745253 (80h17m54.390856505s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797331690 (80h17m54.39144294s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797656318 (80h17m54.391767558s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798142741 (80h17m54.392253986s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798157151 (80h17m54.392268399s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798678244 (80h17m54.392789496s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799132157 (80h17m54.393243403s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799320325 (80h17m54.393431558s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799544546 (80h17m54.393655788s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799924439 (80h17m54.394035679s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.802551628 (80h17m54.396662878s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.803497747 (80h17m54.39760899s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.804533930 (80h17m54.39864517s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.805383432 (80h17m54.399494676s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.807796955 (80h17m54.401908205s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:44.421299600 (80h17m55.015410843s) [upload=77069]: Next returned nil segment
Apr 11 00:34:44.421345070 (80h17m55.015456316s) [upload=77069]: Waiting for error group managing segments...
Apr 11 00:34:44.582519405 (80h17m55.176630646s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.597152765 (80h17m55.191264015s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.864642755 (80h17m55.458753998s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.872186316 (80h17m55.466297574s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.026075990 (80h17m55.62018723s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.028603622 (80h17m55.622714861s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.035500804 (80h17m55.629612044s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.039638177 (80h17m55.633749417s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.041344727 (80h17m55.63545598s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.402114414 (80h17m55.996225664s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:46.503621244 (80h17m57.097732493s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:35:43.391436872 (80h18m53.985548124s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:127RhN3jMBTt6LXsgym5RNMM4CvjhcSRhgjbRvSpE1TQTgQdJi9, address:145.79.227.9:4381): protocol: rpc: dial tcp 145.79.227.9:4381: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.392202791 (80h18m53.986314041s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1Rj4hUGWvUPuU49QDsHYhibiJhyY5KDGus1rzNeP7KusgyvTsU, address:145.79.227.23:4568): protocol: rpc: dial tcp 145.79.227.23:4568: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393141126 (80h18m53.987252383s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:122z4Fys9DYGB3WnjvjiZ5TSA3h8dXvMFcaFfYdpMtxGT6ZiiGH, address:145.79.227.44:4218): protocol: rpc: dial tcp 145.79.227.44:4218: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393196062 (80h18m53.987307312s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4, address:145.79.227.30:4768): protocol: rpc: dial tcp 145.79.227.30:4768: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393473160 (80h18m53.987584419s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1q6zcreD1FCjZXuYowJz9amfcQtWySL2iyrFLU8BcPzbD8PhPb, address:145.79.227.37:4597): protocol: rpc: dial tcp 145.79.227.37:4597: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393616973 (80h18m53.987728225s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:12EMGZ4hpXmBp4TwemLrbKAVtjBmtexMzEahRTzFw42qQipsAG, address:145.79.227.16:4918): protocol: rpc: dial tcp 145.79.227.16:4918: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.395024614 (80h18m53.989135865s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc, address:145.79.227.33:4882): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:43.395084251 (80h18m53.989195495s) [seg_pos={0 0}, upload=77069]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil>
Apr 11 00:35:43.502941515 (80h18m54.097052761s) [upload=77069]: Done uploading segments...
Apr 11 00:35:43.503021418 (80h18m54.097132663s) [upload=77069]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11)
Duration: 2.949032304s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
3.089440502s trace:
https://jaeger.dev.storj.io/trace/46130f6078efa43c
FAIL (2026-04-11 01:17:54 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 11)
======LOGS======
ad=77069]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Apr 11 00:34:43.794844927 (80h17m54.388956169s) [seg_pos={0 0}, upload=77069]: Begin upload segment...
Apr 11 00:34:43.794911260 (80h17m54.389022502s) [seg_pos={0 0}, upload=77069]: Done begin upload segment.
Apr 11 00:34:43.795033432 (80h17m54.389144662s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.795106843 (80h17m54.389218085s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796325376 (80h17m54.390436621s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796745253 (80h17m54.390856505s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797331690 (80h17m54.39144294s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797656318 (80h17m54.391767558s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798142741 (80h17m54.392253986s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798157151 (80h17m54.392268399s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798678244 (80h17m54.392789496s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799132157 (80h17m54.393243403s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799320325 (80h17m54.393431558s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799544546 (80h17m54.393655788s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799924439 (80h17m54.394035679s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.802551628 (80h17m54.396662878s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.803497747 (80h17m54.39760899s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.804533930 (80h17m54.39864517s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.805383432 (80h17m54.399494676s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.807796955 (80h17m54.401908205s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:44.421299600 (80h17m55.015410843s) [upload=77069]: Next returned nil segment
Apr 11 00:34:44.421345070 (80h17m55.015456316s) [upload=77069]: Waiting for error group managing segments...
Apr 11 00:34:44.582519405 (80h17m55.176630646s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.597152765 (80h17m55.191264015s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.864642755 (80h17m55.458753998s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.872186316 (80h17m55.466297574s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.026075990 (80h17m55.62018723s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.028603622 (80h17m55.622714861s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.035500804 (80h17m55.629612044s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.039638177 (80h17m55.633749417s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.041344727 (80h17m55.63545598s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.402114414 (80h17m55.996225664s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:46.503621244 (80h17m57.097732493s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:35:43.391436872 (80h18m53.985548124s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:127RhN3jMBTt6LXsgym5RNMM4CvjhcSRhgjbRvSpE1TQTgQdJi9, address:145.79.227.9:4381): protocol: rpc: dial tcp 145.79.227.9:4381: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.392202791 (80h18m53.986314041s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1Rj4hUGWvUPuU49QDsHYhibiJhyY5KDGus1rzNeP7KusgyvTsU, address:145.79.227.23:4568): protocol: rpc: dial tcp 145.79.227.23:4568: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393141126 (80h18m53.987252383s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:122z4Fys9DYGB3WnjvjiZ5TSA3h8dXvMFcaFfYdpMtxGT6ZiiGH, address:145.79.227.44:4218): protocol: rpc: dial tcp 145.79.227.44:4218: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393196062 (80h18m53.987307312s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4, address:145.79.227.30:4768): protocol: rpc: dial tcp 145.79.227.30:4768: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393473160 (80h18m53.987584419s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1q6zcreD1FCjZXuYowJz9amfcQtWySL2iyrFLU8BcPzbD8PhPb, address:145.79.227.37:4597): protocol: rpc: dial tcp 145.79.227.37:4597: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393616973 (80h18m53.987728225s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:12EMGZ4hpXmBp4TwemLrbKAVtjBmtexMzEahRTzFw42qQipsAG, address:145.79.227.16:4918): protocol: rpc: dial tcp 145.79.227.16:4918: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.395024614 (80h18m53.989135865s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc, address:145.79.227.33:4882): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:43.395084251 (80h18m53.989195495s) [seg_pos={0 0}, upload=77069]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil>
Apr 11 00:35:43.502941515 (80h18m54.097052761s) [upload=77069]: Done uploading segments...
Apr 11 00:35:43.503021418 (80h18m54.097132663s) [upload=77069]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11)
Duration: 3.089468814s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [XX..........................................................] duration:
2.351208534s trace:
https://jaeger.dev.storj.io/trace/2b156773aff13a05
FAIL (2026-04-11 01:17:26 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 4)
======LOGS======
os={0 0}, upload=77081]: Done begin upload segment.
Apr 11 00:35:36.607753362 (80h18m47.201864604s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609152089 (80h18m47.203263338s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609360181 (80h18m47.203471424s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609753819 (80h18m47.203865081s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610072200 (80h18m47.20418344s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610268942 (80h18m47.20438019s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610574800 (80h18m47.204686042s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610731353 (80h18m47.204842599s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611095332 (80h18m47.205206571s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611192823 (80h18m47.205304062s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611615833 (80h18m47.205727076s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611772854 (80h18m47.2058841s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612002035 (80h18m47.206113277s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612449991 (80h18m47.206561233s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612488077 (80h18m47.206599327s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.614557840 (80h18m47.208669079s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:37.207182182 (80h18m47.801293437s) [upload=77081]: Next returned nil segment
Apr 11 00:35:37.207198012 (80h18m47.801309263s) [upload=77081]: Waiting for error group managing segments...
Apr 11 00:35:37.354583621 (80h18m47.948694872s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.355618116 (80h18m47.949729356s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.363570054 (80h18m47.957681298s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.380605963 (80h18m47.974717212s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:36:06.234067450 (80h19m16.828178701s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1DmQeoQWTnFhQeJFseJoNYzFyo7vZjYKwDtQVJ6pomksWExsTb, address:145.79.227.45:4233): protocol: rpc: dial tcp 145.79.227.45:4233: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234463037 (80h19m16.828574286s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1KrucHjbKjhBZEUGd2p8cmfyCwm6rx4BVwwVhkpgnEv9EQiJRB, address:145.79.227.10:4406): protocol: rpc: dial tcp 145.79.227.10:4406: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234715782 (80h19m16.828827038s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234764656 (80h19m16.828875916s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: rpc: dial tcp 145.79.227.48:4304: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.235062285 (80h19m16.829173536s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: rpc: dial tcp 145.79.227.15:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237352242 (80h19m16.831463499s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, address:145.79.227.7:4297): protocol: rpc: dial tcp 145.79.227.7:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237418191 (80h19m16.831529449s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: rpc: dial tcp 145.79.227.4:4188: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238026103 (80h19m16.832137353s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw, address:145.79.227.25:4617): protocol: rpc: dial tcp 145.79.227.25:4617: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238136590 (80h19m16.832247826s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: rpc: dial tcp 145.79.227.46:4604: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238167799 (80h19m16.832279058s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12t9rGfCQ7bLkNnZzaPVgkE2Y9rXsiFp1XGdeECzv9M1jk8Qqmp, address:145.79.227.19:4081): protocol: rpc: dial tcp 145.79.227.19:4081: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238878716 (80h19m16.832989974s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12MnvtjSKHuNKUcVEjTviEGThRKQxzmGiMu1ofCwkv8BLAhGuzL, address:145.79.227.50:4376): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239664077 (80h19m16.833775331s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:128nLffK44ioiTpjWjAm2vHFTLouLZdw7BijqBAbsdjo99kPYNo, address:145.79.227.5:4224): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239722363 (80h19m16.833833613s) [seg_pos={0 0}, upload=77081]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil>
Apr 11 00:36:06.339820291 (80h19m16.93393154s) [upload=77081]: Done uploading segments...
Apr 11 00:36:06.339893921 (80h19m16.934005173s) [upload=77081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4)
Duration: 2.351413427s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [.XX.........................................................] duration:
2.219983232s trace:
https://jaeger.dev.storj.io/trace/62f95f2197417e9a
FAIL (2026-04-11 01:17:19 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.220016241s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
3.401924985s trace:
https://jaeger.dev.storj.io/trace/46130f6078efa43c
FAIL (2026-04-11 01:17:00 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 11)
======LOGS======
ad=77069]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Apr 11 00:34:43.794844927 (80h17m54.388956169s) [seg_pos={0 0}, upload=77069]: Begin upload segment...
Apr 11 00:34:43.794911260 (80h17m54.389022502s) [seg_pos={0 0}, upload=77069]: Done begin upload segment.
Apr 11 00:34:43.795033432 (80h17m54.389144662s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.795106843 (80h17m54.389218085s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796325376 (80h17m54.390436621s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796745253 (80h17m54.390856505s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797331690 (80h17m54.39144294s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797656318 (80h17m54.391767558s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798142741 (80h17m54.392253986s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798157151 (80h17m54.392268399s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798678244 (80h17m54.392789496s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799132157 (80h17m54.393243403s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799320325 (80h17m54.393431558s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799544546 (80h17m54.393655788s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799924439 (80h17m54.394035679s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.802551628 (80h17m54.396662878s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.803497747 (80h17m54.39760899s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.804533930 (80h17m54.39864517s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.805383432 (80h17m54.399494676s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.807796955 (80h17m54.401908205s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:44.421299600 (80h17m55.015410843s) [upload=77069]: Next returned nil segment
Apr 11 00:34:44.421345070 (80h17m55.015456316s) [upload=77069]: Waiting for error group managing segments...
Apr 11 00:34:44.582519405 (80h17m55.176630646s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.597152765 (80h17m55.191264015s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.864642755 (80h17m55.458753998s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.872186316 (80h17m55.466297574s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.026075990 (80h17m55.62018723s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.028603622 (80h17m55.622714861s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.035500804 (80h17m55.629612044s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.039638177 (80h17m55.633749417s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.041344727 (80h17m55.63545598s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.402114414 (80h17m55.996225664s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:46.503621244 (80h17m57.097732493s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:35:43.391436872 (80h18m53.985548124s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:127RhN3jMBTt6LXsgym5RNMM4CvjhcSRhgjbRvSpE1TQTgQdJi9, address:145.79.227.9:4381): protocol: rpc: dial tcp 145.79.227.9:4381: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.392202791 (80h18m53.986314041s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1Rj4hUGWvUPuU49QDsHYhibiJhyY5KDGus1rzNeP7KusgyvTsU, address:145.79.227.23:4568): protocol: rpc: dial tcp 145.79.227.23:4568: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393141126 (80h18m53.987252383s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:122z4Fys9DYGB3WnjvjiZ5TSA3h8dXvMFcaFfYdpMtxGT6ZiiGH, address:145.79.227.44:4218): protocol: rpc: dial tcp 145.79.227.44:4218: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393196062 (80h18m53.987307312s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4, address:145.79.227.30:4768): protocol: rpc: dial tcp 145.79.227.30:4768: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393473160 (80h18m53.987584419s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1q6zcreD1FCjZXuYowJz9amfcQtWySL2iyrFLU8BcPzbD8PhPb, address:145.79.227.37:4597): protocol: rpc: dial tcp 145.79.227.37:4597: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393616973 (80h18m53.987728225s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:12EMGZ4hpXmBp4TwemLrbKAVtjBmtexMzEahRTzFw42qQipsAG, address:145.79.227.16:4918): protocol: rpc: dial tcp 145.79.227.16:4918: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.395024614 (80h18m53.989135865s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc, address:145.79.227.33:4882): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:43.395084251 (80h18m53.989195495s) [seg_pos={0 0}, upload=77069]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil>
Apr 11 00:35:43.502941515 (80h18m54.097052761s) [upload=77069]: Done uploading segments...
Apr 11 00:35:43.503021418 (80h18m54.097132663s) [upload=77069]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11)
Duration: 3.40195356s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [..XX........................................................] duration:
1.715587031s trace:
https://jaeger.dev.storj.io/trace/62f95f2197417e9a
FAIL (2026-04-11 01:16:59 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.715624162s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.XX.........................................................] duration:
2.34166971s trace:
https://jaeger.dev.storj.io/trace/2b156773aff13a05
FAIL (2026-04-11 01:16:37 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 4)
======LOGS======
os={0 0}, upload=77081]: Done begin upload segment.
Apr 11 00:35:36.607753362 (80h18m47.201864604s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609152089 (80h18m47.203263338s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609360181 (80h18m47.203471424s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609753819 (80h18m47.203865081s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610072200 (80h18m47.20418344s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610268942 (80h18m47.20438019s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610574800 (80h18m47.204686042s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610731353 (80h18m47.204842599s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611095332 (80h18m47.205206571s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611192823 (80h18m47.205304062s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611615833 (80h18m47.205727076s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611772854 (80h18m47.2058841s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612002035 (80h18m47.206113277s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612449991 (80h18m47.206561233s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612488077 (80h18m47.206599327s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.614557840 (80h18m47.208669079s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:37.207182182 (80h18m47.801293437s) [upload=77081]: Next returned nil segment
Apr 11 00:35:37.207198012 (80h18m47.801309263s) [upload=77081]: Waiting for error group managing segments...
Apr 11 00:35:37.354583621 (80h18m47.948694872s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.355618116 (80h18m47.949729356s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.363570054 (80h18m47.957681298s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.380605963 (80h18m47.974717212s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:36:06.234067450 (80h19m16.828178701s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1DmQeoQWTnFhQeJFseJoNYzFyo7vZjYKwDtQVJ6pomksWExsTb, address:145.79.227.45:4233): protocol: rpc: dial tcp 145.79.227.45:4233: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234463037 (80h19m16.828574286s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1KrucHjbKjhBZEUGd2p8cmfyCwm6rx4BVwwVhkpgnEv9EQiJRB, address:145.79.227.10:4406): protocol: rpc: dial tcp 145.79.227.10:4406: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234715782 (80h19m16.828827038s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234764656 (80h19m16.828875916s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: rpc: dial tcp 145.79.227.48:4304: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.235062285 (80h19m16.829173536s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: rpc: dial tcp 145.79.227.15:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237352242 (80h19m16.831463499s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, address:145.79.227.7:4297): protocol: rpc: dial tcp 145.79.227.7:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237418191 (80h19m16.831529449s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: rpc: dial tcp 145.79.227.4:4188: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238026103 (80h19m16.832137353s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw, address:145.79.227.25:4617): protocol: rpc: dial tcp 145.79.227.25:4617: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238136590 (80h19m16.832247826s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: rpc: dial tcp 145.79.227.46:4604: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238167799 (80h19m16.832279058s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12t9rGfCQ7bLkNnZzaPVgkE2Y9rXsiFp1XGdeECzv9M1jk8Qqmp, address:145.79.227.19:4081): protocol: rpc: dial tcp 145.79.227.19:4081: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238878716 (80h19m16.832989974s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12MnvtjSKHuNKUcVEjTviEGThRKQxzmGiMu1ofCwkv8BLAhGuzL, address:145.79.227.50:4376): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239664077 (80h19m16.833775331s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:128nLffK44ioiTpjWjAm2vHFTLouLZdw7BijqBAbsdjo99kPYNo, address:145.79.227.5:4224): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239722363 (80h19m16.833833613s) [seg_pos={0 0}, upload=77081]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil>
Apr 11 00:36:06.339820291 (80h19m16.93393154s) [upload=77081]: Done uploading segments...
Apr 11 00:36:06.339893921 (80h19m16.934005173s) [upload=77081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4)
Duration: 2.34170096s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [...XX.......................................................] duration:
1.590857364s trace:
https://jaeger.dev.storj.io/trace/62f95f2197417e9a
FAIL (2026-04-11 01:16:29 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.590880621s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [....XX......................................................] duration:
2.16650432s trace:
https://jaeger.dev.storj.io/trace/62f95f2197417e9a
FAIL (2026-04-11 01:16:10 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.166542867s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..XX........................................................] duration:
2.361074031s trace:
https://jaeger.dev.storj.io/trace/2b156773aff13a05
FAIL (2026-04-11 01:16:00 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 4)
======LOGS======
os={0 0}, upload=77081]: Done begin upload segment.
Apr 11 00:35:36.607753362 (80h18m47.201864604s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609152089 (80h18m47.203263338s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609360181 (80h18m47.203471424s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609753819 (80h18m47.203865081s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610072200 (80h18m47.20418344s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610268942 (80h18m47.20438019s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610574800 (80h18m47.204686042s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610731353 (80h18m47.204842599s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611095332 (80h18m47.205206571s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611192823 (80h18m47.205304062s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611615833 (80h18m47.205727076s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611772854 (80h18m47.2058841s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612002035 (80h18m47.206113277s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612449991 (80h18m47.206561233s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612488077 (80h18m47.206599327s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.614557840 (80h18m47.208669079s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:37.207182182 (80h18m47.801293437s) [upload=77081]: Next returned nil segment
Apr 11 00:35:37.207198012 (80h18m47.801309263s) [upload=77081]: Waiting for error group managing segments...
Apr 11 00:35:37.354583621 (80h18m47.948694872s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.355618116 (80h18m47.949729356s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.363570054 (80h18m47.957681298s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.380605963 (80h18m47.974717212s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:36:06.234067450 (80h19m16.828178701s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1DmQeoQWTnFhQeJFseJoNYzFyo7vZjYKwDtQVJ6pomksWExsTb, address:145.79.227.45:4233): protocol: rpc: dial tcp 145.79.227.45:4233: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234463037 (80h19m16.828574286s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1KrucHjbKjhBZEUGd2p8cmfyCwm6rx4BVwwVhkpgnEv9EQiJRB, address:145.79.227.10:4406): protocol: rpc: dial tcp 145.79.227.10:4406: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234715782 (80h19m16.828827038s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234764656 (80h19m16.828875916s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: rpc: dial tcp 145.79.227.48:4304: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.235062285 (80h19m16.829173536s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: rpc: dial tcp 145.79.227.15:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237352242 (80h19m16.831463499s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, address:145.79.227.7:4297): protocol: rpc: dial tcp 145.79.227.7:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237418191 (80h19m16.831529449s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: rpc: dial tcp 145.79.227.4:4188: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238026103 (80h19m16.832137353s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw, address:145.79.227.25:4617): protocol: rpc: dial tcp 145.79.227.25:4617: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238136590 (80h19m16.832247826s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: rpc: dial tcp 145.79.227.46:4604: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238167799 (80h19m16.832279058s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12t9rGfCQ7bLkNnZzaPVgkE2Y9rXsiFp1XGdeECzv9M1jk8Qqmp, address:145.79.227.19:4081): protocol: rpc: dial tcp 145.79.227.19:4081: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238878716 (80h19m16.832989974s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12MnvtjSKHuNKUcVEjTviEGThRKQxzmGiMu1ofCwkv8BLAhGuzL, address:145.79.227.50:4376): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239664077 (80h19m16.833775331s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:128nLffK44ioiTpjWjAm2vHFTLouLZdw7BijqBAbsdjo99kPYNo, address:145.79.227.5:4224): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239722363 (80h19m16.833833613s) [seg_pos={0 0}, upload=77081]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil>
Apr 11 00:36:06.339820291 (80h19m16.93393154s) [upload=77081]: Done uploading segments...
Apr 11 00:36:06.339893921 (80h19m16.934005173s) [upload=77081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4)
Duration: 2.361103663s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
3.608475686s trace:
https://jaeger.dev.storj.io/trace/46130f6078efa43c
FAIL (2026-04-11 01:15:51 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 11)
======LOGS======
ad=77069]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Apr 11 00:34:43.794844927 (80h17m54.388956169s) [seg_pos={0 0}, upload=77069]: Begin upload segment...
Apr 11 00:34:43.794911260 (80h17m54.389022502s) [seg_pos={0 0}, upload=77069]: Done begin upload segment.
Apr 11 00:34:43.795033432 (80h17m54.389144662s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.795106843 (80h17m54.389218085s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796325376 (80h17m54.390436621s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796745253 (80h17m54.390856505s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797331690 (80h17m54.39144294s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797656318 (80h17m54.391767558s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798142741 (80h17m54.392253986s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798157151 (80h17m54.392268399s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798678244 (80h17m54.392789496s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799132157 (80h17m54.393243403s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799320325 (80h17m54.393431558s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799544546 (80h17m54.393655788s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799924439 (80h17m54.394035679s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.802551628 (80h17m54.396662878s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.803497747 (80h17m54.39760899s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.804533930 (80h17m54.39864517s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.805383432 (80h17m54.399494676s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.807796955 (80h17m54.401908205s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:44.421299600 (80h17m55.015410843s) [upload=77069]: Next returned nil segment
Apr 11 00:34:44.421345070 (80h17m55.015456316s) [upload=77069]: Waiting for error group managing segments...
Apr 11 00:34:44.582519405 (80h17m55.176630646s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.597152765 (80h17m55.191264015s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.864642755 (80h17m55.458753998s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.872186316 (80h17m55.466297574s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.026075990 (80h17m55.62018723s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.028603622 (80h17m55.622714861s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.035500804 (80h17m55.629612044s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.039638177 (80h17m55.633749417s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.041344727 (80h17m55.63545598s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.402114414 (80h17m55.996225664s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:46.503621244 (80h17m57.097732493s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:35:43.391436872 (80h18m53.985548124s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:127RhN3jMBTt6LXsgym5RNMM4CvjhcSRhgjbRvSpE1TQTgQdJi9, address:145.79.227.9:4381): protocol: rpc: dial tcp 145.79.227.9:4381: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.392202791 (80h18m53.986314041s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1Rj4hUGWvUPuU49QDsHYhibiJhyY5KDGus1rzNeP7KusgyvTsU, address:145.79.227.23:4568): protocol: rpc: dial tcp 145.79.227.23:4568: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393141126 (80h18m53.987252383s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:122z4Fys9DYGB3WnjvjiZ5TSA3h8dXvMFcaFfYdpMtxGT6ZiiGH, address:145.79.227.44:4218): protocol: rpc: dial tcp 145.79.227.44:4218: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393196062 (80h18m53.987307312s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4, address:145.79.227.30:4768): protocol: rpc: dial tcp 145.79.227.30:4768: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393473160 (80h18m53.987584419s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1q6zcreD1FCjZXuYowJz9amfcQtWySL2iyrFLU8BcPzbD8PhPb, address:145.79.227.37:4597): protocol: rpc: dial tcp 145.79.227.37:4597: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393616973 (80h18m53.987728225s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:12EMGZ4hpXmBp4TwemLrbKAVtjBmtexMzEahRTzFw42qQipsAG, address:145.79.227.16:4918): protocol: rpc: dial tcp 145.79.227.16:4918: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.395024614 (80h18m53.989135865s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc, address:145.79.227.33:4882): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:43.395084251 (80h18m53.989195495s) [seg_pos={0 0}, upload=77069]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil>
Apr 11 00:35:43.502941515 (80h18m54.097052761s) [upload=77069]: Done uploading segments...
Apr 11 00:35:43.503021418 (80h18m54.097132663s) [upload=77069]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11)
Duration: 3.60852405s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...XX.......................................................] duration:
2.280444571s trace:
https://jaeger.dev.storj.io/trace/2b156773aff13a05
FAIL (2026-04-11 01:15:27 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 4)
======LOGS======
os={0 0}, upload=77081]: Done begin upload segment.
Apr 11 00:35:36.607753362 (80h18m47.201864604s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609152089 (80h18m47.203263338s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609360181 (80h18m47.203471424s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609753819 (80h18m47.203865081s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610072200 (80h18m47.20418344s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610268942 (80h18m47.20438019s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610574800 (80h18m47.204686042s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610731353 (80h18m47.204842599s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611095332 (80h18m47.205206571s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611192823 (80h18m47.205304062s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611615833 (80h18m47.205727076s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611772854 (80h18m47.2058841s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612002035 (80h18m47.206113277s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612449991 (80h18m47.206561233s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612488077 (80h18m47.206599327s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.614557840 (80h18m47.208669079s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:37.207182182 (80h18m47.801293437s) [upload=77081]: Next returned nil segment
Apr 11 00:35:37.207198012 (80h18m47.801309263s) [upload=77081]: Waiting for error group managing segments...
Apr 11 00:35:37.354583621 (80h18m47.948694872s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.355618116 (80h18m47.949729356s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.363570054 (80h18m47.957681298s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.380605963 (80h18m47.974717212s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:36:06.234067450 (80h19m16.828178701s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1DmQeoQWTnFhQeJFseJoNYzFyo7vZjYKwDtQVJ6pomksWExsTb, address:145.79.227.45:4233): protocol: rpc: dial tcp 145.79.227.45:4233: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234463037 (80h19m16.828574286s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1KrucHjbKjhBZEUGd2p8cmfyCwm6rx4BVwwVhkpgnEv9EQiJRB, address:145.79.227.10:4406): protocol: rpc: dial tcp 145.79.227.10:4406: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234715782 (80h19m16.828827038s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234764656 (80h19m16.828875916s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: rpc: dial tcp 145.79.227.48:4304: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.235062285 (80h19m16.829173536s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: rpc: dial tcp 145.79.227.15:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237352242 (80h19m16.831463499s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, address:145.79.227.7:4297): protocol: rpc: dial tcp 145.79.227.7:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237418191 (80h19m16.831529449s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: rpc: dial tcp 145.79.227.4:4188: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238026103 (80h19m16.832137353s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw, address:145.79.227.25:4617): protocol: rpc: dial tcp 145.79.227.25:4617: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238136590 (80h19m16.832247826s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: rpc: dial tcp 145.79.227.46:4604: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238167799 (80h19m16.832279058s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12t9rGfCQ7bLkNnZzaPVgkE2Y9rXsiFp1XGdeECzv9M1jk8Qqmp, address:145.79.227.19:4081): protocol: rpc: dial tcp 145.79.227.19:4081: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238878716 (80h19m16.832989974s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12MnvtjSKHuNKUcVEjTviEGThRKQxzmGiMu1ofCwkv8BLAhGuzL, address:145.79.227.50:4376): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239664077 (80h19m16.833775331s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:128nLffK44ioiTpjWjAm2vHFTLouLZdw7BijqBAbsdjo99kPYNo, address:145.79.227.5:4224): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239722363 (80h19m16.833833613s) [seg_pos={0 0}, upload=77081]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil>
Apr 11 00:36:06.339820291 (80h19m16.93393154s) [upload=77081]: Done uploading segments...
Apr 11 00:36:06.339893921 (80h19m16.934005173s) [upload=77081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4)
Duration: 2.280493217s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
3.224834646s trace:
https://jaeger.dev.storj.io/trace/46130f6078efa43c
FAIL (2026-04-11 01:15:19 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 11)
======LOGS======
ad=77069]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Apr 11 00:34:43.794844927 (80h17m54.388956169s) [seg_pos={0 0}, upload=77069]: Begin upload segment...
Apr 11 00:34:43.794911260 (80h17m54.389022502s) [seg_pos={0 0}, upload=77069]: Done begin upload segment.
Apr 11 00:34:43.795033432 (80h17m54.389144662s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.795106843 (80h17m54.389218085s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796325376 (80h17m54.390436621s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796745253 (80h17m54.390856505s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797331690 (80h17m54.39144294s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797656318 (80h17m54.391767558s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798142741 (80h17m54.392253986s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798157151 (80h17m54.392268399s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798678244 (80h17m54.392789496s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799132157 (80h17m54.393243403s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799320325 (80h17m54.393431558s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799544546 (80h17m54.393655788s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799924439 (80h17m54.394035679s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.802551628 (80h17m54.396662878s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.803497747 (80h17m54.39760899s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.804533930 (80h17m54.39864517s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.805383432 (80h17m54.399494676s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.807796955 (80h17m54.401908205s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:44.421299600 (80h17m55.015410843s) [upload=77069]: Next returned nil segment
Apr 11 00:34:44.421345070 (80h17m55.015456316s) [upload=77069]: Waiting for error group managing segments...
Apr 11 00:34:44.582519405 (80h17m55.176630646s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.597152765 (80h17m55.191264015s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.864642755 (80h17m55.458753998s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.872186316 (80h17m55.466297574s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.026075990 (80h17m55.62018723s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.028603622 (80h17m55.622714861s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.035500804 (80h17m55.629612044s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.039638177 (80h17m55.633749417s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.041344727 (80h17m55.63545598s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.402114414 (80h17m55.996225664s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:46.503621244 (80h17m57.097732493s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:35:43.391436872 (80h18m53.985548124s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:127RhN3jMBTt6LXsgym5RNMM4CvjhcSRhgjbRvSpE1TQTgQdJi9, address:145.79.227.9:4381): protocol: rpc: dial tcp 145.79.227.9:4381: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.392202791 (80h18m53.986314041s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1Rj4hUGWvUPuU49QDsHYhibiJhyY5KDGus1rzNeP7KusgyvTsU, address:145.79.227.23:4568): protocol: rpc: dial tcp 145.79.227.23:4568: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393141126 (80h18m53.987252383s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:122z4Fys9DYGB3WnjvjiZ5TSA3h8dXvMFcaFfYdpMtxGT6ZiiGH, address:145.79.227.44:4218): protocol: rpc: dial tcp 145.79.227.44:4218: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393196062 (80h18m53.987307312s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4, address:145.79.227.30:4768): protocol: rpc: dial tcp 145.79.227.30:4768: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393473160 (80h18m53.987584419s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1q6zcreD1FCjZXuYowJz9amfcQtWySL2iyrFLU8BcPzbD8PhPb, address:145.79.227.37:4597): protocol: rpc: dial tcp 145.79.227.37:4597: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393616973 (80h18m53.987728225s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:12EMGZ4hpXmBp4TwemLrbKAVtjBmtexMzEahRTzFw42qQipsAG, address:145.79.227.16:4918): protocol: rpc: dial tcp 145.79.227.16:4918: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.395024614 (80h18m53.989135865s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc, address:145.79.227.33:4882): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:43.395084251 (80h18m53.989195495s) [seg_pos={0 0}, upload=77069]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil>
Apr 11 00:35:43.502941515 (80h18m54.097052761s) [upload=77069]: Done uploading segments...
Apr 11 00:35:43.503021418 (80h18m54.097132663s) [upload=77069]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11)
Duration: 3.224862005s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [.....XX.....................................................] duration:
1.627179722s trace:
https://jaeger.dev.storj.io/trace/62f95f2197417e9a
FAIL (2026-04-11 01:15:01 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.627213888s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
3.402238521s trace:
https://jaeger.dev.storj.io/trace/46130f6078efa43c
FAIL (2026-04-11 01:14:53 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 11)
======LOGS======
ad=77069]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Apr 11 00:34:43.794844927 (80h17m54.388956169s) [seg_pos={0 0}, upload=77069]: Begin upload segment...
Apr 11 00:34:43.794911260 (80h17m54.389022502s) [seg_pos={0 0}, upload=77069]: Done begin upload segment.
Apr 11 00:34:43.795033432 (80h17m54.389144662s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.795106843 (80h17m54.389218085s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796325376 (80h17m54.390436621s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796745253 (80h17m54.390856505s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797331690 (80h17m54.39144294s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797656318 (80h17m54.391767558s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798142741 (80h17m54.392253986s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798157151 (80h17m54.392268399s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798678244 (80h17m54.392789496s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799132157 (80h17m54.393243403s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799320325 (80h17m54.393431558s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799544546 (80h17m54.393655788s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799924439 (80h17m54.394035679s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.802551628 (80h17m54.396662878s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.803497747 (80h17m54.39760899s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.804533930 (80h17m54.39864517s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.805383432 (80h17m54.399494676s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.807796955 (80h17m54.401908205s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:44.421299600 (80h17m55.015410843s) [upload=77069]: Next returned nil segment
Apr 11 00:34:44.421345070 (80h17m55.015456316s) [upload=77069]: Waiting for error group managing segments...
Apr 11 00:34:44.582519405 (80h17m55.176630646s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.597152765 (80h17m55.191264015s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.864642755 (80h17m55.458753998s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.872186316 (80h17m55.466297574s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.026075990 (80h17m55.62018723s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.028603622 (80h17m55.622714861s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.035500804 (80h17m55.629612044s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.039638177 (80h17m55.633749417s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.041344727 (80h17m55.63545598s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.402114414 (80h17m55.996225664s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:46.503621244 (80h17m57.097732493s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:35:43.391436872 (80h18m53.985548124s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:127RhN3jMBTt6LXsgym5RNMM4CvjhcSRhgjbRvSpE1TQTgQdJi9, address:145.79.227.9:4381): protocol: rpc: dial tcp 145.79.227.9:4381: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.392202791 (80h18m53.986314041s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1Rj4hUGWvUPuU49QDsHYhibiJhyY5KDGus1rzNeP7KusgyvTsU, address:145.79.227.23:4568): protocol: rpc: dial tcp 145.79.227.23:4568: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393141126 (80h18m53.987252383s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:122z4Fys9DYGB3WnjvjiZ5TSA3h8dXvMFcaFfYdpMtxGT6ZiiGH, address:145.79.227.44:4218): protocol: rpc: dial tcp 145.79.227.44:4218: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393196062 (80h18m53.987307312s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4, address:145.79.227.30:4768): protocol: rpc: dial tcp 145.79.227.30:4768: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393473160 (80h18m53.987584419s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1q6zcreD1FCjZXuYowJz9amfcQtWySL2iyrFLU8BcPzbD8PhPb, address:145.79.227.37:4597): protocol: rpc: dial tcp 145.79.227.37:4597: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393616973 (80h18m53.987728225s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:12EMGZ4hpXmBp4TwemLrbKAVtjBmtexMzEahRTzFw42qQipsAG, address:145.79.227.16:4918): protocol: rpc: dial tcp 145.79.227.16:4918: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.395024614 (80h18m53.989135865s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc, address:145.79.227.33:4882): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:43.395084251 (80h18m53.989195495s) [seg_pos={0 0}, upload=77069]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil>
Apr 11 00:35:43.502941515 (80h18m54.097052761s) [upload=77069]: Done uploading segments...
Apr 11 00:35:43.503021418 (80h18m54.097132663s) [upload=77069]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11)
Duration: 3.402266248s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [....XX......................................................] duration:
2.492897757s trace:
https://jaeger.dev.storj.io/trace/2b156773aff13a05
FAIL (2026-04-11 01:14:29 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 4)
======LOGS======
os={0 0}, upload=77081]: Done begin upload segment.
Apr 11 00:35:36.607753362 (80h18m47.201864604s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609152089 (80h18m47.203263338s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609360181 (80h18m47.203471424s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609753819 (80h18m47.203865081s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610072200 (80h18m47.20418344s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610268942 (80h18m47.20438019s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610574800 (80h18m47.204686042s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610731353 (80h18m47.204842599s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611095332 (80h18m47.205206571s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611192823 (80h18m47.205304062s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611615833 (80h18m47.205727076s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611772854 (80h18m47.2058841s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612002035 (80h18m47.206113277s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612449991 (80h18m47.206561233s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612488077 (80h18m47.206599327s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.614557840 (80h18m47.208669079s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:37.207182182 (80h18m47.801293437s) [upload=77081]: Next returned nil segment
Apr 11 00:35:37.207198012 (80h18m47.801309263s) [upload=77081]: Waiting for error group managing segments...
Apr 11 00:35:37.354583621 (80h18m47.948694872s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.355618116 (80h18m47.949729356s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.363570054 (80h18m47.957681298s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.380605963 (80h18m47.974717212s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:36:06.234067450 (80h19m16.828178701s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1DmQeoQWTnFhQeJFseJoNYzFyo7vZjYKwDtQVJ6pomksWExsTb, address:145.79.227.45:4233): protocol: rpc: dial tcp 145.79.227.45:4233: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234463037 (80h19m16.828574286s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1KrucHjbKjhBZEUGd2p8cmfyCwm6rx4BVwwVhkpgnEv9EQiJRB, address:145.79.227.10:4406): protocol: rpc: dial tcp 145.79.227.10:4406: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234715782 (80h19m16.828827038s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234764656 (80h19m16.828875916s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: rpc: dial tcp 145.79.227.48:4304: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.235062285 (80h19m16.829173536s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: rpc: dial tcp 145.79.227.15:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237352242 (80h19m16.831463499s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, address:145.79.227.7:4297): protocol: rpc: dial tcp 145.79.227.7:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237418191 (80h19m16.831529449s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: rpc: dial tcp 145.79.227.4:4188: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238026103 (80h19m16.832137353s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw, address:145.79.227.25:4617): protocol: rpc: dial tcp 145.79.227.25:4617: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238136590 (80h19m16.832247826s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: rpc: dial tcp 145.79.227.46:4604: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238167799 (80h19m16.832279058s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12t9rGfCQ7bLkNnZzaPVgkE2Y9rXsiFp1XGdeECzv9M1jk8Qqmp, address:145.79.227.19:4081): protocol: rpc: dial tcp 145.79.227.19:4081: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238878716 (80h19m16.832989974s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12MnvtjSKHuNKUcVEjTviEGThRKQxzmGiMu1ofCwkv8BLAhGuzL, address:145.79.227.50:4376): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239664077 (80h19m16.833775331s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:128nLffK44ioiTpjWjAm2vHFTLouLZdw7BijqBAbsdjo99kPYNo, address:145.79.227.5:4224): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239722363 (80h19m16.833833613s) [seg_pos={0 0}, upload=77081]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil>
Apr 11 00:36:06.339820291 (80h19m16.93393154s) [upload=77081]: Done uploading segments...
Apr 11 00:36:06.339893921 (80h19m16.934005173s) [upload=77081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4)
Duration: 2.492963852s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
3.083577467s trace:
https://jaeger.dev.storj.io/trace/46130f6078efa43c
FAIL (2026-04-11 01:14:03 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 11)
======LOGS======
ad=77069]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Apr 11 00:34:43.794844927 (80h17m54.388956169s) [seg_pos={0 0}, upload=77069]: Begin upload segment...
Apr 11 00:34:43.794911260 (80h17m54.389022502s) [seg_pos={0 0}, upload=77069]: Done begin upload segment.
Apr 11 00:34:43.795033432 (80h17m54.389144662s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.795106843 (80h17m54.389218085s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796325376 (80h17m54.390436621s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796745253 (80h17m54.390856505s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797331690 (80h17m54.39144294s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797656318 (80h17m54.391767558s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798142741 (80h17m54.392253986s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798157151 (80h17m54.392268399s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798678244 (80h17m54.392789496s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799132157 (80h17m54.393243403s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799320325 (80h17m54.393431558s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799544546 (80h17m54.393655788s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799924439 (80h17m54.394035679s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.802551628 (80h17m54.396662878s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.803497747 (80h17m54.39760899s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.804533930 (80h17m54.39864517s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.805383432 (80h17m54.399494676s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.807796955 (80h17m54.401908205s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:44.421299600 (80h17m55.015410843s) [upload=77069]: Next returned nil segment
Apr 11 00:34:44.421345070 (80h17m55.015456316s) [upload=77069]: Waiting for error group managing segments...
Apr 11 00:34:44.582519405 (80h17m55.176630646s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.597152765 (80h17m55.191264015s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.864642755 (80h17m55.458753998s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.872186316 (80h17m55.466297574s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.026075990 (80h17m55.62018723s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.028603622 (80h17m55.622714861s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.035500804 (80h17m55.629612044s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.039638177 (80h17m55.633749417s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.041344727 (80h17m55.63545598s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.402114414 (80h17m55.996225664s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:46.503621244 (80h17m57.097732493s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:35:43.391436872 (80h18m53.985548124s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:127RhN3jMBTt6LXsgym5RNMM4CvjhcSRhgjbRvSpE1TQTgQdJi9, address:145.79.227.9:4381): protocol: rpc: dial tcp 145.79.227.9:4381: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.392202791 (80h18m53.986314041s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1Rj4hUGWvUPuU49QDsHYhibiJhyY5KDGus1rzNeP7KusgyvTsU, address:145.79.227.23:4568): protocol: rpc: dial tcp 145.79.227.23:4568: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393141126 (80h18m53.987252383s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:122z4Fys9DYGB3WnjvjiZ5TSA3h8dXvMFcaFfYdpMtxGT6ZiiGH, address:145.79.227.44:4218): protocol: rpc: dial tcp 145.79.227.44:4218: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393196062 (80h18m53.987307312s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4, address:145.79.227.30:4768): protocol: rpc: dial tcp 145.79.227.30:4768: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393473160 (80h18m53.987584419s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1q6zcreD1FCjZXuYowJz9amfcQtWySL2iyrFLU8BcPzbD8PhPb, address:145.79.227.37:4597): protocol: rpc: dial tcp 145.79.227.37:4597: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393616973 (80h18m53.987728225s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:12EMGZ4hpXmBp4TwemLrbKAVtjBmtexMzEahRTzFw42qQipsAG, address:145.79.227.16:4918): protocol: rpc: dial tcp 145.79.227.16:4918: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.395024614 (80h18m53.989135865s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc, address:145.79.227.33:4882): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:43.395084251 (80h18m53.989195495s) [seg_pos={0 0}, upload=77069]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil>
Apr 11 00:35:43.502941515 (80h18m54.097052761s) [upload=77069]: Done uploading segments...
Apr 11 00:35:43.503021418 (80h18m54.097132663s) [upload=77069]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11)
Duration: 3.083608756s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [......XX....................................................] duration:
1.60363312s trace:
https://jaeger.dev.storj.io/trace/62f95f2197417e9a
FAIL (2026-04-11 01:13:53 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.603662204s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
3.380128583s trace:
https://jaeger.dev.storj.io/trace/46130f6078efa43c
FAIL (2026-04-11 01:13:33 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 11)
======LOGS======
ad=77069]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Apr 11 00:34:43.794844927 (80h17m54.388956169s) [seg_pos={0 0}, upload=77069]: Begin upload segment...
Apr 11 00:34:43.794911260 (80h17m54.389022502s) [seg_pos={0 0}, upload=77069]: Done begin upload segment.
Apr 11 00:34:43.795033432 (80h17m54.389144662s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.795106843 (80h17m54.389218085s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796325376 (80h17m54.390436621s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796745253 (80h17m54.390856505s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797331690 (80h17m54.39144294s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797656318 (80h17m54.391767558s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798142741 (80h17m54.392253986s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798157151 (80h17m54.392268399s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798678244 (80h17m54.392789496s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799132157 (80h17m54.393243403s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799320325 (80h17m54.393431558s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799544546 (80h17m54.393655788s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799924439 (80h17m54.394035679s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.802551628 (80h17m54.396662878s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.803497747 (80h17m54.39760899s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.804533930 (80h17m54.39864517s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.805383432 (80h17m54.399494676s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.807796955 (80h17m54.401908205s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:44.421299600 (80h17m55.015410843s) [upload=77069]: Next returned nil segment
Apr 11 00:34:44.421345070 (80h17m55.015456316s) [upload=77069]: Waiting for error group managing segments...
Apr 11 00:34:44.582519405 (80h17m55.176630646s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.597152765 (80h17m55.191264015s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.864642755 (80h17m55.458753998s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.872186316 (80h17m55.466297574s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.026075990 (80h17m55.62018723s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.028603622 (80h17m55.622714861s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.035500804 (80h17m55.629612044s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.039638177 (80h17m55.633749417s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.041344727 (80h17m55.63545598s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.402114414 (80h17m55.996225664s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:46.503621244 (80h17m57.097732493s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:35:43.391436872 (80h18m53.985548124s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:127RhN3jMBTt6LXsgym5RNMM4CvjhcSRhgjbRvSpE1TQTgQdJi9, address:145.79.227.9:4381): protocol: rpc: dial tcp 145.79.227.9:4381: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.392202791 (80h18m53.986314041s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1Rj4hUGWvUPuU49QDsHYhibiJhyY5KDGus1rzNeP7KusgyvTsU, address:145.79.227.23:4568): protocol: rpc: dial tcp 145.79.227.23:4568: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393141126 (80h18m53.987252383s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:122z4Fys9DYGB3WnjvjiZ5TSA3h8dXvMFcaFfYdpMtxGT6ZiiGH, address:145.79.227.44:4218): protocol: rpc: dial tcp 145.79.227.44:4218: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393196062 (80h18m53.987307312s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4, address:145.79.227.30:4768): protocol: rpc: dial tcp 145.79.227.30:4768: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393473160 (80h18m53.987584419s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1q6zcreD1FCjZXuYowJz9amfcQtWySL2iyrFLU8BcPzbD8PhPb, address:145.79.227.37:4597): protocol: rpc: dial tcp 145.79.227.37:4597: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393616973 (80h18m53.987728225s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:12EMGZ4hpXmBp4TwemLrbKAVtjBmtexMzEahRTzFw42qQipsAG, address:145.79.227.16:4918): protocol: rpc: dial tcp 145.79.227.16:4918: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.395024614 (80h18m53.989135865s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc, address:145.79.227.33:4882): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:43.395084251 (80h18m53.989195495s) [seg_pos={0 0}, upload=77069]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil>
Apr 11 00:35:43.502941515 (80h18m54.097052761s) [upload=77069]: Done uploading segments...
Apr 11 00:35:43.503021418 (80h18m54.097132663s) [upload=77069]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11)
Duration: 3.380179979s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.....XX.....................................................] duration:
2.381118568s trace:
https://jaeger.dev.storj.io/trace/2b156773aff13a05
FAIL (2026-04-11 01:13:27 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 4)
======LOGS======
os={0 0}, upload=77081]: Done begin upload segment.
Apr 11 00:35:36.607753362 (80h18m47.201864604s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609152089 (80h18m47.203263338s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609360181 (80h18m47.203471424s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609753819 (80h18m47.203865081s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610072200 (80h18m47.20418344s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610268942 (80h18m47.20438019s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610574800 (80h18m47.204686042s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610731353 (80h18m47.204842599s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611095332 (80h18m47.205206571s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611192823 (80h18m47.205304062s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611615833 (80h18m47.205727076s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611772854 (80h18m47.2058841s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612002035 (80h18m47.206113277s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612449991 (80h18m47.206561233s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612488077 (80h18m47.206599327s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.614557840 (80h18m47.208669079s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:37.207182182 (80h18m47.801293437s) [upload=77081]: Next returned nil segment
Apr 11 00:35:37.207198012 (80h18m47.801309263s) [upload=77081]: Waiting for error group managing segments...
Apr 11 00:35:37.354583621 (80h18m47.948694872s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.355618116 (80h18m47.949729356s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.363570054 (80h18m47.957681298s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.380605963 (80h18m47.974717212s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:36:06.234067450 (80h19m16.828178701s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1DmQeoQWTnFhQeJFseJoNYzFyo7vZjYKwDtQVJ6pomksWExsTb, address:145.79.227.45:4233): protocol: rpc: dial tcp 145.79.227.45:4233: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234463037 (80h19m16.828574286s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1KrucHjbKjhBZEUGd2p8cmfyCwm6rx4BVwwVhkpgnEv9EQiJRB, address:145.79.227.10:4406): protocol: rpc: dial tcp 145.79.227.10:4406: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234715782 (80h19m16.828827038s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234764656 (80h19m16.828875916s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: rpc: dial tcp 145.79.227.48:4304: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.235062285 (80h19m16.829173536s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: rpc: dial tcp 145.79.227.15:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237352242 (80h19m16.831463499s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, address:145.79.227.7:4297): protocol: rpc: dial tcp 145.79.227.7:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237418191 (80h19m16.831529449s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: rpc: dial tcp 145.79.227.4:4188: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238026103 (80h19m16.832137353s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw, address:145.79.227.25:4617): protocol: rpc: dial tcp 145.79.227.25:4617: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238136590 (80h19m16.832247826s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: rpc: dial tcp 145.79.227.46:4604: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238167799 (80h19m16.832279058s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12t9rGfCQ7bLkNnZzaPVgkE2Y9rXsiFp1XGdeECzv9M1jk8Qqmp, address:145.79.227.19:4081): protocol: rpc: dial tcp 145.79.227.19:4081: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238878716 (80h19m16.832989974s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12MnvtjSKHuNKUcVEjTviEGThRKQxzmGiMu1ofCwkv8BLAhGuzL, address:145.79.227.50:4376): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239664077 (80h19m16.833775331s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:128nLffK44ioiTpjWjAm2vHFTLouLZdw7BijqBAbsdjo99kPYNo, address:145.79.227.5:4224): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239722363 (80h19m16.833833613s) [seg_pos={0 0}, upload=77081]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil>
Apr 11 00:36:06.339820291 (80h19m16.93393154s) [upload=77081]: Done uploading segments...
Apr 11 00:36:06.339893921 (80h19m16.934005173s) [upload=77081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4)
Duration: 2.381147544s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
3.517657539s trace:
https://jaeger.dev.storj.io/trace/46130f6078efa43c
FAIL (2026-04-11 01:12:57 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 11)
======LOGS======
ad=77069]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Apr 11 00:34:43.794844927 (80h17m54.388956169s) [seg_pos={0 0}, upload=77069]: Begin upload segment...
Apr 11 00:34:43.794911260 (80h17m54.389022502s) [seg_pos={0 0}, upload=77069]: Done begin upload segment.
Apr 11 00:34:43.795033432 (80h17m54.389144662s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.795106843 (80h17m54.389218085s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796325376 (80h17m54.390436621s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796745253 (80h17m54.390856505s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797331690 (80h17m54.39144294s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797656318 (80h17m54.391767558s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798142741 (80h17m54.392253986s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798157151 (80h17m54.392268399s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798678244 (80h17m54.392789496s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799132157 (80h17m54.393243403s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799320325 (80h17m54.393431558s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799544546 (80h17m54.393655788s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799924439 (80h17m54.394035679s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.802551628 (80h17m54.396662878s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.803497747 (80h17m54.39760899s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.804533930 (80h17m54.39864517s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.805383432 (80h17m54.399494676s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.807796955 (80h17m54.401908205s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:44.421299600 (80h17m55.015410843s) [upload=77069]: Next returned nil segment
Apr 11 00:34:44.421345070 (80h17m55.015456316s) [upload=77069]: Waiting for error group managing segments...
Apr 11 00:34:44.582519405 (80h17m55.176630646s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.597152765 (80h17m55.191264015s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.864642755 (80h17m55.458753998s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.872186316 (80h17m55.466297574s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.026075990 (80h17m55.62018723s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.028603622 (80h17m55.622714861s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.035500804 (80h17m55.629612044s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.039638177 (80h17m55.633749417s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.041344727 (80h17m55.63545598s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.402114414 (80h17m55.996225664s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:46.503621244 (80h17m57.097732493s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:35:43.391436872 (80h18m53.985548124s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:127RhN3jMBTt6LXsgym5RNMM4CvjhcSRhgjbRvSpE1TQTgQdJi9, address:145.79.227.9:4381): protocol: rpc: dial tcp 145.79.227.9:4381: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.392202791 (80h18m53.986314041s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1Rj4hUGWvUPuU49QDsHYhibiJhyY5KDGus1rzNeP7KusgyvTsU, address:145.79.227.23:4568): protocol: rpc: dial tcp 145.79.227.23:4568: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393141126 (80h18m53.987252383s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:122z4Fys9DYGB3WnjvjiZ5TSA3h8dXvMFcaFfYdpMtxGT6ZiiGH, address:145.79.227.44:4218): protocol: rpc: dial tcp 145.79.227.44:4218: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393196062 (80h18m53.987307312s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4, address:145.79.227.30:4768): protocol: rpc: dial tcp 145.79.227.30:4768: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393473160 (80h18m53.987584419s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1q6zcreD1FCjZXuYowJz9amfcQtWySL2iyrFLU8BcPzbD8PhPb, address:145.79.227.37:4597): protocol: rpc: dial tcp 145.79.227.37:4597: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393616973 (80h18m53.987728225s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:12EMGZ4hpXmBp4TwemLrbKAVtjBmtexMzEahRTzFw42qQipsAG, address:145.79.227.16:4918): protocol: rpc: dial tcp 145.79.227.16:4918: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.395024614 (80h18m53.989135865s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc, address:145.79.227.33:4882): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:43.395084251 (80h18m53.989195495s) [seg_pos={0 0}, upload=77069]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil>
Apr 11 00:35:43.502941515 (80h18m54.097052761s) [upload=77069]: Done uploading segments...
Apr 11 00:35:43.503021418 (80h18m54.097132663s) [upload=77069]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11)
Duration: 3.517680233s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [.......XX...................................................] duration:
2.210848266s trace:
https://jaeger.dev.storj.io/trace/62f95f2197417e9a
FAIL (2026-04-11 01:12:51 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.21086699s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
3.390457305s trace:
https://jaeger.dev.storj.io/trace/46130f6078efa43c
FAIL (2026-04-11 01:12:37 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 11)
======LOGS======
ad=77069]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Apr 11 00:34:43.794844927 (80h17m54.388956169s) [seg_pos={0 0}, upload=77069]: Begin upload segment...
Apr 11 00:34:43.794911260 (80h17m54.389022502s) [seg_pos={0 0}, upload=77069]: Done begin upload segment.
Apr 11 00:34:43.795033432 (80h17m54.389144662s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.795106843 (80h17m54.389218085s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796325376 (80h17m54.390436621s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796745253 (80h17m54.390856505s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797331690 (80h17m54.39144294s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797656318 (80h17m54.391767558s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798142741 (80h17m54.392253986s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798157151 (80h17m54.392268399s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798678244 (80h17m54.392789496s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799132157 (80h17m54.393243403s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799320325 (80h17m54.393431558s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799544546 (80h17m54.393655788s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799924439 (80h17m54.394035679s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.802551628 (80h17m54.396662878s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.803497747 (80h17m54.39760899s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.804533930 (80h17m54.39864517s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.805383432 (80h17m54.399494676s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.807796955 (80h17m54.401908205s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:44.421299600 (80h17m55.015410843s) [upload=77069]: Next returned nil segment
Apr 11 00:34:44.421345070 (80h17m55.015456316s) [upload=77069]: Waiting for error group managing segments...
Apr 11 00:34:44.582519405 (80h17m55.176630646s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.597152765 (80h17m55.191264015s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.864642755 (80h17m55.458753998s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.872186316 (80h17m55.466297574s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.026075990 (80h17m55.62018723s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.028603622 (80h17m55.622714861s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.035500804 (80h17m55.629612044s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.039638177 (80h17m55.633749417s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.041344727 (80h17m55.63545598s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.402114414 (80h17m55.996225664s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:46.503621244 (80h17m57.097732493s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:35:43.391436872 (80h18m53.985548124s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:127RhN3jMBTt6LXsgym5RNMM4CvjhcSRhgjbRvSpE1TQTgQdJi9, address:145.79.227.9:4381): protocol: rpc: dial tcp 145.79.227.9:4381: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.392202791 (80h18m53.986314041s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1Rj4hUGWvUPuU49QDsHYhibiJhyY5KDGus1rzNeP7KusgyvTsU, address:145.79.227.23:4568): protocol: rpc: dial tcp 145.79.227.23:4568: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393141126 (80h18m53.987252383s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:122z4Fys9DYGB3WnjvjiZ5TSA3h8dXvMFcaFfYdpMtxGT6ZiiGH, address:145.79.227.44:4218): protocol: rpc: dial tcp 145.79.227.44:4218: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393196062 (80h18m53.987307312s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4, address:145.79.227.30:4768): protocol: rpc: dial tcp 145.79.227.30:4768: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393473160 (80h18m53.987584419s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1q6zcreD1FCjZXuYowJz9amfcQtWySL2iyrFLU8BcPzbD8PhPb, address:145.79.227.37:4597): protocol: rpc: dial tcp 145.79.227.37:4597: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393616973 (80h18m53.987728225s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:12EMGZ4hpXmBp4TwemLrbKAVtjBmtexMzEahRTzFw42qQipsAG, address:145.79.227.16:4918): protocol: rpc: dial tcp 145.79.227.16:4918: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.395024614 (80h18m53.989135865s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc, address:145.79.227.33:4882): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:43.395084251 (80h18m53.989195495s) [seg_pos={0 0}, upload=77069]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil>
Apr 11 00:35:43.502941515 (80h18m54.097052761s) [upload=77069]: Done uploading segments...
Apr 11 00:35:43.503021418 (80h18m54.097132663s) [upload=77069]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11)
Duration: 3.390484538s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [........XX..................................................] duration:
1.614820935s trace:
https://jaeger.dev.storj.io/trace/62f95f2197417e9a
FAIL (2026-04-11 01:12:35 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.615171022s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [......XX....................................................] duration:
2.331792613s trace:
https://jaeger.dev.storj.io/trace/2b156773aff13a05
FAIL (2026-04-11 01:12:23 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 4)
======LOGS======
os={0 0}, upload=77081]: Done begin upload segment.
Apr 11 00:35:36.607753362 (80h18m47.201864604s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609152089 (80h18m47.203263338s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609360181 (80h18m47.203471424s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609753819 (80h18m47.203865081s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610072200 (80h18m47.20418344s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610268942 (80h18m47.20438019s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610574800 (80h18m47.204686042s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610731353 (80h18m47.204842599s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611095332 (80h18m47.205206571s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611192823 (80h18m47.205304062s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611615833 (80h18m47.205727076s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611772854 (80h18m47.2058841s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612002035 (80h18m47.206113277s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612449991 (80h18m47.206561233s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612488077 (80h18m47.206599327s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.614557840 (80h18m47.208669079s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:37.207182182 (80h18m47.801293437s) [upload=77081]: Next returned nil segment
Apr 11 00:35:37.207198012 (80h18m47.801309263s) [upload=77081]: Waiting for error group managing segments...
Apr 11 00:35:37.354583621 (80h18m47.948694872s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.355618116 (80h18m47.949729356s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.363570054 (80h18m47.957681298s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.380605963 (80h18m47.974717212s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:36:06.234067450 (80h19m16.828178701s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1DmQeoQWTnFhQeJFseJoNYzFyo7vZjYKwDtQVJ6pomksWExsTb, address:145.79.227.45:4233): protocol: rpc: dial tcp 145.79.227.45:4233: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234463037 (80h19m16.828574286s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1KrucHjbKjhBZEUGd2p8cmfyCwm6rx4BVwwVhkpgnEv9EQiJRB, address:145.79.227.10:4406): protocol: rpc: dial tcp 145.79.227.10:4406: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234715782 (80h19m16.828827038s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234764656 (80h19m16.828875916s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: rpc: dial tcp 145.79.227.48:4304: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.235062285 (80h19m16.829173536s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: rpc: dial tcp 145.79.227.15:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237352242 (80h19m16.831463499s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, address:145.79.227.7:4297): protocol: rpc: dial tcp 145.79.227.7:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237418191 (80h19m16.831529449s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: rpc: dial tcp 145.79.227.4:4188: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238026103 (80h19m16.832137353s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw, address:145.79.227.25:4617): protocol: rpc: dial tcp 145.79.227.25:4617: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238136590 (80h19m16.832247826s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: rpc: dial tcp 145.79.227.46:4604: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238167799 (80h19m16.832279058s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12t9rGfCQ7bLkNnZzaPVgkE2Y9rXsiFp1XGdeECzv9M1jk8Qqmp, address:145.79.227.19:4081): protocol: rpc: dial tcp 145.79.227.19:4081: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238878716 (80h19m16.832989974s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12MnvtjSKHuNKUcVEjTviEGThRKQxzmGiMu1ofCwkv8BLAhGuzL, address:145.79.227.50:4376): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239664077 (80h19m16.833775331s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:128nLffK44ioiTpjWjAm2vHFTLouLZdw7BijqBAbsdjo99kPYNo, address:145.79.227.5:4224): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239722363 (80h19m16.833833613s) [seg_pos={0 0}, upload=77081]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil>
Apr 11 00:36:06.339820291 (80h19m16.93393154s) [upload=77081]: Done uploading segments...
Apr 11 00:36:06.339893921 (80h19m16.934005173s) [upload=77081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4)
Duration: 2.331823845s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [.........XX.................................................] duration:
1.704743857s trace:
https://jaeger.dev.storj.io/trace/62f95f2197417e9a
FAIL (2026-04-11 01:12:03 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.704778858s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
3.559147757s trace:
https://jaeger.dev.storj.io/trace/46130f6078efa43c
FAIL (2026-04-11 01:11:57 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 11)
======LOGS======
ad=77069]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Apr 11 00:34:43.794844927 (80h17m54.388956169s) [seg_pos={0 0}, upload=77069]: Begin upload segment...
Apr 11 00:34:43.794911260 (80h17m54.389022502s) [seg_pos={0 0}, upload=77069]: Done begin upload segment.
Apr 11 00:34:43.795033432 (80h17m54.389144662s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.795106843 (80h17m54.389218085s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796325376 (80h17m54.390436621s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.796745253 (80h17m54.390856505s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797331690 (80h17m54.39144294s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.797656318 (80h17m54.391767558s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798142741 (80h17m54.392253986s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798157151 (80h17m54.392268399s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.798678244 (80h17m54.392789496s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799132157 (80h17m54.393243403s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799320325 (80h17m54.393431558s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799544546 (80h17m54.393655788s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.799924439 (80h17m54.394035679s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.802551628 (80h17m54.396662878s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.803497747 (80h17m54.39760899s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.804533930 (80h17m54.39864517s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.805383432 (80h17m54.399494676s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:43.807796955 (80h17m54.401908205s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Uploading piece...
Apr 11 00:34:44.421299600 (80h17m55.015410843s) [upload=77069]: Next returned nil segment
Apr 11 00:34:44.421345070 (80h17m55.015456316s) [upload=77069]: Waiting for error group managing segments...
Apr 11 00:34:44.582519405 (80h17m55.176630646s) [noise=true , address=77.90.31.18:4811 , piece_id=3QX2UC3JZRIGKNDO7DALVXZY34MTVERLQPLAU23BLF6DTBZDLSZA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.597152765 (80h17m55.191264015s) [noise=true , address=77.90.31.14:4671 , piece_id=6IFVKC4LOSJEKP364ER4UQCHPCZLXFH2UOY4F2J66ZEFLKMMDVOA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.864642755 (80h17m55.458753998s) [noise=true , address=167.17.72.70:4768 , piece_id=QC3LRQZYKIUIJ7OYYUNEWW6BJQPPQKI7F2VB5XIMWWRQOAC6GAYQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:44.872186316 (80h17m55.466297574s) [noise=true , address=167.17.72.71:4810 , piece_id=DFLFS45MBIRJAVMM4DAGBRIOG3CW2T5OSKGBJE4X4QNYHOZYPM3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.026075990 (80h17m55.62018723s) [noise=true , address=193.149.164.67:4997 , piece_id=B4SVQK5AMTDEDZC3QXZRRR4EWMBDBOAGFK3FYHVDE4FMML5JVHYA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.028603622 (80h17m55.622714861s) [noise=true , address=193.149.164.69:4080 , piece_id=UJ654YGP7Y7F4EFUJCEZNUGYKL3N5RUZBVSLHQLIPVLHVEMDX2SA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.035500804 (80h17m55.629612044s) [noise=true , address=193.149.164.66:4963 , piece_id=3KMGDMXFE2OVGFWS4I3KJ7RYMFO2S3NFVX2H5T74KIPMMXZDHYXQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.039638177 (80h17m55.633749417s) [noise=true , address=64.49.13.154:4935 , piece_id=O5CJQSGLXQGRESPQL63XLSPDK5SPLXP67YB3SARDLT2KZWO6QL3Q, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.041344727 (80h17m55.63545598s) [noise=true , address=193.149.164.71:4286 , piece_id=M4FEYS2J5XVPKGZAT5PDMYUSXBXXO4QNXRFXCGYCJVQVLJXGWG3A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:45.402114414 (80h17m55.996225664s) [noise=true , address=167.17.72.69:4743 , piece_id=4Y47KFB2G7Z4C6GB3QI4DJ4WEM5ODKSOSBDKBJVER6Y3NUH62X2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:34:46.503621244 (80h17m57.097732493s) [noise=true , address=193.149.164.68:4054 , piece_id=SMDX727ECDRB7AFTW7OHZDUDPFURQ6VPIUOW67XZBFSBTZ7YSF2A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: <nil>
Apr 11 00:35:43.391436872 (80h18m53.985548124s) [noise=true , address=145.79.227.9:4381 , piece_id=C4KYKKMQXYIYYIHTUYK4FJQ2EPEW2NV7PAPF7YRXYF4CF2QMUVIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:127RhN3jMBTt6LXsgym5RNMM4CvjhcSRhgjbRvSpE1TQTgQdJi9, address:145.79.227.9:4381): protocol: rpc: dial tcp 145.79.227.9:4381: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.392202791 (80h18m53.986314041s) [noise=true , address=145.79.227.23:4568 , piece_id=JUFAVT5QZVVXR6LZLSRNZEODSKOOYOCWQC7FUOM5PH7GEAUXDXJA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1Rj4hUGWvUPuU49QDsHYhibiJhyY5KDGus1rzNeP7KusgyvTsU, address:145.79.227.23:4568): protocol: rpc: dial tcp 145.79.227.23:4568: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393141126 (80h18m53.987252383s) [noise=true , address=145.79.227.44:4218 , piece_id=37V2OIPK7VIZQ3GA3BOJHEN63FPJRVXIWSXJWUMKRTBLBJHLAI6A, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:122z4Fys9DYGB3WnjvjiZ5TSA3h8dXvMFcaFfYdpMtxGT6ZiiGH, address:145.79.227.44:4218): protocol: rpc: dial tcp 145.79.227.44:4218: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393196062 (80h18m53.987307312s) [noise=true , address=145.79.227.30:4768 , piece_id=PZAS2FLM4CNR7KUWBPT3BCGSPATUOXIHQNH6Q6I6O64IL52G7KIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4, address:145.79.227.30:4768): protocol: rpc: dial tcp 145.79.227.30:4768: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393473160 (80h18m53.987584419s) [noise=true , address=145.79.227.37:4597 , piece_id=VO234LURTVAKFZPNKAHIX4PCECRF6PKMP2K2EPO2RNYBGHY4KSBQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1q6zcreD1FCjZXuYowJz9amfcQtWySL2iyrFLU8BcPzbD8PhPb, address:145.79.227.37:4597): protocol: rpc: dial tcp 145.79.227.37:4597: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.393616973 (80h18m53.987728225s) [noise=true , address=145.79.227.16:4918 , piece_id=3VNDON4EHAY4RLZBKQ2YOOJ4XXYCSEZFZ3BDIBEIX5ARERGYSMEA, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:12EMGZ4hpXmBp4TwemLrbKAVtjBmtexMzEahRTzFw42qQipsAG, address:145.79.227.16:4918): protocol: rpc: dial tcp 145.79.227.16:4918: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:35:43.395024614 (80h18m53.989135865s) [noise=true , address=145.79.227.33:4882 , piece_id=MYCNFWZ5IB6V2IKFEQNLFEREMBT5J7VI6OQH6W5TCLVEGGTMKIIQ, seg_pos={0 0}, upload=77069]: Done uploading piece. err: ecclient: upload failed (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc, address:145.79.227.33:4882): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:35:43.395084251 (80h18m53.989195495s) [seg_pos={0 0}, upload=77069]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil>
Apr 11 00:35:43.502941515 (80h18m54.097052761s) [upload=77069]: Done uploading segments...
Apr 11 00:35:43.503021418 (80h18m54.097132663s) [upload=77069]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11)
Duration: 3.559175836s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.......XX...................................................] duration:
2.292683315s trace:
https://jaeger.dev.storj.io/trace/2b156773aff13a05
FAIL (2026-04-11 01:11:56 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 4)
======LOGS======
os={0 0}, upload=77081]: Done begin upload segment.
Apr 11 00:35:36.607753362 (80h18m47.201864604s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609152089 (80h18m47.203263338s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609360181 (80h18m47.203471424s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.609753819 (80h18m47.203865081s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610072200 (80h18m47.20418344s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610268942 (80h18m47.20438019s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610574800 (80h18m47.204686042s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.610731353 (80h18m47.204842599s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611095332 (80h18m47.205206571s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611192823 (80h18m47.205304062s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611615833 (80h18m47.205727076s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.611772854 (80h18m47.2058841s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612002035 (80h18m47.206113277s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612449991 (80h18m47.206561233s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.612488077 (80h18m47.206599327s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:36.614557840 (80h18m47.208669079s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Uploading piece...
Apr 11 00:35:37.207182182 (80h18m47.801293437s) [upload=77081]: Next returned nil segment
Apr 11 00:35:37.207198012 (80h18m47.801309263s) [upload=77081]: Waiting for error group managing segments...
Apr 11 00:35:37.354583621 (80h18m47.948694872s) [noise=true , address=77.90.31.16:4736 , piece_id=JCA4KDMKEDI2GM6WBNUMUUGO4LP5SXLBVKAPGTZ2LVGZIOJ3O55A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.355618116 (80h18m47.949729356s) [noise=true , address=45.91.116.12:4227 , piece_id=UMM7DUJUTYOQUQOFNKBQGOR3MHHQCGLDDYFY6HSPWUQGXLVZ4AVA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.363570054 (80h18m47.957681298s) [noise=true , address=77.90.31.12:4590 , piece_id=2DXVGZ2FAGZ5IRVJDOE542WIGB5B6PKRDBRUQOHRK573LCVXYVGA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:35:37.380605963 (80h18m47.974717212s) [noise=true , address=77.90.31.11:4557 , piece_id=AEM6JGKYC7S55TSE5UROYACPZ7IVUGA63EYXUI64PA7A6ZZMH4QQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: <nil>
Apr 11 00:36:06.234067450 (80h19m16.828178701s) [noise=true , address=145.79.227.45:4233 , piece_id=EZWKNCV7T66LCBUVSD5UAXJ3KK2JFV5CGWHW6JXNHZX3ZY5IQK2A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1DmQeoQWTnFhQeJFseJoNYzFyo7vZjYKwDtQVJ6pomksWExsTb, address:145.79.227.45:4233): protocol: rpc: dial tcp 145.79.227.45:4233: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234463037 (80h19m16.828574286s) [noise=true , address=145.79.227.10:4406 , piece_id=LWXXWS7M7RCJEE6OJVKIWCUGONFWPEPB4LKYMXO76M6FZD54TDLA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1KrucHjbKjhBZEUGd2p8cmfyCwm6rx4BVwwVhkpgnEv9EQiJRB, address:145.79.227.10:4406): protocol: rpc: dial tcp 145.79.227.10:4406: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234715782 (80h19m16.828827038s) [noise=true , address=145.79.227.12:4769 , piece_id=JBBFBZ57MZMP6WG3V7BFXORXTLSXDOZA4HAQ3CYGQLEIFOPGIAOQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:145.79.227.12:4769): protocol: rpc: dial tcp 145.79.227.12:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.234764656 (80h19m16.828875916s) [noise=true , address=145.79.227.48:4304 , piece_id=HTIH74T3OG3MJM3CG3X6KLBD3I442ZRAN7YAXPUQZ23YCPWK7ZCA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: rpc: dial tcp 145.79.227.48:4304: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.235062285 (80h19m16.829173536s) [noise=true , address=145.79.227.15:4875 , piece_id=T5YLOUSJBRXXU2UKXUHXLPK2GAJNNV7LACYLPLWXWSKXRRSSXAEA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: rpc: dial tcp 145.79.227.15:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237352242 (80h19m16.831463499s) [noise=true , address=145.79.227.7:4297 , piece_id=HADWUGMD2TGY3K4AE5GTSK5BV6DICUY6WHQ3M22Z4DKQQ3NJOW5A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:1g2pJxnoFW2TAzcAzNhjzDRJ3yYDK2mibG8tdX2Qy2YabNTq86, address:145.79.227.7:4297): protocol: rpc: dial tcp 145.79.227.7:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.237418191 (80h19m16.831529449s) [noise=true , address=145.79.227.4:4188 , piece_id=3BVCINUKA4X5MMDNA32XLSKJT3GJGLQL2LHORI5ANGCZNACWADBQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: rpc: dial tcp 145.79.227.4:4188: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238026103 (80h19m16.832137353s) [noise=true , address=145.79.227.25:4617 , piece_id=X7CHP4G7TCEWLDJLKP6VB6GOXZPZGHPRHVZWPRIWKP722GJIQE3A, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw, address:145.79.227.25:4617): protocol: rpc: dial tcp 145.79.227.25:4617: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238136590 (80h19m16.832247826s) [noise=true , address=145.79.227.46:4604 , piece_id=QW6HXGFQGRC6F44DC4UQ6V6T7LFY4GCT3N6VIRHLTLK67A3MM5EQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: rpc: dial tcp 145.79.227.46:4604: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238167799 (80h19m16.832279058s) [noise=true , address=145.79.227.19:4081 , piece_id=PPKC2Z5XYI6LFIIBYJDMOK6KNPNIKUVUV6P65NT55APN73HOEWHQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12t9rGfCQ7bLkNnZzaPVgkE2Y9rXsiFp1XGdeECzv9M1jk8Qqmp, address:145.79.227.19:4081): protocol: rpc: dial tcp 145.79.227.19:4081: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:06.238878716 (80h19m16.832989974s) [noise=true , address=145.79.227.50:4376 , piece_id=KB5XKS4RKDNT4BG2Q2L3XSFJDJUAZNMIHIOYBL4DOB35YVBCTXSA, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:12MnvtjSKHuNKUcVEjTviEGThRKQxzmGiMu1ofCwkv8BLAhGuzL, address:145.79.227.50:4376): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239664077 (80h19m16.833775331s) [noise=true , address=145.79.227.5:4224 , piece_id=EXHZT4P2TVQKG4QZHGNNYDEX2QIMBYAP6CCWOFCFQHY4N3F4JFIQ, seg_pos={0 0}, upload=77081]: Done uploading piece. err: ecclient: upload failed (node:128nLffK44ioiTpjWjAm2vHFTLouLZdw7BijqBAbsdjo99kPYNo, address:145.79.227.5:4224): protocol: context deadline exceeded; context deadline exceeded
Apr 11 00:36:06.239722363 (80h19m16.833833613s) [seg_pos={0 0}, upload=77081]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil>
Apr 11 00:36:06.339820291 (80h19m16.93393154s) [upload=77081]: Done uploading segments...
Apr 11 00:36:06.339893921 (80h19m16.934005173s) [upload=77081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4)
Duration: 2.292720052s
core
uplink
us-select-1
|