Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.................X..........................................] duration:
1.658135716s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:54:27 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.658172587s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..................X.........................................] duration:
1.599420283s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:53:34 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.599449906s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...................X........................................] duration:
1.553668099s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:52:50 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.553694246s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [....................X.......................................] duration:
1.534136749s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:51:53 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.534163974s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.....................X......................................] duration:
1.677412572s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:51:36 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.677440326s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [......................X.....................................] duration:
2.061479143s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:50:30 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 2.061575242s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.......................X....................................] duration:
1.553854634s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:50:10 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.553968488s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [........................X...................................] duration:
1.675182014s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:49:11 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.675211741s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.........................X..................................] duration:
2.065511259s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:47:58 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 2.065539472s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..........................X.................................] duration:
2.084893098s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:47:07 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 2.084920916s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...........................X................................] duration:
1.588659726s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:46:42 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.588716958s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [............................X...............................] duration:
1.587860839s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:46:07 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.58789162s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.............................X..............................] duration:
2.330107678s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:44:54 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 2.33013701s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..............................X.............................] duration:
1.578841081s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:43:58 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.578866552s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...............................X............................] duration:
1.616155556s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:43:20 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.616244542s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [................................X...........................] duration:
2.198984279s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:42:18 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 2.199017167s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.................................X..........................] duration:
1.540234736s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:41:15 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.540267423s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..................................X.........................] duration:
1.72773318s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:40:34 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.727837754s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...................................X........................] duration:
1.604226905s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:39:46 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.604257254s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [....................................X.......................] duration:
2.212791913s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:38:57 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 2.213000797s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.....................................X......................] duration:
1.632401906s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:38:15 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.632446454s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [......................................X.....................] duration:
2.206861636s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:37:14 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 2.206892216s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.......................................X....................] duration:
1.638967746s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:36:16 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.639185273s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [........................................X...................] duration:
1.73761534s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:35:41 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.737644714s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.........................................X..................] duration:
1.657806188s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:34:36 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.657913722s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..........................................X.................] duration:
1.68265572s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:34:21 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.682722661s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...........................................X................] duration:
2.114174619s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:33:15 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 2.114216173s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [............................................X...............] duration:
1.535116803s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:32:27 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.535150676s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.............................................X..............] duration:
1.556069146s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:31:22 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.556134966s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..............................................X.............] duration:
2.042741238s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:30:58 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 2.042834769s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...............................................X............] duration:
2.097484589s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:29:57 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 2.097507604s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [................................................X...........] duration:
1.62794624s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:29:27 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.628037061s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.................................................X..........] duration:
1.567597755s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:28:55 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.567625978s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..................................................X.........] duration:
2.066535225s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:27:41 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 2.066562457s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...................................................X........] duration:
1.606709722s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:26:56 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.606737062s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [....................................................X.......] duration:
1.710589425s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:25:52 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.710626157s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.....................................................X......] duration:
1.717695658s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:25:30 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.717802065s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [......................................................X.....] duration:
1.638911567s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:24:46 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.639014887s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.......................................................X....] duration:
1.585075468s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:23:57 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.585155433s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [........................................................X...] duration:
1.599431118s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:23:09 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.599473795s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.........................................................X..] duration:
1.584028237s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:22:20 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.584058104s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..........................................................X.] duration:
1.674475746s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:21:06 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.674505884s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...........................................................X] duration:
5.000865566s trace:
https://jaeger.dev.storj.io/trace/64d40cf45c1bd85
FAIL (2026-03-29 00:20:00 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 5.000914658s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
2.499021142s trace:
https://jaeger.dev.storj.io/trace/409ed236b50eb53
FAIL (2026-03-28 15:26:42 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Mar 28 14:45:35.315648730 (430h34m38.331318331s) [upload=413459]: Starting upload
Mar 28 14:45:35.315733599 (430h34m38.331403199s) [upload=413459]: Done starting upload
Mar 28 14:45:35.315794102 (430h34m38.331463702s) [upload=413459]: Uploading segments...
Mar 28 14:45:35.316032212 (430h34m38.331701813s) [upload=413459]: Got next segment. Inline: false
Mar 28 14:45:35.316200570 (430h34m38.331870171s) [upload=413459]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 28 14:45:35.316206390 (430h34m38.33187599s) [upload=413459]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 28 14:45:35.430724217 (430h34m38.446393826s) [seg_pos={0 0}, upload=413459]: Begin upload segment...
Mar 28 14:45:35.430795561 (430h34m38.446465164s) [seg_pos={0 0}, upload=413459]: Done begin upload segment.
Mar 28 14:45:35.430836859 (430h34m38.446506464s) [noise=true , address=77.90.31.19:4846 , piece_id=MOMLIPX2V2BHWIU66SML35WWK6LRRDUWRUJZ6UT2HYKOA5ZWQ5IQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.430848244 (430h34m38.446517846s) [noise=true , address=145.79.227.14:4843 , piece_id=QXIASECCPGVCCC36Y6NW3NTLJMKVPNKGXWNHTCCHME2HOOW72UIQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.431376292 (430h34m38.447045883s) [noise=true , address=79.127.219.41:4925 , piece_id=UUW3LL2MKMROAJ62NW4A66AZR337DUXIUQACIGUK2XFGV6TOC67Q, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.431872685 (430h34m38.447542294s) [noise=true , address=145.79.227.40:4051 , piece_id=6SFERDADQONW3YTOVRYHDZ7VXAOLTNZ5I3QQAS2XHG62KCNRP7TQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.432614239 (430h34m38.448283849s) [noise=true , address=145.79.227.12:4768 , piece_id=Q4ISGVZD47GU3RPO55DR4JOKIJ4SQ6UWMQPKHJAKKPOFAUJ2PZ4Q, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.432736747 (430h34m38.448406359s) [noise=true , address=145.79.227.49:4341 , piece_id=DOBHYVALR2MNRJX2ABYYBHUASIGTPTBJ5POZAZYIGSJX6K5EJLTQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.433092040 (430h34m38.448761652s) [noise=true , address=77.90.31.15:4702 , piece_id=56EUQLUUJUASC4QNLK4WJIE53EG5MF2DPD3Z7IBF3Q5J3ERJE3BQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.433377489 (430h34m38.449047102s) [noise=true , address=145.79.227.16:4911 , piece_id=JYKGUHIZQ4U74SM5BCQGXFA37I42PYXKOWB5CQQRFDX3I2EV527Q, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.434098354 (430h34m38.449767944s) [noise=true , address=79.127.219.38:4818 , piece_id=YKIWBWXFSEFPB63JTXQC4BA5YIIOCZWIOKQ4GRIBXZJ3KFCZRQKA, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.434769512 (430h34m38.450439123s) [noise=true , address=145.79.227.41:4086 , piece_id=6HKTS55E7ZO6NCX45RPJD5RF5K6DRLVRQTIZMV7ATXFVR4SACDRA, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.435075797 (430h34m38.450745407s) [noise=true , address=145.79.227.25:4617 , piece_id=WSDGMWM7CASNFGW2RPAH7MDX6O7VXEXZYLSN45ZYB2S46LPHDGOA, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.435582204 (430h34m38.451251817s) [noise=true , address=145.79.227.9:4366 , piece_id=KHTUQ2PXIXRD5Y5XIB2L3X6TMQWGZCBIS3NPOY2RS3275SCAI5BA, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.436022166 (430h34m38.451691756s) [noise=true , address=45.91.116.4:4953 , piece_id=E62ZS235DKLPVBQY6FEI3DDF3DU66LDI23ISVXANBB4M5WHQEO5Q, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.436454955 (430h34m38.452124562s) [noise=true , address=145.79.227.2:4115 , piece_id=TA4GUBLNRHKJ7DSCENFIQ5L5M7NLSJFDT4S4PVN3RRBEIIPEUYNQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.436879556 (430h34m38.452549169s) [noise=true , address=145.79.227.17:4973 , piece_id=ZZMYZGBYYVWXSOHDJGVKLNG6R3NMS2AZP4XMNCDNGJDE2DYADWIQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.437343138 (430h34m38.453012746s) [noise=true , address=145.79.227.21:4478 , piece_id=Z75OK7DR2TUDRPTM2RMX5FCKF456FYTMEVJVDBUZESZYFE6QQUTQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:36.049495517 (430h34m39.065165118s) [upload=413459]: Next returned nil segment
Mar 28 14:45:36.049527631 (430h34m39.065197233s) [upload=413459]: Waiting for error group managing segments...
Mar 28 14:45:36.182328788 (430h34m39.197998388s) [noise=true , address=145.79.227.17:4973 , piece_id=ZZMYZGBYYVWXSOHDJGVKLNG6R3NMS2AZP4XMNCDNGJDE2DYADWIQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.182575788 (430h34m39.198245412s) [noise=true , address=145.79.227.25:4617 , piece_id=WSDGMWM7CASNFGW2RPAH7MDX6O7VXEXZYLSN45ZYB2S46LPHDGOA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.182982538 (430h34m39.198652144s) [noise=true , address=145.79.227.9:4366 , piece_id=KHTUQ2PXIXRD5Y5XIB2L3X6TMQWGZCBIS3NPOY2RS3275SCAI5BA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.183438894 (430h34m39.199108504s) [noise=true , address=45.91.116.4:4953 , piece_id=E62ZS235DKLPVBQY6FEI3DDF3DU66LDI23ISVXANBB4M5WHQEO5Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.184168119 (430h34m39.199837707s) [noise=true , address=145.79.227.12:4768 , piece_id=Q4ISGVZD47GU3RPO55DR4JOKIJ4SQ6UWMQPKHJAKKPOFAUJ2PZ4Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.184865710 (430h34m39.200535314s) [noise=true , address=145.79.227.40:4051 , piece_id=6SFERDADQONW3YTOVRYHDZ7VXAOLTNZ5I3QQAS2XHG62KCNRP7TQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.185089145 (430h34m39.20075875s) [noise=true , address=145.79.227.16:4911 , piece_id=JYKGUHIZQ4U74SM5BCQGXFA37I42PYXKOWB5CQQRFDX3I2EV527Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.188553089 (430h34m39.204222694s) [noise=true , address=145.79.227.14:4843 , piece_id=QXIASECCPGVCCC36Y6NW3NTLJMKVPNKGXWNHTCCHME2HOOW72UIQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.190225530 (430h34m39.205895143s) [noise=true , address=145.79.227.2:4115 , piece_id=TA4GUBLNRHKJ7DSCENFIQ5L5M7NLSJFDT4S4PVN3RRBEIIPEUYNQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.190551513 (430h34m39.206221103s) [noise=true , address=145.79.227.21:4478 , piece_id=Z75OK7DR2TUDRPTM2RMX5FCKF456FYTMEVJVDBUZESZYFE6QQUTQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.191113981 (430h34m39.206783572s) [noise=true , address=145.79.227.41:4086 , piece_id=6HKTS55E7ZO6NCX45RPJD5RF5K6DRLVRQTIZMV7ATXFVR4SACDRA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.289167312 (430h34m39.304836941s) [noise=true , address=145.79.227.49:4341 , piece_id=DOBHYVALR2MNRJX2ABYYBHUASIGTPTBJ5POZAZYIGSJX6K5EJLTQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:46:05.029585012 (430h35m8.045254629s) [noise=true , address=77.90.31.15:4702 , piece_id=56EUQLUUJUASC4QNLK4WJIE53EG5MF2DPD3Z7IBF3Q5J3ERJE3BQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:1ucSAZVBqRRJU7WZDKs7Cs6yKV73haX7px76sfukA6b8Q8DDsW, address:77.90.31.15:4702): protocol: write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection; write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection
Mar 28 14:46:05.030074673 (430h35m8.045744285s) [noise=true , address=79.127.219.41:4925 , piece_id=UUW3LL2MKMROAJ62NW4A66AZR337DUXIUQACIGUK2XFGV6TOC67Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:1kBG5b1fGn5dLzZoD8v4KG6CHStmw1QhGpqFdtGsMsDUEXwZzs, address:79.127.219.41:4925): protocol: read tcp 139.59.144.22:33812->79.127.219.41:4925: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 28 14:46:05.030297717 (430h35m8.04596733s) [noise=true , address=77.90.31.19:4846 , piece_id=MOMLIPX2V2BHWIU66SML35WWK6LRRDUWRUJZ6UT2HYKOA5ZWQ5IQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:12V76dqXhBHet772yUXG9oRU9r1m7rtXQeARRVzTeGrcJxxzyvM, address:77.90.31.19:4846): protocol: context deadline exceeded; context deadline exceeded
Mar 28 14:46:05.030604513 (430h35m8.04627423s) [noise=true , address=79.127.219.38:4818 , piece_id=YKIWBWXFSEFPB63JTXQC4BA5YIIOCZWIOKQ4GRIBXZJ3KFCZRQKA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:12ZvUdXiXPFGqmVzYffAp735GjzaSMcyUfTD61Yi6us9ETg7Ath, address:79.127.219.38:4818): protocol: read tcp 139.59.144.22:58514->79.127.219.38:4818: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 28 14:46:05.030645703 (430h35m8.046315306s) [seg_pos={0 0}, upload=413459]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Mar 28 14:46:05.141824246 (430h35m8.157493849s) [upload=413459]: Done uploading segments...
Mar 28 14:46:05.141873159 (430h35m8.157542765s) [upload=413459]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.499050345s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
2.444942531s trace:
https://jaeger.dev.storj.io/trace/409ed236b50eb53
FAIL (2026-03-28 15:25:49 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Mar 28 14:45:35.315648730 (430h34m38.331318331s) [upload=413459]: Starting upload
Mar 28 14:45:35.315733599 (430h34m38.331403199s) [upload=413459]: Done starting upload
Mar 28 14:45:35.315794102 (430h34m38.331463702s) [upload=413459]: Uploading segments...
Mar 28 14:45:35.316032212 (430h34m38.331701813s) [upload=413459]: Got next segment. Inline: false
Mar 28 14:45:35.316200570 (430h34m38.331870171s) [upload=413459]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 28 14:45:35.316206390 (430h34m38.33187599s) [upload=413459]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 28 14:45:35.430724217 (430h34m38.446393826s) [seg_pos={0 0}, upload=413459]: Begin upload segment...
Mar 28 14:45:35.430795561 (430h34m38.446465164s) [seg_pos={0 0}, upload=413459]: Done begin upload segment.
Mar 28 14:45:35.430836859 (430h34m38.446506464s) [noise=true , address=77.90.31.19:4846 , piece_id=MOMLIPX2V2BHWIU66SML35WWK6LRRDUWRUJZ6UT2HYKOA5ZWQ5IQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.430848244 (430h34m38.446517846s) [noise=true , address=145.79.227.14:4843 , piece_id=QXIASECCPGVCCC36Y6NW3NTLJMKVPNKGXWNHTCCHME2HOOW72UIQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.431376292 (430h34m38.447045883s) [noise=true , address=79.127.219.41:4925 , piece_id=UUW3LL2MKMROAJ62NW4A66AZR337DUXIUQACIGUK2XFGV6TOC67Q, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.431872685 (430h34m38.447542294s) [noise=true , address=145.79.227.40:4051 , piece_id=6SFERDADQONW3YTOVRYHDZ7VXAOLTNZ5I3QQAS2XHG62KCNRP7TQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.432614239 (430h34m38.448283849s) [noise=true , address=145.79.227.12:4768 , piece_id=Q4ISGVZD47GU3RPO55DR4JOKIJ4SQ6UWMQPKHJAKKPOFAUJ2PZ4Q, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.432736747 (430h34m38.448406359s) [noise=true , address=145.79.227.49:4341 , piece_id=DOBHYVALR2MNRJX2ABYYBHUASIGTPTBJ5POZAZYIGSJX6K5EJLTQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.433092040 (430h34m38.448761652s) [noise=true , address=77.90.31.15:4702 , piece_id=56EUQLUUJUASC4QNLK4WJIE53EG5MF2DPD3Z7IBF3Q5J3ERJE3BQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.433377489 (430h34m38.449047102s) [noise=true , address=145.79.227.16:4911 , piece_id=JYKGUHIZQ4U74SM5BCQGXFA37I42PYXKOWB5CQQRFDX3I2EV527Q, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.434098354 (430h34m38.449767944s) [noise=true , address=79.127.219.38:4818 , piece_id=YKIWBWXFSEFPB63JTXQC4BA5YIIOCZWIOKQ4GRIBXZJ3KFCZRQKA, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.434769512 (430h34m38.450439123s) [noise=true , address=145.79.227.41:4086 , piece_id=6HKTS55E7ZO6NCX45RPJD5RF5K6DRLVRQTIZMV7ATXFVR4SACDRA, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.435075797 (430h34m38.450745407s) [noise=true , address=145.79.227.25:4617 , piece_id=WSDGMWM7CASNFGW2RPAH7MDX6O7VXEXZYLSN45ZYB2S46LPHDGOA, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.435582204 (430h34m38.451251817s) [noise=true , address=145.79.227.9:4366 , piece_id=KHTUQ2PXIXRD5Y5XIB2L3X6TMQWGZCBIS3NPOY2RS3275SCAI5BA, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.436022166 (430h34m38.451691756s) [noise=true , address=45.91.116.4:4953 , piece_id=E62ZS235DKLPVBQY6FEI3DDF3DU66LDI23ISVXANBB4M5WHQEO5Q, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.436454955 (430h34m38.452124562s) [noise=true , address=145.79.227.2:4115 , piece_id=TA4GUBLNRHKJ7DSCENFIQ5L5M7NLSJFDT4S4PVN3RRBEIIPEUYNQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.436879556 (430h34m38.452549169s) [noise=true , address=145.79.227.17:4973 , piece_id=ZZMYZGBYYVWXSOHDJGVKLNG6R3NMS2AZP4XMNCDNGJDE2DYADWIQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.437343138 (430h34m38.453012746s) [noise=true , address=145.79.227.21:4478 , piece_id=Z75OK7DR2TUDRPTM2RMX5FCKF456FYTMEVJVDBUZESZYFE6QQUTQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:36.049495517 (430h34m39.065165118s) [upload=413459]: Next returned nil segment
Mar 28 14:45:36.049527631 (430h34m39.065197233s) [upload=413459]: Waiting for error group managing segments...
Mar 28 14:45:36.182328788 (430h34m39.197998388s) [noise=true , address=145.79.227.17:4973 , piece_id=ZZMYZGBYYVWXSOHDJGVKLNG6R3NMS2AZP4XMNCDNGJDE2DYADWIQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.182575788 (430h34m39.198245412s) [noise=true , address=145.79.227.25:4617 , piece_id=WSDGMWM7CASNFGW2RPAH7MDX6O7VXEXZYLSN45ZYB2S46LPHDGOA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.182982538 (430h34m39.198652144s) [noise=true , address=145.79.227.9:4366 , piece_id=KHTUQ2PXIXRD5Y5XIB2L3X6TMQWGZCBIS3NPOY2RS3275SCAI5BA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.183438894 (430h34m39.199108504s) [noise=true , address=45.91.116.4:4953 , piece_id=E62ZS235DKLPVBQY6FEI3DDF3DU66LDI23ISVXANBB4M5WHQEO5Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.184168119 (430h34m39.199837707s) [noise=true , address=145.79.227.12:4768 , piece_id=Q4ISGVZD47GU3RPO55DR4JOKIJ4SQ6UWMQPKHJAKKPOFAUJ2PZ4Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.184865710 (430h34m39.200535314s) [noise=true , address=145.79.227.40:4051 , piece_id=6SFERDADQONW3YTOVRYHDZ7VXAOLTNZ5I3QQAS2XHG62KCNRP7TQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.185089145 (430h34m39.20075875s) [noise=true , address=145.79.227.16:4911 , piece_id=JYKGUHIZQ4U74SM5BCQGXFA37I42PYXKOWB5CQQRFDX3I2EV527Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.188553089 (430h34m39.204222694s) [noise=true , address=145.79.227.14:4843 , piece_id=QXIASECCPGVCCC36Y6NW3NTLJMKVPNKGXWNHTCCHME2HOOW72UIQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.190225530 (430h34m39.205895143s) [noise=true , address=145.79.227.2:4115 , piece_id=TA4GUBLNRHKJ7DSCENFIQ5L5M7NLSJFDT4S4PVN3RRBEIIPEUYNQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.190551513 (430h34m39.206221103s) [noise=true , address=145.79.227.21:4478 , piece_id=Z75OK7DR2TUDRPTM2RMX5FCKF456FYTMEVJVDBUZESZYFE6QQUTQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.191113981 (430h34m39.206783572s) [noise=true , address=145.79.227.41:4086 , piece_id=6HKTS55E7ZO6NCX45RPJD5RF5K6DRLVRQTIZMV7ATXFVR4SACDRA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.289167312 (430h34m39.304836941s) [noise=true , address=145.79.227.49:4341 , piece_id=DOBHYVALR2MNRJX2ABYYBHUASIGTPTBJ5POZAZYIGSJX6K5EJLTQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:46:05.029585012 (430h35m8.045254629s) [noise=true , address=77.90.31.15:4702 , piece_id=56EUQLUUJUASC4QNLK4WJIE53EG5MF2DPD3Z7IBF3Q5J3ERJE3BQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:1ucSAZVBqRRJU7WZDKs7Cs6yKV73haX7px76sfukA6b8Q8DDsW, address:77.90.31.15:4702): protocol: write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection; write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection
Mar 28 14:46:05.030074673 (430h35m8.045744285s) [noise=true , address=79.127.219.41:4925 , piece_id=UUW3LL2MKMROAJ62NW4A66AZR337DUXIUQACIGUK2XFGV6TOC67Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:1kBG5b1fGn5dLzZoD8v4KG6CHStmw1QhGpqFdtGsMsDUEXwZzs, address:79.127.219.41:4925): protocol: read tcp 139.59.144.22:33812->79.127.219.41:4925: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 28 14:46:05.030297717 (430h35m8.04596733s) [noise=true , address=77.90.31.19:4846 , piece_id=MOMLIPX2V2BHWIU66SML35WWK6LRRDUWRUJZ6UT2HYKOA5ZWQ5IQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:12V76dqXhBHet772yUXG9oRU9r1m7rtXQeARRVzTeGrcJxxzyvM, address:77.90.31.19:4846): protocol: context deadline exceeded; context deadline exceeded
Mar 28 14:46:05.030604513 (430h35m8.04627423s) [noise=true , address=79.127.219.38:4818 , piece_id=YKIWBWXFSEFPB63JTXQC4BA5YIIOCZWIOKQ4GRIBXZJ3KFCZRQKA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:12ZvUdXiXPFGqmVzYffAp735GjzaSMcyUfTD61Yi6us9ETg7Ath, address:79.127.219.38:4818): protocol: read tcp 139.59.144.22:58514->79.127.219.38:4818: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 28 14:46:05.030645703 (430h35m8.046315306s) [seg_pos={0 0}, upload=413459]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Mar 28 14:46:05.141824246 (430h35m8.157493849s) [upload=413459]: Done uploading segments...
Mar 28 14:46:05.141873159 (430h35m8.157542765s) [upload=413459]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.444973542s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
2.497811815s trace:
https://jaeger.dev.storj.io/trace/409ed236b50eb53
FAIL (2026-03-28 15:24:50 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Mar 28 14:45:35.315648730 (430h34m38.331318331s) [upload=413459]: Starting upload
Mar 28 14:45:35.315733599 (430h34m38.331403199s) [upload=413459]: Done starting upload
Mar 28 14:45:35.315794102 (430h34m38.331463702s) [upload=413459]: Uploading segments...
Mar 28 14:45:35.316032212 (430h34m38.331701813s) [upload=413459]: Got next segment. Inline: false
Mar 28 14:45:35.316200570 (430h34m38.331870171s) [upload=413459]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 28 14:45:35.316206390 (430h34m38.33187599s) [upload=413459]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 28 14:45:35.430724217 (430h34m38.446393826s) [seg_pos={0 0}, upload=413459]: Begin upload segment...
Mar 28 14:45:35.430795561 (430h34m38.446465164s) [seg_pos={0 0}, upload=413459]: Done begin upload segment.
Mar 28 14:45:35.430836859 (430h34m38.446506464s) [noise=true , address=77.90.31.19:4846 , piece_id=MOMLIPX2V2BHWIU66SML35WWK6LRRDUWRUJZ6UT2HYKOA5ZWQ5IQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.430848244 (430h34m38.446517846s) [noise=true , address=145.79.227.14:4843 , piece_id=QXIASECCPGVCCC36Y6NW3NTLJMKVPNKGXWNHTCCHME2HOOW72UIQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.431376292 (430h34m38.447045883s) [noise=true , address=79.127.219.41:4925 , piece_id=UUW3LL2MKMROAJ62NW4A66AZR337DUXIUQACIGUK2XFGV6TOC67Q, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.431872685 (430h34m38.447542294s) [noise=true , address=145.79.227.40:4051 , piece_id=6SFERDADQONW3YTOVRYHDZ7VXAOLTNZ5I3QQAS2XHG62KCNRP7TQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.432614239 (430h34m38.448283849s) [noise=true , address=145.79.227.12:4768 , piece_id=Q4ISGVZD47GU3RPO55DR4JOKIJ4SQ6UWMQPKHJAKKPOFAUJ2PZ4Q, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.432736747 (430h34m38.448406359s) [noise=true , address=145.79.227.49:4341 , piece_id=DOBHYVALR2MNRJX2ABYYBHUASIGTPTBJ5POZAZYIGSJX6K5EJLTQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.433092040 (430h34m38.448761652s) [noise=true , address=77.90.31.15:4702 , piece_id=56EUQLUUJUASC4QNLK4WJIE53EG5MF2DPD3Z7IBF3Q5J3ERJE3BQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.433377489 (430h34m38.449047102s) [noise=true , address=145.79.227.16:4911 , piece_id=JYKGUHIZQ4U74SM5BCQGXFA37I42PYXKOWB5CQQRFDX3I2EV527Q, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.434098354 (430h34m38.449767944s) [noise=true , address=79.127.219.38:4818 , piece_id=YKIWBWXFSEFPB63JTXQC4BA5YIIOCZWIOKQ4GRIBXZJ3KFCZRQKA, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.434769512 (430h34m38.450439123s) [noise=true , address=145.79.227.41:4086 , piece_id=6HKTS55E7ZO6NCX45RPJD5RF5K6DRLVRQTIZMV7ATXFVR4SACDRA, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.435075797 (430h34m38.450745407s) [noise=true , address=145.79.227.25:4617 , piece_id=WSDGMWM7CASNFGW2RPAH7MDX6O7VXEXZYLSN45ZYB2S46LPHDGOA, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.435582204 (430h34m38.451251817s) [noise=true , address=145.79.227.9:4366 , piece_id=KHTUQ2PXIXRD5Y5XIB2L3X6TMQWGZCBIS3NPOY2RS3275SCAI5BA, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.436022166 (430h34m38.451691756s) [noise=true , address=45.91.116.4:4953 , piece_id=E62ZS235DKLPVBQY6FEI3DDF3DU66LDI23ISVXANBB4M5WHQEO5Q, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.436454955 (430h34m38.452124562s) [noise=true , address=145.79.227.2:4115 , piece_id=TA4GUBLNRHKJ7DSCENFIQ5L5M7NLSJFDT4S4PVN3RRBEIIPEUYNQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.436879556 (430h34m38.452549169s) [noise=true , address=145.79.227.17:4973 , piece_id=ZZMYZGBYYVWXSOHDJGVKLNG6R3NMS2AZP4XMNCDNGJDE2DYADWIQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.437343138 (430h34m38.453012746s) [noise=true , address=145.79.227.21:4478 , piece_id=Z75OK7DR2TUDRPTM2RMX5FCKF456FYTMEVJVDBUZESZYFE6QQUTQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:36.049495517 (430h34m39.065165118s) [upload=413459]: Next returned nil segment
Mar 28 14:45:36.049527631 (430h34m39.065197233s) [upload=413459]: Waiting for error group managing segments...
Mar 28 14:45:36.182328788 (430h34m39.197998388s) [noise=true , address=145.79.227.17:4973 , piece_id=ZZMYZGBYYVWXSOHDJGVKLNG6R3NMS2AZP4XMNCDNGJDE2DYADWIQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.182575788 (430h34m39.198245412s) [noise=true , address=145.79.227.25:4617 , piece_id=WSDGMWM7CASNFGW2RPAH7MDX6O7VXEXZYLSN45ZYB2S46LPHDGOA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.182982538 (430h34m39.198652144s) [noise=true , address=145.79.227.9:4366 , piece_id=KHTUQ2PXIXRD5Y5XIB2L3X6TMQWGZCBIS3NPOY2RS3275SCAI5BA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.183438894 (430h34m39.199108504s) [noise=true , address=45.91.116.4:4953 , piece_id=E62ZS235DKLPVBQY6FEI3DDF3DU66LDI23ISVXANBB4M5WHQEO5Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.184168119 (430h34m39.199837707s) [noise=true , address=145.79.227.12:4768 , piece_id=Q4ISGVZD47GU3RPO55DR4JOKIJ4SQ6UWMQPKHJAKKPOFAUJ2PZ4Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.184865710 (430h34m39.200535314s) [noise=true , address=145.79.227.40:4051 , piece_id=6SFERDADQONW3YTOVRYHDZ7VXAOLTNZ5I3QQAS2XHG62KCNRP7TQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.185089145 (430h34m39.20075875s) [noise=true , address=145.79.227.16:4911 , piece_id=JYKGUHIZQ4U74SM5BCQGXFA37I42PYXKOWB5CQQRFDX3I2EV527Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.188553089 (430h34m39.204222694s) [noise=true , address=145.79.227.14:4843 , piece_id=QXIASECCPGVCCC36Y6NW3NTLJMKVPNKGXWNHTCCHME2HOOW72UIQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.190225530 (430h34m39.205895143s) [noise=true , address=145.79.227.2:4115 , piece_id=TA4GUBLNRHKJ7DSCENFIQ5L5M7NLSJFDT4S4PVN3RRBEIIPEUYNQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.190551513 (430h34m39.206221103s) [noise=true , address=145.79.227.21:4478 , piece_id=Z75OK7DR2TUDRPTM2RMX5FCKF456FYTMEVJVDBUZESZYFE6QQUTQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.191113981 (430h34m39.206783572s) [noise=true , address=145.79.227.41:4086 , piece_id=6HKTS55E7ZO6NCX45RPJD5RF5K6DRLVRQTIZMV7ATXFVR4SACDRA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.289167312 (430h34m39.304836941s) [noise=true , address=145.79.227.49:4341 , piece_id=DOBHYVALR2MNRJX2ABYYBHUASIGTPTBJ5POZAZYIGSJX6K5EJLTQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:46:05.029585012 (430h35m8.045254629s) [noise=true , address=77.90.31.15:4702 , piece_id=56EUQLUUJUASC4QNLK4WJIE53EG5MF2DPD3Z7IBF3Q5J3ERJE3BQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:1ucSAZVBqRRJU7WZDKs7Cs6yKV73haX7px76sfukA6b8Q8DDsW, address:77.90.31.15:4702): protocol: write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection; write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection
Mar 28 14:46:05.030074673 (430h35m8.045744285s) [noise=true , address=79.127.219.41:4925 , piece_id=UUW3LL2MKMROAJ62NW4A66AZR337DUXIUQACIGUK2XFGV6TOC67Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:1kBG5b1fGn5dLzZoD8v4KG6CHStmw1QhGpqFdtGsMsDUEXwZzs, address:79.127.219.41:4925): protocol: read tcp 139.59.144.22:33812->79.127.219.41:4925: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 28 14:46:05.030297717 (430h35m8.04596733s) [noise=true , address=77.90.31.19:4846 , piece_id=MOMLIPX2V2BHWIU66SML35WWK6LRRDUWRUJZ6UT2HYKOA5ZWQ5IQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:12V76dqXhBHet772yUXG9oRU9r1m7rtXQeARRVzTeGrcJxxzyvM, address:77.90.31.19:4846): protocol: context deadline exceeded; context deadline exceeded
Mar 28 14:46:05.030604513 (430h35m8.04627423s) [noise=true , address=79.127.219.38:4818 , piece_id=YKIWBWXFSEFPB63JTXQC4BA5YIIOCZWIOKQ4GRIBXZJ3KFCZRQKA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:12ZvUdXiXPFGqmVzYffAp735GjzaSMcyUfTD61Yi6us9ETg7Ath, address:79.127.219.38:4818): protocol: read tcp 139.59.144.22:58514->79.127.219.38:4818: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 28 14:46:05.030645703 (430h35m8.046315306s) [seg_pos={0 0}, upload=413459]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Mar 28 14:46:05.141824246 (430h35m8.157493849s) [upload=413459]: Done uploading segments...
Mar 28 14:46:05.141873159 (430h35m8.157542765s) [upload=413459]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.497847984s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
2.424364325s trace:
https://jaeger.dev.storj.io/trace/409ed236b50eb53
FAIL (2026-03-28 15:24:34 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Mar 28 14:45:35.315648730 (430h34m38.331318331s) [upload=413459]: Starting upload
Mar 28 14:45:35.315733599 (430h34m38.331403199s) [upload=413459]: Done starting upload
Mar 28 14:45:35.315794102 (430h34m38.331463702s) [upload=413459]: Uploading segments...
Mar 28 14:45:35.316032212 (430h34m38.331701813s) [upload=413459]: Got next segment. Inline: false
Mar 28 14:45:35.316200570 (430h34m38.331870171s) [upload=413459]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 28 14:45:35.316206390 (430h34m38.33187599s) [upload=413459]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 28 14:45:35.430724217 (430h34m38.446393826s) [seg_pos={0 0}, upload=413459]: Begin upload segment...
Mar 28 14:45:35.430795561 (430h34m38.446465164s) [seg_pos={0 0}, upload=413459]: Done begin upload segment.
Mar 28 14:45:35.430836859 (430h34m38.446506464s) [noise=true , address=77.90.31.19:4846 , piece_id=MOMLIPX2V2BHWIU66SML35WWK6LRRDUWRUJZ6UT2HYKOA5ZWQ5IQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.430848244 (430h34m38.446517846s) [noise=true , address=145.79.227.14:4843 , piece_id=QXIASECCPGVCCC36Y6NW3NTLJMKVPNKGXWNHTCCHME2HOOW72UIQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.431376292 (430h34m38.447045883s) [noise=true , address=79.127.219.41:4925 , piece_id=UUW3LL2MKMROAJ62NW4A66AZR337DUXIUQACIGUK2XFGV6TOC67Q, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.431872685 (430h34m38.447542294s) [noise=true , address=145.79.227.40:4051 , piece_id=6SFERDADQONW3YTOVRYHDZ7VXAOLTNZ5I3QQAS2XHG62KCNRP7TQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.432614239 (430h34m38.448283849s) [noise=true , address=145.79.227.12:4768 , piece_id=Q4ISGVZD47GU3RPO55DR4JOKIJ4SQ6UWMQPKHJAKKPOFAUJ2PZ4Q, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.432736747 (430h34m38.448406359s) [noise=true , address=145.79.227.49:4341 , piece_id=DOBHYVALR2MNRJX2ABYYBHUASIGTPTBJ5POZAZYIGSJX6K5EJLTQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.433092040 (430h34m38.448761652s) [noise=true , address=77.90.31.15:4702 , piece_id=56EUQLUUJUASC4QNLK4WJIE53EG5MF2DPD3Z7IBF3Q5J3ERJE3BQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.433377489 (430h34m38.449047102s) [noise=true , address=145.79.227.16:4911 , piece_id=JYKGUHIZQ4U74SM5BCQGXFA37I42PYXKOWB5CQQRFDX3I2EV527Q, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.434098354 (430h34m38.449767944s) [noise=true , address=79.127.219.38:4818 , piece_id=YKIWBWXFSEFPB63JTXQC4BA5YIIOCZWIOKQ4GRIBXZJ3KFCZRQKA, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.434769512 (430h34m38.450439123s) [noise=true , address=145.79.227.41:4086 , piece_id=6HKTS55E7ZO6NCX45RPJD5RF5K6DRLVRQTIZMV7ATXFVR4SACDRA, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.435075797 (430h34m38.450745407s) [noise=true , address=145.79.227.25:4617 , piece_id=WSDGMWM7CASNFGW2RPAH7MDX6O7VXEXZYLSN45ZYB2S46LPHDGOA, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.435582204 (430h34m38.451251817s) [noise=true , address=145.79.227.9:4366 , piece_id=KHTUQ2PXIXRD5Y5XIB2L3X6TMQWGZCBIS3NPOY2RS3275SCAI5BA, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.436022166 (430h34m38.451691756s) [noise=true , address=45.91.116.4:4953 , piece_id=E62ZS235DKLPVBQY6FEI3DDF3DU66LDI23ISVXANBB4M5WHQEO5Q, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.436454955 (430h34m38.452124562s) [noise=true , address=145.79.227.2:4115 , piece_id=TA4GUBLNRHKJ7DSCENFIQ5L5M7NLSJFDT4S4PVN3RRBEIIPEUYNQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.436879556 (430h34m38.452549169s) [noise=true , address=145.79.227.17:4973 , piece_id=ZZMYZGBYYVWXSOHDJGVKLNG6R3NMS2AZP4XMNCDNGJDE2DYADWIQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.437343138 (430h34m38.453012746s) [noise=true , address=145.79.227.21:4478 , piece_id=Z75OK7DR2TUDRPTM2RMX5FCKF456FYTMEVJVDBUZESZYFE6QQUTQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:36.049495517 (430h34m39.065165118s) [upload=413459]: Next returned nil segment
Mar 28 14:45:36.049527631 (430h34m39.065197233s) [upload=413459]: Waiting for error group managing segments...
Mar 28 14:45:36.182328788 (430h34m39.197998388s) [noise=true , address=145.79.227.17:4973 , piece_id=ZZMYZGBYYVWXSOHDJGVKLNG6R3NMS2AZP4XMNCDNGJDE2DYADWIQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.182575788 (430h34m39.198245412s) [noise=true , address=145.79.227.25:4617 , piece_id=WSDGMWM7CASNFGW2RPAH7MDX6O7VXEXZYLSN45ZYB2S46LPHDGOA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.182982538 (430h34m39.198652144s) [noise=true , address=145.79.227.9:4366 , piece_id=KHTUQ2PXIXRD5Y5XIB2L3X6TMQWGZCBIS3NPOY2RS3275SCAI5BA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.183438894 (430h34m39.199108504s) [noise=true , address=45.91.116.4:4953 , piece_id=E62ZS235DKLPVBQY6FEI3DDF3DU66LDI23ISVXANBB4M5WHQEO5Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.184168119 (430h34m39.199837707s) [noise=true , address=145.79.227.12:4768 , piece_id=Q4ISGVZD47GU3RPO55DR4JOKIJ4SQ6UWMQPKHJAKKPOFAUJ2PZ4Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.184865710 (430h34m39.200535314s) [noise=true , address=145.79.227.40:4051 , piece_id=6SFERDADQONW3YTOVRYHDZ7VXAOLTNZ5I3QQAS2XHG62KCNRP7TQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.185089145 (430h34m39.20075875s) [noise=true , address=145.79.227.16:4911 , piece_id=JYKGUHIZQ4U74SM5BCQGXFA37I42PYXKOWB5CQQRFDX3I2EV527Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.188553089 (430h34m39.204222694s) [noise=true , address=145.79.227.14:4843 , piece_id=QXIASECCPGVCCC36Y6NW3NTLJMKVPNKGXWNHTCCHME2HOOW72UIQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.190225530 (430h34m39.205895143s) [noise=true , address=145.79.227.2:4115 , piece_id=TA4GUBLNRHKJ7DSCENFIQ5L5M7NLSJFDT4S4PVN3RRBEIIPEUYNQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.190551513 (430h34m39.206221103s) [noise=true , address=145.79.227.21:4478 , piece_id=Z75OK7DR2TUDRPTM2RMX5FCKF456FYTMEVJVDBUZESZYFE6QQUTQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.191113981 (430h34m39.206783572s) [noise=true , address=145.79.227.41:4086 , piece_id=6HKTS55E7ZO6NCX45RPJD5RF5K6DRLVRQTIZMV7ATXFVR4SACDRA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.289167312 (430h34m39.304836941s) [noise=true , address=145.79.227.49:4341 , piece_id=DOBHYVALR2MNRJX2ABYYBHUASIGTPTBJ5POZAZYIGSJX6K5EJLTQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:46:05.029585012 (430h35m8.045254629s) [noise=true , address=77.90.31.15:4702 , piece_id=56EUQLUUJUASC4QNLK4WJIE53EG5MF2DPD3Z7IBF3Q5J3ERJE3BQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:1ucSAZVBqRRJU7WZDKs7Cs6yKV73haX7px76sfukA6b8Q8DDsW, address:77.90.31.15:4702): protocol: write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection; write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection
Mar 28 14:46:05.030074673 (430h35m8.045744285s) [noise=true , address=79.127.219.41:4925 , piece_id=UUW3LL2MKMROAJ62NW4A66AZR337DUXIUQACIGUK2XFGV6TOC67Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:1kBG5b1fGn5dLzZoD8v4KG6CHStmw1QhGpqFdtGsMsDUEXwZzs, address:79.127.219.41:4925): protocol: read tcp 139.59.144.22:33812->79.127.219.41:4925: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 28 14:46:05.030297717 (430h35m8.04596733s) [noise=true , address=77.90.31.19:4846 , piece_id=MOMLIPX2V2BHWIU66SML35WWK6LRRDUWRUJZ6UT2HYKOA5ZWQ5IQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:12V76dqXhBHet772yUXG9oRU9r1m7rtXQeARRVzTeGrcJxxzyvM, address:77.90.31.19:4846): protocol: context deadline exceeded; context deadline exceeded
Mar 28 14:46:05.030604513 (430h35m8.04627423s) [noise=true , address=79.127.219.38:4818 , piece_id=YKIWBWXFSEFPB63JTXQC4BA5YIIOCZWIOKQ4GRIBXZJ3KFCZRQKA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:12ZvUdXiXPFGqmVzYffAp735GjzaSMcyUfTD61Yi6us9ETg7Ath, address:79.127.219.38:4818): protocol: read tcp 139.59.144.22:58514->79.127.219.38:4818: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 28 14:46:05.030645703 (430h35m8.046315306s) [seg_pos={0 0}, upload=413459]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Mar 28 14:46:05.141824246 (430h35m8.157493849s) [upload=413459]: Done uploading segments...
Mar 28 14:46:05.141873159 (430h35m8.157542765s) [upload=413459]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.424403515s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
2.459391464s trace:
https://jaeger.dev.storj.io/trace/409ed236b50eb53
FAIL (2026-03-28 15:23:40 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Mar 28 14:45:35.315648730 (430h34m38.331318331s) [upload=413459]: Starting upload
Mar 28 14:45:35.315733599 (430h34m38.331403199s) [upload=413459]: Done starting upload
Mar 28 14:45:35.315794102 (430h34m38.331463702s) [upload=413459]: Uploading segments...
Mar 28 14:45:35.316032212 (430h34m38.331701813s) [upload=413459]: Got next segment. Inline: false
Mar 28 14:45:35.316200570 (430h34m38.331870171s) [upload=413459]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 28 14:45:35.316206390 (430h34m38.33187599s) [upload=413459]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 28 14:45:35.430724217 (430h34m38.446393826s) [seg_pos={0 0}, upload=413459]: Begin upload segment...
Mar 28 14:45:35.430795561 (430h34m38.446465164s) [seg_pos={0 0}, upload=413459]: Done begin upload segment.
Mar 28 14:45:35.430836859 (430h34m38.446506464s) [noise=true , address=77.90.31.19:4846 , piece_id=MOMLIPX2V2BHWIU66SML35WWK6LRRDUWRUJZ6UT2HYKOA5ZWQ5IQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.430848244 (430h34m38.446517846s) [noise=true , address=145.79.227.14:4843 , piece_id=QXIASECCPGVCCC36Y6NW3NTLJMKVPNKGXWNHTCCHME2HOOW72UIQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.431376292 (430h34m38.447045883s) [noise=true , address=79.127.219.41:4925 , piece_id=UUW3LL2MKMROAJ62NW4A66AZR337DUXIUQACIGUK2XFGV6TOC67Q, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.431872685 (430h34m38.447542294s) [noise=true , address=145.79.227.40:4051 , piece_id=6SFERDADQONW3YTOVRYHDZ7VXAOLTNZ5I3QQAS2XHG62KCNRP7TQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.432614239 (430h34m38.448283849s) [noise=true , address=145.79.227.12:4768 , piece_id=Q4ISGVZD47GU3RPO55DR4JOKIJ4SQ6UWMQPKHJAKKPOFAUJ2PZ4Q, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.432736747 (430h34m38.448406359s) [noise=true , address=145.79.227.49:4341 , piece_id=DOBHYVALR2MNRJX2ABYYBHUASIGTPTBJ5POZAZYIGSJX6K5EJLTQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.433092040 (430h34m38.448761652s) [noise=true , address=77.90.31.15:4702 , piece_id=56EUQLUUJUASC4QNLK4WJIE53EG5MF2DPD3Z7IBF3Q5J3ERJE3BQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.433377489 (430h34m38.449047102s) [noise=true , address=145.79.227.16:4911 , piece_id=JYKGUHIZQ4U74SM5BCQGXFA37I42PYXKOWB5CQQRFDX3I2EV527Q, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.434098354 (430h34m38.449767944s) [noise=true , address=79.127.219.38:4818 , piece_id=YKIWBWXFSEFPB63JTXQC4BA5YIIOCZWIOKQ4GRIBXZJ3KFCZRQKA, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.434769512 (430h34m38.450439123s) [noise=true , address=145.79.227.41:4086 , piece_id=6HKTS55E7ZO6NCX45RPJD5RF5K6DRLVRQTIZMV7ATXFVR4SACDRA, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.435075797 (430h34m38.450745407s) [noise=true , address=145.79.227.25:4617 , piece_id=WSDGMWM7CASNFGW2RPAH7MDX6O7VXEXZYLSN45ZYB2S46LPHDGOA, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.435582204 (430h34m38.451251817s) [noise=true , address=145.79.227.9:4366 , piece_id=KHTUQ2PXIXRD5Y5XIB2L3X6TMQWGZCBIS3NPOY2RS3275SCAI5BA, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.436022166 (430h34m38.451691756s) [noise=true , address=45.91.116.4:4953 , piece_id=E62ZS235DKLPVBQY6FEI3DDF3DU66LDI23ISVXANBB4M5WHQEO5Q, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.436454955 (430h34m38.452124562s) [noise=true , address=145.79.227.2:4115 , piece_id=TA4GUBLNRHKJ7DSCENFIQ5L5M7NLSJFDT4S4PVN3RRBEIIPEUYNQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.436879556 (430h34m38.452549169s) [noise=true , address=145.79.227.17:4973 , piece_id=ZZMYZGBYYVWXSOHDJGVKLNG6R3NMS2AZP4XMNCDNGJDE2DYADWIQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.437343138 (430h34m38.453012746s) [noise=true , address=145.79.227.21:4478 , piece_id=Z75OK7DR2TUDRPTM2RMX5FCKF456FYTMEVJVDBUZESZYFE6QQUTQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:36.049495517 (430h34m39.065165118s) [upload=413459]: Next returned nil segment
Mar 28 14:45:36.049527631 (430h34m39.065197233s) [upload=413459]: Waiting for error group managing segments...
Mar 28 14:45:36.182328788 (430h34m39.197998388s) [noise=true , address=145.79.227.17:4973 , piece_id=ZZMYZGBYYVWXSOHDJGVKLNG6R3NMS2AZP4XMNCDNGJDE2DYADWIQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.182575788 (430h34m39.198245412s) [noise=true , address=145.79.227.25:4617 , piece_id=WSDGMWM7CASNFGW2RPAH7MDX6O7VXEXZYLSN45ZYB2S46LPHDGOA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.182982538 (430h34m39.198652144s) [noise=true , address=145.79.227.9:4366 , piece_id=KHTUQ2PXIXRD5Y5XIB2L3X6TMQWGZCBIS3NPOY2RS3275SCAI5BA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.183438894 (430h34m39.199108504s) [noise=true , address=45.91.116.4:4953 , piece_id=E62ZS235DKLPVBQY6FEI3DDF3DU66LDI23ISVXANBB4M5WHQEO5Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.184168119 (430h34m39.199837707s) [noise=true , address=145.79.227.12:4768 , piece_id=Q4ISGVZD47GU3RPO55DR4JOKIJ4SQ6UWMQPKHJAKKPOFAUJ2PZ4Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.184865710 (430h34m39.200535314s) [noise=true , address=145.79.227.40:4051 , piece_id=6SFERDADQONW3YTOVRYHDZ7VXAOLTNZ5I3QQAS2XHG62KCNRP7TQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.185089145 (430h34m39.20075875s) [noise=true , address=145.79.227.16:4911 , piece_id=JYKGUHIZQ4U74SM5BCQGXFA37I42PYXKOWB5CQQRFDX3I2EV527Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.188553089 (430h34m39.204222694s) [noise=true , address=145.79.227.14:4843 , piece_id=QXIASECCPGVCCC36Y6NW3NTLJMKVPNKGXWNHTCCHME2HOOW72UIQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.190225530 (430h34m39.205895143s) [noise=true , address=145.79.227.2:4115 , piece_id=TA4GUBLNRHKJ7DSCENFIQ5L5M7NLSJFDT4S4PVN3RRBEIIPEUYNQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.190551513 (430h34m39.206221103s) [noise=true , address=145.79.227.21:4478 , piece_id=Z75OK7DR2TUDRPTM2RMX5FCKF456FYTMEVJVDBUZESZYFE6QQUTQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.191113981 (430h34m39.206783572s) [noise=true , address=145.79.227.41:4086 , piece_id=6HKTS55E7ZO6NCX45RPJD5RF5K6DRLVRQTIZMV7ATXFVR4SACDRA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.289167312 (430h34m39.304836941s) [noise=true , address=145.79.227.49:4341 , piece_id=DOBHYVALR2MNRJX2ABYYBHUASIGTPTBJ5POZAZYIGSJX6K5EJLTQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:46:05.029585012 (430h35m8.045254629s) [noise=true , address=77.90.31.15:4702 , piece_id=56EUQLUUJUASC4QNLK4WJIE53EG5MF2DPD3Z7IBF3Q5J3ERJE3BQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:1ucSAZVBqRRJU7WZDKs7Cs6yKV73haX7px76sfukA6b8Q8DDsW, address:77.90.31.15:4702): protocol: write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection; write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection
Mar 28 14:46:05.030074673 (430h35m8.045744285s) [noise=true , address=79.127.219.41:4925 , piece_id=UUW3LL2MKMROAJ62NW4A66AZR337DUXIUQACIGUK2XFGV6TOC67Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:1kBG5b1fGn5dLzZoD8v4KG6CHStmw1QhGpqFdtGsMsDUEXwZzs, address:79.127.219.41:4925): protocol: read tcp 139.59.144.22:33812->79.127.219.41:4925: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 28 14:46:05.030297717 (430h35m8.04596733s) [noise=true , address=77.90.31.19:4846 , piece_id=MOMLIPX2V2BHWIU66SML35WWK6LRRDUWRUJZ6UT2HYKOA5ZWQ5IQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:12V76dqXhBHet772yUXG9oRU9r1m7rtXQeARRVzTeGrcJxxzyvM, address:77.90.31.19:4846): protocol: context deadline exceeded; context deadline exceeded
Mar 28 14:46:05.030604513 (430h35m8.04627423s) [noise=true , address=79.127.219.38:4818 , piece_id=YKIWBWXFSEFPB63JTXQC4BA5YIIOCZWIOKQ4GRIBXZJ3KFCZRQKA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:12ZvUdXiXPFGqmVzYffAp735GjzaSMcyUfTD61Yi6us9ETg7Ath, address:79.127.219.38:4818): protocol: read tcp 139.59.144.22:58514->79.127.219.38:4818: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 28 14:46:05.030645703 (430h35m8.046315306s) [seg_pos={0 0}, upload=413459]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Mar 28 14:46:05.141824246 (430h35m8.157493849s) [upload=413459]: Done uploading segments...
Mar 28 14:46:05.141873159 (430h35m8.157542765s) [upload=413459]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.45942836s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
2.444681606s trace:
https://jaeger.dev.storj.io/trace/409ed236b50eb53
FAIL (2026-03-28 15:23:15 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Mar 28 14:45:35.315648730 (430h34m38.331318331s) [upload=413459]: Starting upload
Mar 28 14:45:35.315733599 (430h34m38.331403199s) [upload=413459]: Done starting upload
Mar 28 14:45:35.315794102 (430h34m38.331463702s) [upload=413459]: Uploading segments...
Mar 28 14:45:35.316032212 (430h34m38.331701813s) [upload=413459]: Got next segment. Inline: false
Mar 28 14:45:35.316200570 (430h34m38.331870171s) [upload=413459]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 28 14:45:35.316206390 (430h34m38.33187599s) [upload=413459]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 28 14:45:35.430724217 (430h34m38.446393826s) [seg_pos={0 0}, upload=413459]: Begin upload segment...
Mar 28 14:45:35.430795561 (430h34m38.446465164s) [seg_pos={0 0}, upload=413459]: Done begin upload segment.
Mar 28 14:45:35.430836859 (430h34m38.446506464s) [noise=true , address=77.90.31.19:4846 , piece_id=MOMLIPX2V2BHWIU66SML35WWK6LRRDUWRUJZ6UT2HYKOA5ZWQ5IQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.430848244 (430h34m38.446517846s) [noise=true , address=145.79.227.14:4843 , piece_id=QXIASECCPGVCCC36Y6NW3NTLJMKVPNKGXWNHTCCHME2HOOW72UIQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.431376292 (430h34m38.447045883s) [noise=true , address=79.127.219.41:4925 , piece_id=UUW3LL2MKMROAJ62NW4A66AZR337DUXIUQACIGUK2XFGV6TOC67Q, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.431872685 (430h34m38.447542294s) [noise=true , address=145.79.227.40:4051 , piece_id=6SFERDADQONW3YTOVRYHDZ7VXAOLTNZ5I3QQAS2XHG62KCNRP7TQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.432614239 (430h34m38.448283849s) [noise=true , address=145.79.227.12:4768 , piece_id=Q4ISGVZD47GU3RPO55DR4JOKIJ4SQ6UWMQPKHJAKKPOFAUJ2PZ4Q, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.432736747 (430h34m38.448406359s) [noise=true , address=145.79.227.49:4341 , piece_id=DOBHYVALR2MNRJX2ABYYBHUASIGTPTBJ5POZAZYIGSJX6K5EJLTQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.433092040 (430h34m38.448761652s) [noise=true , address=77.90.31.15:4702 , piece_id=56EUQLUUJUASC4QNLK4WJIE53EG5MF2DPD3Z7IBF3Q5J3ERJE3BQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.433377489 (430h34m38.449047102s) [noise=true , address=145.79.227.16:4911 , piece_id=JYKGUHIZQ4U74SM5BCQGXFA37I42PYXKOWB5CQQRFDX3I2EV527Q, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.434098354 (430h34m38.449767944s) [noise=true , address=79.127.219.38:4818 , piece_id=YKIWBWXFSEFPB63JTXQC4BA5YIIOCZWIOKQ4GRIBXZJ3KFCZRQKA, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.434769512 (430h34m38.450439123s) [noise=true , address=145.79.227.41:4086 , piece_id=6HKTS55E7ZO6NCX45RPJD5RF5K6DRLVRQTIZMV7ATXFVR4SACDRA, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.435075797 (430h34m38.450745407s) [noise=true , address=145.79.227.25:4617 , piece_id=WSDGMWM7CASNFGW2RPAH7MDX6O7VXEXZYLSN45ZYB2S46LPHDGOA, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.435582204 (430h34m38.451251817s) [noise=true , address=145.79.227.9:4366 , piece_id=KHTUQ2PXIXRD5Y5XIB2L3X6TMQWGZCBIS3NPOY2RS3275SCAI5BA, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.436022166 (430h34m38.451691756s) [noise=true , address=45.91.116.4:4953 , piece_id=E62ZS235DKLPVBQY6FEI3DDF3DU66LDI23ISVXANBB4M5WHQEO5Q, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.436454955 (430h34m38.452124562s) [noise=true , address=145.79.227.2:4115 , piece_id=TA4GUBLNRHKJ7DSCENFIQ5L5M7NLSJFDT4S4PVN3RRBEIIPEUYNQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.436879556 (430h34m38.452549169s) [noise=true , address=145.79.227.17:4973 , piece_id=ZZMYZGBYYVWXSOHDJGVKLNG6R3NMS2AZP4XMNCDNGJDE2DYADWIQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.437343138 (430h34m38.453012746s) [noise=true , address=145.79.227.21:4478 , piece_id=Z75OK7DR2TUDRPTM2RMX5FCKF456FYTMEVJVDBUZESZYFE6QQUTQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:36.049495517 (430h34m39.065165118s) [upload=413459]: Next returned nil segment
Mar 28 14:45:36.049527631 (430h34m39.065197233s) [upload=413459]: Waiting for error group managing segments...
Mar 28 14:45:36.182328788 (430h34m39.197998388s) [noise=true , address=145.79.227.17:4973 , piece_id=ZZMYZGBYYVWXSOHDJGVKLNG6R3NMS2AZP4XMNCDNGJDE2DYADWIQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.182575788 (430h34m39.198245412s) [noise=true , address=145.79.227.25:4617 , piece_id=WSDGMWM7CASNFGW2RPAH7MDX6O7VXEXZYLSN45ZYB2S46LPHDGOA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.182982538 (430h34m39.198652144s) [noise=true , address=145.79.227.9:4366 , piece_id=KHTUQ2PXIXRD5Y5XIB2L3X6TMQWGZCBIS3NPOY2RS3275SCAI5BA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.183438894 (430h34m39.199108504s) [noise=true , address=45.91.116.4:4953 , piece_id=E62ZS235DKLPVBQY6FEI3DDF3DU66LDI23ISVXANBB4M5WHQEO5Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.184168119 (430h34m39.199837707s) [noise=true , address=145.79.227.12:4768 , piece_id=Q4ISGVZD47GU3RPO55DR4JOKIJ4SQ6UWMQPKHJAKKPOFAUJ2PZ4Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.184865710 (430h34m39.200535314s) [noise=true , address=145.79.227.40:4051 , piece_id=6SFERDADQONW3YTOVRYHDZ7VXAOLTNZ5I3QQAS2XHG62KCNRP7TQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.185089145 (430h34m39.20075875s) [noise=true , address=145.79.227.16:4911 , piece_id=JYKGUHIZQ4U74SM5BCQGXFA37I42PYXKOWB5CQQRFDX3I2EV527Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.188553089 (430h34m39.204222694s) [noise=true , address=145.79.227.14:4843 , piece_id=QXIASECCPGVCCC36Y6NW3NTLJMKVPNKGXWNHTCCHME2HOOW72UIQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.190225530 (430h34m39.205895143s) [noise=true , address=145.79.227.2:4115 , piece_id=TA4GUBLNRHKJ7DSCENFIQ5L5M7NLSJFDT4S4PVN3RRBEIIPEUYNQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.190551513 (430h34m39.206221103s) [noise=true , address=145.79.227.21:4478 , piece_id=Z75OK7DR2TUDRPTM2RMX5FCKF456FYTMEVJVDBUZESZYFE6QQUTQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.191113981 (430h34m39.206783572s) [noise=true , address=145.79.227.41:4086 , piece_id=6HKTS55E7ZO6NCX45RPJD5RF5K6DRLVRQTIZMV7ATXFVR4SACDRA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.289167312 (430h34m39.304836941s) [noise=true , address=145.79.227.49:4341 , piece_id=DOBHYVALR2MNRJX2ABYYBHUASIGTPTBJ5POZAZYIGSJX6K5EJLTQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:46:05.029585012 (430h35m8.045254629s) [noise=true , address=77.90.31.15:4702 , piece_id=56EUQLUUJUASC4QNLK4WJIE53EG5MF2DPD3Z7IBF3Q5J3ERJE3BQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:1ucSAZVBqRRJU7WZDKs7Cs6yKV73haX7px76sfukA6b8Q8DDsW, address:77.90.31.15:4702): protocol: write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection; write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection
Mar 28 14:46:05.030074673 (430h35m8.045744285s) [noise=true , address=79.127.219.41:4925 , piece_id=UUW3LL2MKMROAJ62NW4A66AZR337DUXIUQACIGUK2XFGV6TOC67Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:1kBG5b1fGn5dLzZoD8v4KG6CHStmw1QhGpqFdtGsMsDUEXwZzs, address:79.127.219.41:4925): protocol: read tcp 139.59.144.22:33812->79.127.219.41:4925: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 28 14:46:05.030297717 (430h35m8.04596733s) [noise=true , address=77.90.31.19:4846 , piece_id=MOMLIPX2V2BHWIU66SML35WWK6LRRDUWRUJZ6UT2HYKOA5ZWQ5IQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:12V76dqXhBHet772yUXG9oRU9r1m7rtXQeARRVzTeGrcJxxzyvM, address:77.90.31.19:4846): protocol: context deadline exceeded; context deadline exceeded
Mar 28 14:46:05.030604513 (430h35m8.04627423s) [noise=true , address=79.127.219.38:4818 , piece_id=YKIWBWXFSEFPB63JTXQC4BA5YIIOCZWIOKQ4GRIBXZJ3KFCZRQKA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:12ZvUdXiXPFGqmVzYffAp735GjzaSMcyUfTD61Yi6us9ETg7Ath, address:79.127.219.38:4818): protocol: read tcp 139.59.144.22:58514->79.127.219.38:4818: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 28 14:46:05.030645703 (430h35m8.046315306s) [seg_pos={0 0}, upload=413459]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Mar 28 14:46:05.141824246 (430h35m8.157493849s) [upload=413459]: Done uploading segments...
Mar 28 14:46:05.141873159 (430h35m8.157542765s) [upload=413459]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.444715078s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
2.552873673s trace:
https://jaeger.dev.storj.io/trace/409ed236b50eb53
FAIL (2026-03-28 15:22:10 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Mar 28 14:45:35.315648730 (430h34m38.331318331s) [upload=413459]: Starting upload
Mar 28 14:45:35.315733599 (430h34m38.331403199s) [upload=413459]: Done starting upload
Mar 28 14:45:35.315794102 (430h34m38.331463702s) [upload=413459]: Uploading segments...
Mar 28 14:45:35.316032212 (430h34m38.331701813s) [upload=413459]: Got next segment. Inline: false
Mar 28 14:45:35.316200570 (430h34m38.331870171s) [upload=413459]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 28 14:45:35.316206390 (430h34m38.33187599s) [upload=413459]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 28 14:45:35.430724217 (430h34m38.446393826s) [seg_pos={0 0}, upload=413459]: Begin upload segment...
Mar 28 14:45:35.430795561 (430h34m38.446465164s) [seg_pos={0 0}, upload=413459]: Done begin upload segment.
Mar 28 14:45:35.430836859 (430h34m38.446506464s) [noise=true , address=77.90.31.19:4846 , piece_id=MOMLIPX2V2BHWIU66SML35WWK6LRRDUWRUJZ6UT2HYKOA5ZWQ5IQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.430848244 (430h34m38.446517846s) [noise=true , address=145.79.227.14:4843 , piece_id=QXIASECCPGVCCC36Y6NW3NTLJMKVPNKGXWNHTCCHME2HOOW72UIQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.431376292 (430h34m38.447045883s) [noise=true , address=79.127.219.41:4925 , piece_id=UUW3LL2MKMROAJ62NW4A66AZR337DUXIUQACIGUK2XFGV6TOC67Q, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.431872685 (430h34m38.447542294s) [noise=true , address=145.79.227.40:4051 , piece_id=6SFERDADQONW3YTOVRYHDZ7VXAOLTNZ5I3QQAS2XHG62KCNRP7TQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.432614239 (430h34m38.448283849s) [noise=true , address=145.79.227.12:4768 , piece_id=Q4ISGVZD47GU3RPO55DR4JOKIJ4SQ6UWMQPKHJAKKPOFAUJ2PZ4Q, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.432736747 (430h34m38.448406359s) [noise=true , address=145.79.227.49:4341 , piece_id=DOBHYVALR2MNRJX2ABYYBHUASIGTPTBJ5POZAZYIGSJX6K5EJLTQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.433092040 (430h34m38.448761652s) [noise=true , address=77.90.31.15:4702 , piece_id=56EUQLUUJUASC4QNLK4WJIE53EG5MF2DPD3Z7IBF3Q5J3ERJE3BQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.433377489 (430h34m38.449047102s) [noise=true , address=145.79.227.16:4911 , piece_id=JYKGUHIZQ4U74SM5BCQGXFA37I42PYXKOWB5CQQRFDX3I2EV527Q, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.434098354 (430h34m38.449767944s) [noise=true , address=79.127.219.38:4818 , piece_id=YKIWBWXFSEFPB63JTXQC4BA5YIIOCZWIOKQ4GRIBXZJ3KFCZRQKA, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.434769512 (430h34m38.450439123s) [noise=true , address=145.79.227.41:4086 , piece_id=6HKTS55E7ZO6NCX45RPJD5RF5K6DRLVRQTIZMV7ATXFVR4SACDRA, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.435075797 (430h34m38.450745407s) [noise=true , address=145.79.227.25:4617 , piece_id=WSDGMWM7CASNFGW2RPAH7MDX6O7VXEXZYLSN45ZYB2S46LPHDGOA, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.435582204 (430h34m38.451251817s) [noise=true , address=145.79.227.9:4366 , piece_id=KHTUQ2PXIXRD5Y5XIB2L3X6TMQWGZCBIS3NPOY2RS3275SCAI5BA, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.436022166 (430h34m38.451691756s) [noise=true , address=45.91.116.4:4953 , piece_id=E62ZS235DKLPVBQY6FEI3DDF3DU66LDI23ISVXANBB4M5WHQEO5Q, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.436454955 (430h34m38.452124562s) [noise=true , address=145.79.227.2:4115 , piece_id=TA4GUBLNRHKJ7DSCENFIQ5L5M7NLSJFDT4S4PVN3RRBEIIPEUYNQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.436879556 (430h34m38.452549169s) [noise=true , address=145.79.227.17:4973 , piece_id=ZZMYZGBYYVWXSOHDJGVKLNG6R3NMS2AZP4XMNCDNGJDE2DYADWIQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:35.437343138 (430h34m38.453012746s) [noise=true , address=145.79.227.21:4478 , piece_id=Z75OK7DR2TUDRPTM2RMX5FCKF456FYTMEVJVDBUZESZYFE6QQUTQ, seg_pos={0 0}, upload=413459]: Uploading piece...
Mar 28 14:45:36.049495517 (430h34m39.065165118s) [upload=413459]: Next returned nil segment
Mar 28 14:45:36.049527631 (430h34m39.065197233s) [upload=413459]: Waiting for error group managing segments...
Mar 28 14:45:36.182328788 (430h34m39.197998388s) [noise=true , address=145.79.227.17:4973 , piece_id=ZZMYZGBYYVWXSOHDJGVKLNG6R3NMS2AZP4XMNCDNGJDE2DYADWIQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.182575788 (430h34m39.198245412s) [noise=true , address=145.79.227.25:4617 , piece_id=WSDGMWM7CASNFGW2RPAH7MDX6O7VXEXZYLSN45ZYB2S46LPHDGOA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.182982538 (430h34m39.198652144s) [noise=true , address=145.79.227.9:4366 , piece_id=KHTUQ2PXIXRD5Y5XIB2L3X6TMQWGZCBIS3NPOY2RS3275SCAI5BA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.183438894 (430h34m39.199108504s) [noise=true , address=45.91.116.4:4953 , piece_id=E62ZS235DKLPVBQY6FEI3DDF3DU66LDI23ISVXANBB4M5WHQEO5Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.184168119 (430h34m39.199837707s) [noise=true , address=145.79.227.12:4768 , piece_id=Q4ISGVZD47GU3RPO55DR4JOKIJ4SQ6UWMQPKHJAKKPOFAUJ2PZ4Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.184865710 (430h34m39.200535314s) [noise=true , address=145.79.227.40:4051 , piece_id=6SFERDADQONW3YTOVRYHDZ7VXAOLTNZ5I3QQAS2XHG62KCNRP7TQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.185089145 (430h34m39.20075875s) [noise=true , address=145.79.227.16:4911 , piece_id=JYKGUHIZQ4U74SM5BCQGXFA37I42PYXKOWB5CQQRFDX3I2EV527Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.188553089 (430h34m39.204222694s) [noise=true , address=145.79.227.14:4843 , piece_id=QXIASECCPGVCCC36Y6NW3NTLJMKVPNKGXWNHTCCHME2HOOW72UIQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.190225530 (430h34m39.205895143s) [noise=true , address=145.79.227.2:4115 , piece_id=TA4GUBLNRHKJ7DSCENFIQ5L5M7NLSJFDT4S4PVN3RRBEIIPEUYNQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.190551513 (430h34m39.206221103s) [noise=true , address=145.79.227.21:4478 , piece_id=Z75OK7DR2TUDRPTM2RMX5FCKF456FYTMEVJVDBUZESZYFE6QQUTQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.191113981 (430h34m39.206783572s) [noise=true , address=145.79.227.41:4086 , piece_id=6HKTS55E7ZO6NCX45RPJD5RF5K6DRLVRQTIZMV7ATXFVR4SACDRA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:45:36.289167312 (430h34m39.304836941s) [noise=true , address=145.79.227.49:4341 , piece_id=DOBHYVALR2MNRJX2ABYYBHUASIGTPTBJ5POZAZYIGSJX6K5EJLTQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil>
Mar 28 14:46:05.029585012 (430h35m8.045254629s) [noise=true , address=77.90.31.15:4702 , piece_id=56EUQLUUJUASC4QNLK4WJIE53EG5MF2DPD3Z7IBF3Q5J3ERJE3BQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:1ucSAZVBqRRJU7WZDKs7Cs6yKV73haX7px76sfukA6b8Q8DDsW, address:77.90.31.15:4702): protocol: write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection; write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection
Mar 28 14:46:05.030074673 (430h35m8.045744285s) [noise=true , address=79.127.219.41:4925 , piece_id=UUW3LL2MKMROAJ62NW4A66AZR337DUXIUQACIGUK2XFGV6TOC67Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:1kBG5b1fGn5dLzZoD8v4KG6CHStmw1QhGpqFdtGsMsDUEXwZzs, address:79.127.219.41:4925): protocol: read tcp 139.59.144.22:33812->79.127.219.41:4925: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 28 14:46:05.030297717 (430h35m8.04596733s) [noise=true , address=77.90.31.19:4846 , piece_id=MOMLIPX2V2BHWIU66SML35WWK6LRRDUWRUJZ6UT2HYKOA5ZWQ5IQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:12V76dqXhBHet772yUXG9oRU9r1m7rtXQeARRVzTeGrcJxxzyvM, address:77.90.31.19:4846): protocol: context deadline exceeded; context deadline exceeded
Mar 28 14:46:05.030604513 (430h35m8.04627423s) [noise=true , address=79.127.219.38:4818 , piece_id=YKIWBWXFSEFPB63JTXQC4BA5YIIOCZWIOKQ4GRIBXZJ3KFCZRQKA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:12ZvUdXiXPFGqmVzYffAp735GjzaSMcyUfTD61Yi6us9ETg7Ath, address:79.127.219.38:4818): protocol: read tcp 139.59.144.22:58514->79.127.219.38:4818: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 28 14:46:05.030645703 (430h35m8.046315306s) [seg_pos={0 0}, upload=413459]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Mar 28 14:46:05.141824246 (430h35m8.157493849s) [upload=413459]: Done uploading segments...
Mar 28 14:46:05.141873159 (430h35m8.157542765s) [upload=413459]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.552912619s
core
uplink
us-select-1
|