Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..............................X.............................] duration:
2.316895248s trace:
https://jaeger.dev.storj.io/trace/2a81c80381aaeb47
FAIL (2026-03-19 04:00:51 UTC)
orbiter warning: context deadline exceeded
Duration: 2.316924581s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...............................X............................] duration:
1.699418725s trace:
https://jaeger.dev.storj.io/trace/2a81c80381aaeb47
FAIL (2026-03-19 03:59:52 UTC)
orbiter warning: context deadline exceeded
Duration: 1.699445121s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [................................X...........................] duration:
2.259744434s trace:
https://jaeger.dev.storj.io/trace/2a81c80381aaeb47
FAIL (2026-03-19 03:59:19 UTC)
orbiter warning: context deadline exceeded
Duration: 2.259775685s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.................................X..........................] duration:
1.677311004s trace:
https://jaeger.dev.storj.io/trace/2a81c80381aaeb47
FAIL (2026-03-19 03:58:08 UTC)
orbiter warning: context deadline exceeded
Duration: 1.677338652s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..................................X.........................] duration:
2.192343917s trace:
https://jaeger.dev.storj.io/trace/2a81c80381aaeb47
FAIL (2026-03-19 03:56:57 UTC)
orbiter warning: context deadline exceeded
Duration: 2.192370448s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...................................X........................] duration:
2.21990912s trace:
https://jaeger.dev.storj.io/trace/2a81c80381aaeb47
FAIL (2026-03-19 03:55:57 UTC)
orbiter warning: context deadline exceeded
Duration: 2.219937402s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [....................................X.......................] duration:
1.578619181s trace:
https://jaeger.dev.storj.io/trace/2a81c80381aaeb47
FAIL (2026-03-19 03:54:48 UTC)
orbiter warning: context deadline exceeded
Duration: 1.578646419s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.....................................X......................] duration:
1.701332095s trace:
https://jaeger.dev.storj.io/trace/2a81c80381aaeb47
FAIL (2026-03-19 03:53:43 UTC)
orbiter warning: context deadline exceeded
Duration: 1.701367438s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [......................................X.....................] duration:
1.653527364s trace:
https://jaeger.dev.storj.io/trace/2a81c80381aaeb47
FAIL (2026-03-19 03:52:58 UTC)
orbiter warning: context deadline exceeded
Duration: 1.653605289s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.......................................X....................] duration:
1.629614007s trace:
https://jaeger.dev.storj.io/trace/2a81c80381aaeb47
FAIL (2026-03-19 03:51:50 UTC)
orbiter warning: context deadline exceeded
Duration: 1.629645853s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [........................................X...................] duration:
2.077404548s trace:
https://jaeger.dev.storj.io/trace/2a81c80381aaeb47
FAIL (2026-03-19 03:50:39 UTC)
orbiter warning: context deadline exceeded
Duration: 2.077433947s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.........................................X..................] duration:
1.610540572s trace:
https://jaeger.dev.storj.io/trace/2a81c80381aaeb47
FAIL (2026-03-19 03:49:58 UTC)
orbiter warning: context deadline exceeded
Duration: 1.610589944s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..........................................X.................] duration:
1.680818683s trace:
https://jaeger.dev.storj.io/trace/2a81c80381aaeb47
FAIL (2026-03-19 03:49:22 UTC)
orbiter warning: context deadline exceeded
Duration: 1.680903176s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...........................................X................] duration:
1.612850135s trace:
https://jaeger.dev.storj.io/trace/2a81c80381aaeb47
FAIL (2026-03-19 03:48:28 UTC)
orbiter warning: context deadline exceeded
Duration: 1.612885685s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [............................................X...............] duration:
2.098720425s trace:
https://jaeger.dev.storj.io/trace/2a81c80381aaeb47
FAIL (2026-03-19 03:48:13 UTC)
orbiter warning: context deadline exceeded
Duration: 2.098748236s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.............................................X..............] duration:
1.639575719s trace:
https://jaeger.dev.storj.io/trace/2a81c80381aaeb47
FAIL (2026-03-19 03:46:59 UTC)
orbiter warning: context deadline exceeded
Duration: 1.639617476s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..............................................X.............] duration:
1.717006344s trace:
https://jaeger.dev.storj.io/trace/2a81c80381aaeb47
FAIL (2026-03-19 03:46:42 UTC)
orbiter warning: context deadline exceeded
Duration: 1.717039534s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...............................................X............] duration:
1.629166855s trace:
https://jaeger.dev.storj.io/trace/2a81c80381aaeb47
FAIL (2026-03-19 03:45:28 UTC)
orbiter warning: context deadline exceeded
Duration: 1.629194201s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [................................................X...........] duration:
1.666871235s trace:
https://jaeger.dev.storj.io/trace/2a81c80381aaeb47
FAIL (2026-03-19 03:44:22 UTC)
orbiter warning: context deadline exceeded
Duration: 1.666895762s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.................................................X..........] duration:
2.258203925s trace:
https://jaeger.dev.storj.io/trace/2a81c80381aaeb47
FAIL (2026-03-19 03:43:55 UTC)
orbiter warning: context deadline exceeded
Duration: 2.258235054s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..................................................X.........] duration:
2.011650152s trace:
https://jaeger.dev.storj.io/trace/2a81c80381aaeb47
FAIL (2026-03-19 03:43:05 UTC)
orbiter warning: context deadline exceeded
Duration: 2.011684712s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...................................................X........] duration:
1.709736151s trace:
https://jaeger.dev.storj.io/trace/2a81c80381aaeb47
FAIL (2026-03-19 03:42:40 UTC)
orbiter warning: context deadline exceeded
Duration: 1.709766008s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [....................................................X.......] duration:
1.64212477s trace:
https://jaeger.dev.storj.io/trace/2a81c80381aaeb47
FAIL (2026-03-19 03:42:04 UTC)
orbiter warning: context deadline exceeded
Duration: 1.642152534s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.....................................................X......] duration:
1.729628796s trace:
https://jaeger.dev.storj.io/trace/2a81c80381aaeb47
FAIL (2026-03-19 03:41:39 UTC)
orbiter warning: context deadline exceeded
Duration: 1.729962631s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [......................................................X.....] duration:
2.237964775s trace:
https://jaeger.dev.storj.io/trace/2a81c80381aaeb47
FAIL (2026-03-19 03:41:20 UTC)
orbiter warning: context deadline exceeded
Duration: 2.238014646s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.......................................................X....] duration:
2.135037081s trace:
https://jaeger.dev.storj.io/trace/2a81c80381aaeb47
FAIL (2026-03-19 03:40:48 UTC)
orbiter warning: context deadline exceeded
Duration: 2.135068181s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [........................................................X...] duration:
2.209441335s trace:
https://jaeger.dev.storj.io/trace/2a81c80381aaeb47
FAIL (2026-03-19 03:40:29 UTC)
orbiter warning: context deadline exceeded
Duration: 2.209666693s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.........................................................X..] duration:
2.391471186s trace:
https://jaeger.dev.storj.io/trace/2a81c80381aaeb47
FAIL (2026-03-19 03:39:34 UTC)
orbiter warning: context deadline exceeded
Duration: 2.391506406s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..........................................................X.] duration:
2.047871875s trace:
https://jaeger.dev.storj.io/trace/2a81c80381aaeb47
FAIL (2026-03-19 03:38:41 UTC)
orbiter warning: context deadline exceeded
Duration: 2.047912435s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...........................................................X] duration:
5.001255649s trace:
https://jaeger.dev.storj.io/trace/2a81c80381aaeb47
FAIL (2026-03-19 03:37:57 UTC)
orbiter warning: context deadline exceeded
Duration: 5.00128303s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
1.027209621s trace:
https://jaeger.dev.storj.io/trace/1e14c800e54addc9
FAIL (2026-03-18 22:40:19 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 1.02725351s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
2 out of 60 is failed (threshold=3) [XX..........................................................] duration:
1.061899987s trace:
https://jaeger.dev.storj.io/trace/1e14c800e54addc9
FAIL (2026-03-18 22:39:45 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 1.061934907s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
3 out of 60 is failed (threshold=3) [XXX.........................................................] duration:
1.01953029s trace:
https://jaeger.dev.storj.io/trace/1e14c800e54addc9
FAIL (2026-03-18 22:38:40 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 1.019560237s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
2.137445852s trace:
https://jaeger.dev.storj.io/trace/61dc9d7af1d4ec01
FAIL (2026-03-18 22:38:20 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 2.137474839s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
4 out of 60 is failed (threshold=3) [XXXX........................................................] duration:
992.711163ms trace:
https://jaeger.dev.storj.io/trace/1e14c800e54addc9
FAIL (2026-03-18 22:38:18 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 992.745119ms
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
2 out of 60 is failed (threshold=3) [XX..........................................................] duration:
2.193739981s trace:
https://jaeger.dev.storj.io/trace/61dc9d7af1d4ec01
FAIL (2026-03-18 22:37:06 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 2.193767232s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
5 out of 60 is failed (threshold=3) [XXXXX.......................................................] duration:
956.183118ms trace:
https://jaeger.dev.storj.io/trace/1e14c800e54addc9
FAIL (2026-03-18 22:37:04 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 956.216711ms
core
uplink
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [X...........................................................] duration:
1.594781737s trace:
https://jaeger.dev.storj.io/trace/2c02709149f42d48
FAIL (2026-03-18 22:36:47 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.594809937s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
2.388612499s trace:
https://jaeger.dev.storj.io/trace/75e0f0868fd0d72
FAIL (2026-03-18 22:36:37 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Mar 18 21:54:18.149554843 (197h43m21.165224435s) [upload=189883]: Starting upload
Mar 18 21:54:18.149629705 (197h43m21.165299296s) [upload=189883]: Done starting upload
Mar 18 21:54:18.149726090 (197h43m21.16539568s) [upload=189883]: Uploading segments...
Mar 18 21:54:18.149968905 (197h43m21.165638498s) [upload=189883]: Got next segment. Inline: false
Mar 18 21:54:18.150024167 (197h43m21.165693769s) [upload=189883]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 18 21:54:18.150031397 (197h43m21.165700997s) [upload=189883]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 18 21:54:26.308528312 (197h43m29.324197912s) [seg_pos={0 0}, upload=189883]: Begin upload segment...
Mar 18 21:54:26.308650989 (197h43m29.324320588s) [noise=true , address=45.91.116.8:4080 , piece_id=CN3YXD7WEAUY57YFADWOVHPPANYZJBRT4VPJ6INVCISIFKB7CKCQ, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.309947151 (197h43m29.325616759s) [seg_pos={0 0}, upload=189883]: Done begin upload segment.
Mar 18 21:54:26.310024877 (197h43m29.325694487s) [noise=true , address=145.79.227.45:4230 , piece_id=O2X5ADE7BIIR3HHLF2PETI6ZWYPQSU4SNV3BVBCVNT5CZ722SEDQ, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.310648388 (197h43m29.326318s) [noise=true , address=45.91.116.4:4955 , piece_id=ZM2JV5BN5U3LKLPEQOOH4D42QHVZJFYPZCHFXU2SHXFSOJFXJVTA, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.311260449 (197h43m29.32693006s) [noise=true , address=145.79.227.22:4515 , piece_id=GYGZ7KRMHQDTPI6XZRGMPOEYXRGXMY2FURVUUYUB7AA6CFEH326Q, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.312476969 (197h43m29.328146592s) [noise=true , address=77.90.31.16:4738 , piece_id=E7IKMPRSNQXCACUY5HBRDPN4YZ37APUXQQZVHED4YQJGC7YPHZ4A, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.313742039 (197h43m29.329411642s) [noise=true , address=145.79.227.14:4841 , piece_id=GZXEN7OM7IAP5PSXSVR5SVSDKHD33ISGVVIGWQQPFS4HGLJMUROQ, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.314129731 (197h43m29.32979934s) [noise=true , address=145.79.227.43:4162 , piece_id=HIMVFURGSWUMODFHILTPC7DM3CLYZ4NSTOWLFIKN7B257EYLZTIA, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.315306083 (197h43m29.330975691s) [noise=true , address=145.79.227.46:4606 , piece_id=IDZPEIF73YXZPU2U35NW2JBCIZXJ42APUZDZLBBH2QN6HP4YNRRA, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.316537735 (197h43m29.332207343s) [noise=true , address=45.91.116.11:4118 , piece_id=YSU73JQIBOQBE5YQ7LQKSQM6FURD5OLQRPPTS55WEEEVW4KMGLPQ, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.317296012 (197h43m29.332965624s) [noise=true , address=77.90.31.13:4626 , piece_id=CLILSPUBHXP3VEGE6OQNFO7QZI564POQHZNN3267TIJVODMG4N2Q, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.318422029 (197h43m29.334091636s) [noise=true , address=145.79.227.44:4195 , piece_id=K3YARWO7WM6N3QEC4ZJWJIGP6AWRSWGZQSBYH6UU3NP6M6QFHX3Q, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.319037770 (197h43m29.334707382s) [noise=true , address=77.90.31.10:4518 , piece_id=HRYWJZEFJJ5NV6DFG7XM6VY2AFWNF5D6CFTPESBTI42J5WAW7MXA, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.320223299 (197h43m29.335892922s) [noise=true , address=145.79.227.32:4833 , piece_id=I4MJO76NNIEOOP44VSTOOY2MDNSWR2JLDSGJWQ4TSF45SYEAF3KA, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.321492900 (197h43m29.337162507s) [noise=true , address=45.91.116.7:4009 , piece_id=DWR7XXGJVAAISBACLPLWVTKZXU3EJY6SXD7CNAW7XSL6O76AMLJA, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.321765536 (197h43m29.337435143s) [noise=true , address=45.91.116.14:4193 , piece_id=KWARE3PWKRLL3DCJ6FX47725P57CGFFS4DR6UOTHDJZQQOKJSKHQ, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.323594932 (197h43m29.339264544s) [noise=true , address=77.90.31.19:4842 , piece_id=PWAEYFLRCTKDT4H7AG7LAUJKUWWU45SOQTPC4PHUIFUSFDRLBR7A, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.935519647 (197h43m29.951189247s) [upload=189883]: Next returned nil segment
Mar 18 21:54:26.935535154 (197h43m29.951204754s) [upload=189883]: Waiting for error group managing segments...
Mar 18 21:54:27.064763826 (197h43m30.080433427s) [noise=true , address=77.90.31.19:4842 , piece_id=PWAEYFLRCTKDT4H7AG7LAUJKUWWU45SOQTPC4PHUIFUSFDRLBR7A, seg_pos={0 0}, upload=189883]: Done uploading piece. err: <nil>
Mar 18 21:54:27.066336450 (197h43m30.08200605s) [noise=true , address=45.91.116.8:4080 , piece_id=CN3YXD7WEAUY57YFADWOVHPPANYZJBRT4VPJ6INVCISIFKB7CKCQ, seg_pos={0 0}, upload=189883]: Done uploading piece. err: <nil>
Mar 18 21:54:27.066710529 (197h43m30.082380129s) [noise=true , address=77.90.31.10:4518 , piece_id=HRYWJZEFJJ5NV6DFG7XM6VY2AFWNF5D6CFTPESBTI42J5WAW7MXA, seg_pos={0 0}, upload=189883]: Done uploading piece. err: <nil>
Mar 18 21:54:27.067163041 (197h43m30.082832641s) [noise=true , address=145.79.227.14:4841 , piece_id=GZXEN7OM7IAP5PSXSVR5SVSDKHD33ISGVVIGWQQPFS4HGLJMUROQ, seg_pos={0 0}, upload=189883]: Done uploading piece. err: <nil>
Mar 18 21:54:27.067539264 (197h43m30.083208864s) [noise=true , address=45.91.116.14:4193 , piece_id=KWARE3PWKRLL3DCJ6FX47725P57CGFFS4DR6UOTHDJZQQOKJSKHQ, seg_pos={0 0}, upload=189883]: Done uploading piece. err: <nil>
Mar 18 21:54:27.067890346 (197h43m30.08355995s) [noise=true , address=45.91.116.11:4118 , piece_id=YSU73JQIBOQBE5YQ7LQKSQM6FURD5OLQRPPTS55WEEEVW4KMGLPQ, seg_pos={0 0}, upload=189883]: Done uploading piece. err: <nil>
Mar 18 21:54:27.068825983 (197h43m30.084495583s) [noise=true , address=145.79.227.32:4833 , piece_id=I4MJO76NNIEOOP44VSTOOY2MDNSWR2JLDSGJWQ4TSF45SYEAF3KA, seg_pos={0 0}, upload=189883]: Done uploading piece. err: <nil>
Mar 18 21:54:27.071631864 (197h43m30.087301453s) [noise=true , address=145.79.227.46:4606 , piece_id=IDZPEIF73YXZPU2U35NW2JBCIZXJ42APUZDZLBBH2QN6HP4YNRRA, seg_pos={0 0}, upload=189883]: Done uploading piece. err: <nil>
Mar 18 21:54:27.071767013 (197h43m30.087436614s) [noise=true , address=45.91.116.7:4009 , piece_id=DWR7XXGJVAAISBACLPLWVTKZXU3EJY6SXD7CNAW7XSL6O76AMLJA, seg_pos={0 0}, upload=189883]: Done uploading piece. err: <nil>
Mar 18 21:54:27.072116600 (197h43m30.0877862s) [noise=true , address=145.79.227.44:4195 , piece_id=K3YARWO7WM6N3QEC4ZJWJIGP6AWRSWGZQSBYH6UU3NP6M6QFHX3Q, seg_pos={0 0}, upload=189883]: Done uploading piece. err: <nil>
Mar 18 21:54:27.073662705 (197h43m30.089332313s) [noise=true , address=77.90.31.13:4626 , piece_id=CLILSPUBHXP3VEGE6OQNFO7QZI564POQHZNN3267TIJVODMG4N2Q, seg_pos={0 0}, upload=189883]: Done uploading piece. err: <nil>
Mar 18 21:54:27.081111494 (197h43m30.096781094s) [noise=true , address=45.91.116.4:4955 , piece_id=ZM2JV5BN5U3LKLPEQOOH4D42QHVZJFYPZCHFXU2SHXFSOJFXJVTA, seg_pos={0 0}, upload=189883]: Done uploading piece. err: <nil>
Mar 18 21:54:27.085500985 (197h43m30.101170602s) [noise=true , address=145.79.227.45:4230 , piece_id=O2X5ADE7BIIR3HHLF2PETI6ZWYPQSU4SNV3BVBCVNT5CZ722SEDQ, seg_pos={0 0}, upload=189883]: Done uploading piece. err: <nil>
Mar 18 21:54:27.085522039 (197h43m30.10119164s) [seg_pos={0 0}, upload=189883]: Satisfied cohort requirements after 13 pieces
Mar 18 21:54:27.086116447 (197h43m30.101786058s) [noise=true , address=145.79.227.22:4515 , piece_id=GYGZ7KRMHQDTPI6XZRGMPOEYXRGXMY2FURVUUYUB7AA6CFEH326Q, seg_pos={0 0}, upload=189883]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZZmn2sRUkYuNmHY3RY7skQwAYoVYth7rJ8DhmjEpzgE4Rtmxb): protocol: expected piece hash; context canceled; EOF
Mar 18 21:54:27.086202338 (197h43m30.101871943s) [noise=true , address=145.79.227.22:4515 , piece_id=GYGZ7KRMHQDTPI6XZRGMPOEYXRGXMY2FURVUUYUB7AA6CFEH326Q, seg_pos={0 0}, upload=189883]: Long tail cancellation due to optimal threshold
Mar 18 21:54:27.086610013 (197h43m30.102279622s) [noise=true , address=77.90.31.16:4738 , piece_id=E7IKMPRSNQXCACUY5HBRDPN4YZ37APUXQQZVHED4YQJGC7YPHZ4A, seg_pos={0 0}, upload=189883]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZGynTzsrzGcGzMCqbeMJkxWwhYtPc2piotu6dhkJzrr4FnXKd): protocol: expected piece hash; context canceled; EOF
Mar 18 21:54:27.086645950 (197h43m30.10231555s) [noise=true , address=77.90.31.16:4738 , piece_id=E7IKMPRSNQXCACUY5HBRDPN4YZ37APUXQQZVHED4YQJGC7YPHZ4A, seg_pos={0 0}, upload=189883]: Long tail cancellation due to optimal threshold
Mar 18 21:54:27.086923172 (197h43m30.102592787s) [noise=true , address=145.79.227.43:4162 , piece_id=HIMVFURGSWUMODFHILTPC7DM3CLYZ4NSTOWLFIKN7B257EYLZTIA, seg_pos={0 0}, upload=189883]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt): protocol: expected piece hash; context canceled; EOF
Mar 18 21:54:27.086949914 (197h43m30.102619517s) [noise=true , address=145.79.227.43:4162 , piece_id=HIMVFURGSWUMODFHILTPC7DM3CLYZ4NSTOWLFIKN7B257EYLZTIA, seg_pos={0 0}, upload=189883]: Long tail cancellation due to optimal threshold
Mar 18 21:54:27.087062900 (197h43m30.10273249s) [seg_pos={0 0}, upload=189883]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Mar 18 21:54:27.087234507 (197h43m30.102904107s) [upload=189883]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 18 21:54:27.087240800 (197h43m30.1029104s) [upload=189883]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 18 21:54:39.677837110 (197h43m42.693506704s) [upload=189883]: Done uploading segments...
Mar 18 21:54:39.677859307 (197h43m42.693528908s) [upload=189883]: Upload finished. err: <nil>
Duration: 2.388699676s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
3 out of 60 is failed (threshold=3) [XXX.........................................................] duration:
2.177047965s trace:
https://jaeger.dev.storj.io/trace/61dc9d7af1d4ec01
FAIL (2026-03-18 22:36:04 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 2.177165616s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
931.801019ms trace:
https://jaeger.dev.storj.io/trace/65f276fda07fe2e
FAIL (2026-03-18 22:35:51 UTC)
orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:50:55.785119820 (197h39m58.800789429s) [upload=189833]: Starting upload Mar 18 21:50:55.785202430 (197h39m58.80087204s) [upload=189833]: Done starting upload Mar 18 21:50:55.785303739 (197h39m58.800973328s) [upload=189833]: Uploading segments... Mar 18 21:50:55.785387166 (197h39m58.801056775s) [upload=189833]: Got next segment. Inline: true Mar 18 21:50:55.785393596 (197h39m58.801063197s) [upload=189833]: Waiting for error group managing segments... Mar 18 21:50:55.785466465 (197h39m58.801136064s) [upload=189833]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Mar 18 21:50:55.785472300 (197h39m58.801141901s) [upload=189833]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Mar 18 21:50:55.785497156 (197h39m58.801166757s) [upload=189833]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Mar 18 21:51:20.588512781 (197h40m23.604182387s) [upload=189833]: Done uploading segments... Mar 18 21:51:20.588537265 (197h40m23.604206873s) [upload=189833]: Upload finished. err: <nil>
Duration: 931.828269ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
6 out of 60 is failed (threshold=3) [XXXXXX......................................................] duration:
934.989658ms trace:
https://jaeger.dev.storj.io/trace/1e14c800e54addc9
FAIL (2026-03-18 22:35:51 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 935.017049ms
core
uplink
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [XX..........................................................] duration:
1.860835465s trace:
https://jaeger.dev.storj.io/trace/2c02709149f42d48
FAIL (2026-03-18 22:35:38 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.860862817s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [XX..........................................................] duration:
2.465712476s trace:
https://jaeger.dev.storj.io/trace/75e0f0868fd0d72
FAIL (2026-03-18 22:35:26 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Mar 18 21:54:18.149554843 (197h43m21.165224435s) [upload=189883]: Starting upload
Mar 18 21:54:18.149629705 (197h43m21.165299296s) [upload=189883]: Done starting upload
Mar 18 21:54:18.149726090 (197h43m21.16539568s) [upload=189883]: Uploading segments...
Mar 18 21:54:18.149968905 (197h43m21.165638498s) [upload=189883]: Got next segment. Inline: false
Mar 18 21:54:18.150024167 (197h43m21.165693769s) [upload=189883]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 18 21:54:18.150031397 (197h43m21.165700997s) [upload=189883]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 18 21:54:26.308528312 (197h43m29.324197912s) [seg_pos={0 0}, upload=189883]: Begin upload segment...
Mar 18 21:54:26.308650989 (197h43m29.324320588s) [noise=true , address=45.91.116.8:4080 , piece_id=CN3YXD7WEAUY57YFADWOVHPPANYZJBRT4VPJ6INVCISIFKB7CKCQ, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.309947151 (197h43m29.325616759s) [seg_pos={0 0}, upload=189883]: Done begin upload segment.
Mar 18 21:54:26.310024877 (197h43m29.325694487s) [noise=true , address=145.79.227.45:4230 , piece_id=O2X5ADE7BIIR3HHLF2PETI6ZWYPQSU4SNV3BVBCVNT5CZ722SEDQ, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.310648388 (197h43m29.326318s) [noise=true , address=45.91.116.4:4955 , piece_id=ZM2JV5BN5U3LKLPEQOOH4D42QHVZJFYPZCHFXU2SHXFSOJFXJVTA, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.311260449 (197h43m29.32693006s) [noise=true , address=145.79.227.22:4515 , piece_id=GYGZ7KRMHQDTPI6XZRGMPOEYXRGXMY2FURVUUYUB7AA6CFEH326Q, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.312476969 (197h43m29.328146592s) [noise=true , address=77.90.31.16:4738 , piece_id=E7IKMPRSNQXCACUY5HBRDPN4YZ37APUXQQZVHED4YQJGC7YPHZ4A, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.313742039 (197h43m29.329411642s) [noise=true , address=145.79.227.14:4841 , piece_id=GZXEN7OM7IAP5PSXSVR5SVSDKHD33ISGVVIGWQQPFS4HGLJMUROQ, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.314129731 (197h43m29.32979934s) [noise=true , address=145.79.227.43:4162 , piece_id=HIMVFURGSWUMODFHILTPC7DM3CLYZ4NSTOWLFIKN7B257EYLZTIA, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.315306083 (197h43m29.330975691s) [noise=true , address=145.79.227.46:4606 , piece_id=IDZPEIF73YXZPU2U35NW2JBCIZXJ42APUZDZLBBH2QN6HP4YNRRA, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.316537735 (197h43m29.332207343s) [noise=true , address=45.91.116.11:4118 , piece_id=YSU73JQIBOQBE5YQ7LQKSQM6FURD5OLQRPPTS55WEEEVW4KMGLPQ, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.317296012 (197h43m29.332965624s) [noise=true , address=77.90.31.13:4626 , piece_id=CLILSPUBHXP3VEGE6OQNFO7QZI564POQHZNN3267TIJVODMG4N2Q, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.318422029 (197h43m29.334091636s) [noise=true , address=145.79.227.44:4195 , piece_id=K3YARWO7WM6N3QEC4ZJWJIGP6AWRSWGZQSBYH6UU3NP6M6QFHX3Q, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.319037770 (197h43m29.334707382s) [noise=true , address=77.90.31.10:4518 , piece_id=HRYWJZEFJJ5NV6DFG7XM6VY2AFWNF5D6CFTPESBTI42J5WAW7MXA, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.320223299 (197h43m29.335892922s) [noise=true , address=145.79.227.32:4833 , piece_id=I4MJO76NNIEOOP44VSTOOY2MDNSWR2JLDSGJWQ4TSF45SYEAF3KA, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.321492900 (197h43m29.337162507s) [noise=true , address=45.91.116.7:4009 , piece_id=DWR7XXGJVAAISBACLPLWVTKZXU3EJY6SXD7CNAW7XSL6O76AMLJA, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.321765536 (197h43m29.337435143s) [noise=true , address=45.91.116.14:4193 , piece_id=KWARE3PWKRLL3DCJ6FX47725P57CGFFS4DR6UOTHDJZQQOKJSKHQ, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.323594932 (197h43m29.339264544s) [noise=true , address=77.90.31.19:4842 , piece_id=PWAEYFLRCTKDT4H7AG7LAUJKUWWU45SOQTPC4PHUIFUSFDRLBR7A, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.935519647 (197h43m29.951189247s) [upload=189883]: Next returned nil segment
Mar 18 21:54:26.935535154 (197h43m29.951204754s) [upload=189883]: Waiting for error group managing segments...
Mar 18 21:54:27.064763826 (197h43m30.080433427s) [noise=true , address=77.90.31.19:4842 , piece_id=PWAEYFLRCTKDT4H7AG7LAUJKUWWU45SOQTPC4PHUIFUSFDRLBR7A, seg_pos={0 0}, upload=189883]: Done uploading piece. err: <nil>
Mar 18 21:54:27.066336450 (197h43m30.08200605s) [noise=true , address=45.91.116.8:4080 , piece_id=CN3YXD7WEAUY57YFADWOVHPPANYZJBRT4VPJ6INVCISIFKB7CKCQ, seg_pos={0 0}, upload=189883]: Done uploading piece. err: <nil>
Mar 18 21:54:27.066710529 (197h43m30.082380129s) [noise=true , address=77.90.31.10:4518 , piece_id=HRYWJZEFJJ5NV6DFG7XM6VY2AFWNF5D6CFTPESBTI42J5WAW7MXA, seg_pos={0 0}, upload=189883]: Done uploading piece. err: <nil>
Mar 18 21:54:27.067163041 (197h43m30.082832641s) [noise=true , address=145.79.227.14:4841 , piece_id=GZXEN7OM7IAP5PSXSVR5SVSDKHD33ISGVVIGWQQPFS4HGLJMUROQ, seg_pos={0 0}, upload=189883]: Done uploading piece. err: <nil>
Mar 18 21:54:27.067539264 (197h43m30.083208864s) [noise=true , address=45.91.116.14:4193 , piece_id=KWARE3PWKRLL3DCJ6FX47725P57CGFFS4DR6UOTHDJZQQOKJSKHQ, seg_pos={0 0}, upload=189883]: Done uploading piece. err: <nil>
Mar 18 21:54:27.067890346 (197h43m30.08355995s) [noise=true , address=45.91.116.11:4118 , piece_id=YSU73JQIBOQBE5YQ7LQKSQM6FURD5OLQRPPTS55WEEEVW4KMGLPQ, seg_pos={0 0}, upload=189883]: Done uploading piece. err: <nil>
Mar 18 21:54:27.068825983 (197h43m30.084495583s) [noise=true , address=145.79.227.32:4833 , piece_id=I4MJO76NNIEOOP44VSTOOY2MDNSWR2JLDSGJWQ4TSF45SYEAF3KA, seg_pos={0 0}, upload=189883]: Done uploading piece. err: <nil>
Mar 18 21:54:27.071631864 (197h43m30.087301453s) [noise=true , address=145.79.227.46:4606 , piece_id=IDZPEIF73YXZPU2U35NW2JBCIZXJ42APUZDZLBBH2QN6HP4YNRRA, seg_pos={0 0}, upload=189883]: Done uploading piece. err: <nil>
Mar 18 21:54:27.071767013 (197h43m30.087436614s) [noise=true , address=45.91.116.7:4009 , piece_id=DWR7XXGJVAAISBACLPLWVTKZXU3EJY6SXD7CNAW7XSL6O76AMLJA, seg_pos={0 0}, upload=189883]: Done uploading piece. err: <nil>
Mar 18 21:54:27.072116600 (197h43m30.0877862s) [noise=true , address=145.79.227.44:4195 , piece_id=K3YARWO7WM6N3QEC4ZJWJIGP6AWRSWGZQSBYH6UU3NP6M6QFHX3Q, seg_pos={0 0}, upload=189883]: Done uploading piece. err: <nil>
Mar 18 21:54:27.073662705 (197h43m30.089332313s) [noise=true , address=77.90.31.13:4626 , piece_id=CLILSPUBHXP3VEGE6OQNFO7QZI564POQHZNN3267TIJVODMG4N2Q, seg_pos={0 0}, upload=189883]: Done uploading piece. err: <nil>
Mar 18 21:54:27.081111494 (197h43m30.096781094s) [noise=true , address=45.91.116.4:4955 , piece_id=ZM2JV5BN5U3LKLPEQOOH4D42QHVZJFYPZCHFXU2SHXFSOJFXJVTA, seg_pos={0 0}, upload=189883]: Done uploading piece. err: <nil>
Mar 18 21:54:27.085500985 (197h43m30.101170602s) [noise=true , address=145.79.227.45:4230 , piece_id=O2X5ADE7BIIR3HHLF2PETI6ZWYPQSU4SNV3BVBCVNT5CZ722SEDQ, seg_pos={0 0}, upload=189883]: Done uploading piece. err: <nil>
Mar 18 21:54:27.085522039 (197h43m30.10119164s) [seg_pos={0 0}, upload=189883]: Satisfied cohort requirements after 13 pieces
Mar 18 21:54:27.086116447 (197h43m30.101786058s) [noise=true , address=145.79.227.22:4515 , piece_id=GYGZ7KRMHQDTPI6XZRGMPOEYXRGXMY2FURVUUYUB7AA6CFEH326Q, seg_pos={0 0}, upload=189883]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZZmn2sRUkYuNmHY3RY7skQwAYoVYth7rJ8DhmjEpzgE4Rtmxb): protocol: expected piece hash; context canceled; EOF
Mar 18 21:54:27.086202338 (197h43m30.101871943s) [noise=true , address=145.79.227.22:4515 , piece_id=GYGZ7KRMHQDTPI6XZRGMPOEYXRGXMY2FURVUUYUB7AA6CFEH326Q, seg_pos={0 0}, upload=189883]: Long tail cancellation due to optimal threshold
Mar 18 21:54:27.086610013 (197h43m30.102279622s) [noise=true , address=77.90.31.16:4738 , piece_id=E7IKMPRSNQXCACUY5HBRDPN4YZ37APUXQQZVHED4YQJGC7YPHZ4A, seg_pos={0 0}, upload=189883]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZGynTzsrzGcGzMCqbeMJkxWwhYtPc2piotu6dhkJzrr4FnXKd): protocol: expected piece hash; context canceled; EOF
Mar 18 21:54:27.086645950 (197h43m30.10231555s) [noise=true , address=77.90.31.16:4738 , piece_id=E7IKMPRSNQXCACUY5HBRDPN4YZ37APUXQQZVHED4YQJGC7YPHZ4A, seg_pos={0 0}, upload=189883]: Long tail cancellation due to optimal threshold
Mar 18 21:54:27.086923172 (197h43m30.102592787s) [noise=true , address=145.79.227.43:4162 , piece_id=HIMVFURGSWUMODFHILTPC7DM3CLYZ4NSTOWLFIKN7B257EYLZTIA, seg_pos={0 0}, upload=189883]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt): protocol: expected piece hash; context canceled; EOF
Mar 18 21:54:27.086949914 (197h43m30.102619517s) [noise=true , address=145.79.227.43:4162 , piece_id=HIMVFURGSWUMODFHILTPC7DM3CLYZ4NSTOWLFIKN7B257EYLZTIA, seg_pos={0 0}, upload=189883]: Long tail cancellation due to optimal threshold
Mar 18 21:54:27.087062900 (197h43m30.10273249s) [seg_pos={0 0}, upload=189883]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Mar 18 21:54:27.087234507 (197h43m30.102904107s) [upload=189883]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 18 21:54:27.087240800 (197h43m30.1029104s) [upload=189883]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 18 21:54:39.677837110 (197h43m42.693506704s) [upload=189883]: Done uploading segments...
Mar 18 21:54:39.677859307 (197h43m42.693528908s) [upload=189883]: Upload finished. err: <nil>
Duration: 2.465761826s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
4 out of 60 is failed (threshold=3) [XXXX........................................................] duration:
2.244700569s trace:
https://jaeger.dev.storj.io/trace/61dc9d7af1d4ec01
FAIL (2026-03-18 22:35:26 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 2.244799835s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
2 out of 60 is failed (threshold=3) [XX..........................................................] duration:
921.400118ms trace:
https://jaeger.dev.storj.io/trace/65f276fda07fe2e
FAIL (2026-03-18 22:35:15 UTC)
orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:50:55.785119820 (197h39m58.800789429s) [upload=189833]: Starting upload Mar 18 21:50:55.785202430 (197h39m58.80087204s) [upload=189833]: Done starting upload Mar 18 21:50:55.785303739 (197h39m58.800973328s) [upload=189833]: Uploading segments... Mar 18 21:50:55.785387166 (197h39m58.801056775s) [upload=189833]: Got next segment. Inline: true Mar 18 21:50:55.785393596 (197h39m58.801063197s) [upload=189833]: Waiting for error group managing segments... Mar 18 21:50:55.785466465 (197h39m58.801136064s) [upload=189833]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Mar 18 21:50:55.785472300 (197h39m58.801141901s) [upload=189833]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Mar 18 21:50:55.785497156 (197h39m58.801166757s) [upload=189833]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Mar 18 21:51:20.588512781 (197h40m23.604182387s) [upload=189833]: Done uploading segments... Mar 18 21:51:20.588537265 (197h40m23.604206873s) [upload=189833]: Upload finished. err: <nil>
Duration: 921.429623ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
7 out of 60 is failed (threshold=3) [XXXXXXX.....................................................] duration:
925.046294ms trace:
https://jaeger.dev.storj.io/trace/1e14c800e54addc9
FAIL (2026-03-18 22:35:09 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 925.081612ms
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.XX.........................................................] duration:
2.558930805s trace:
https://jaeger.dev.storj.io/trace/75e0f0868fd0d72
FAIL (2026-03-18 22:35:07 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Mar 18 21:54:18.149554843 (197h43m21.165224435s) [upload=189883]: Starting upload
Mar 18 21:54:18.149629705 (197h43m21.165299296s) [upload=189883]: Done starting upload
Mar 18 21:54:18.149726090 (197h43m21.16539568s) [upload=189883]: Uploading segments...
Mar 18 21:54:18.149968905 (197h43m21.165638498s) [upload=189883]: Got next segment. Inline: false
Mar 18 21:54:18.150024167 (197h43m21.165693769s) [upload=189883]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 18 21:54:18.150031397 (197h43m21.165700997s) [upload=189883]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 18 21:54:26.308528312 (197h43m29.324197912s) [seg_pos={0 0}, upload=189883]: Begin upload segment...
Mar 18 21:54:26.308650989 (197h43m29.324320588s) [noise=true , address=45.91.116.8:4080 , piece_id=CN3YXD7WEAUY57YFADWOVHPPANYZJBRT4VPJ6INVCISIFKB7CKCQ, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.309947151 (197h43m29.325616759s) [seg_pos={0 0}, upload=189883]: Done begin upload segment.
Mar 18 21:54:26.310024877 (197h43m29.325694487s) [noise=true , address=145.79.227.45:4230 , piece_id=O2X5ADE7BIIR3HHLF2PETI6ZWYPQSU4SNV3BVBCVNT5CZ722SEDQ, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.310648388 (197h43m29.326318s) [noise=true , address=45.91.116.4:4955 , piece_id=ZM2JV5BN5U3LKLPEQOOH4D42QHVZJFYPZCHFXU2SHXFSOJFXJVTA, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.311260449 (197h43m29.32693006s) [noise=true , address=145.79.227.22:4515 , piece_id=GYGZ7KRMHQDTPI6XZRGMPOEYXRGXMY2FURVUUYUB7AA6CFEH326Q, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.312476969 (197h43m29.328146592s) [noise=true , address=77.90.31.16:4738 , piece_id=E7IKMPRSNQXCACUY5HBRDPN4YZ37APUXQQZVHED4YQJGC7YPHZ4A, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.313742039 (197h43m29.329411642s) [noise=true , address=145.79.227.14:4841 , piece_id=GZXEN7OM7IAP5PSXSVR5SVSDKHD33ISGVVIGWQQPFS4HGLJMUROQ, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.314129731 (197h43m29.32979934s) [noise=true , address=145.79.227.43:4162 , piece_id=HIMVFURGSWUMODFHILTPC7DM3CLYZ4NSTOWLFIKN7B257EYLZTIA, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.315306083 (197h43m29.330975691s) [noise=true , address=145.79.227.46:4606 , piece_id=IDZPEIF73YXZPU2U35NW2JBCIZXJ42APUZDZLBBH2QN6HP4YNRRA, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.316537735 (197h43m29.332207343s) [noise=true , address=45.91.116.11:4118 , piece_id=YSU73JQIBOQBE5YQ7LQKSQM6FURD5OLQRPPTS55WEEEVW4KMGLPQ, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.317296012 (197h43m29.332965624s) [noise=true , address=77.90.31.13:4626 , piece_id=CLILSPUBHXP3VEGE6OQNFO7QZI564POQHZNN3267TIJVODMG4N2Q, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.318422029 (197h43m29.334091636s) [noise=true , address=145.79.227.44:4195 , piece_id=K3YARWO7WM6N3QEC4ZJWJIGP6AWRSWGZQSBYH6UU3NP6M6QFHX3Q, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.319037770 (197h43m29.334707382s) [noise=true , address=77.90.31.10:4518 , piece_id=HRYWJZEFJJ5NV6DFG7XM6VY2AFWNF5D6CFTPESBTI42J5WAW7MXA, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.320223299 (197h43m29.335892922s) [noise=true , address=145.79.227.32:4833 , piece_id=I4MJO76NNIEOOP44VSTOOY2MDNSWR2JLDSGJWQ4TSF45SYEAF3KA, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.321492900 (197h43m29.337162507s) [noise=true , address=45.91.116.7:4009 , piece_id=DWR7XXGJVAAISBACLPLWVTKZXU3EJY6SXD7CNAW7XSL6O76AMLJA, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.321765536 (197h43m29.337435143s) [noise=true , address=45.91.116.14:4193 , piece_id=KWARE3PWKRLL3DCJ6FX47725P57CGFFS4DR6UOTHDJZQQOKJSKHQ, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.323594932 (197h43m29.339264544s) [noise=true , address=77.90.31.19:4842 , piece_id=PWAEYFLRCTKDT4H7AG7LAUJKUWWU45SOQTPC4PHUIFUSFDRLBR7A, seg_pos={0 0}, upload=189883]: Uploading piece...
Mar 18 21:54:26.935519647 (197h43m29.951189247s) [upload=189883]: Next returned nil segment
Mar 18 21:54:26.935535154 (197h43m29.951204754s) [upload=189883]: Waiting for error group managing segments...
Mar 18 21:54:27.064763826 (197h43m30.080433427s) [noise=true , address=77.90.31.19:4842 , piece_id=PWAEYFLRCTKDT4H7AG7LAUJKUWWU45SOQTPC4PHUIFUSFDRLBR7A, seg_pos={0 0}, upload=189883]: Done uploading piece. err: <nil>
Mar 18 21:54:27.066336450 (197h43m30.08200605s) [noise=true , address=45.91.116.8:4080 , piece_id=CN3YXD7WEAUY57YFADWOVHPPANYZJBRT4VPJ6INVCISIFKB7CKCQ, seg_pos={0 0}, upload=189883]: Done uploading piece. err: <nil>
Mar 18 21:54:27.066710529 (197h43m30.082380129s) [noise=true , address=77.90.31.10:4518 , piece_id=HRYWJZEFJJ5NV6DFG7XM6VY2AFWNF5D6CFTPESBTI42J5WAW7MXA, seg_pos={0 0}, upload=189883]: Done uploading piece. err: <nil>
Mar 18 21:54:27.067163041 (197h43m30.082832641s) [noise=true , address=145.79.227.14:4841 , piece_id=GZXEN7OM7IAP5PSXSVR5SVSDKHD33ISGVVIGWQQPFS4HGLJMUROQ, seg_pos={0 0}, upload=189883]: Done uploading piece. err: <nil>
Mar 18 21:54:27.067539264 (197h43m30.083208864s) [noise=true , address=45.91.116.14:4193 , piece_id=KWARE3PWKRLL3DCJ6FX47725P57CGFFS4DR6UOTHDJZQQOKJSKHQ, seg_pos={0 0}, upload=189883]: Done uploading piece. err: <nil>
Mar 18 21:54:27.067890346 (197h43m30.08355995s) [noise=true , address=45.91.116.11:4118 , piece_id=YSU73JQIBOQBE5YQ7LQKSQM6FURD5OLQRPPTS55WEEEVW4KMGLPQ, seg_pos={0 0}, upload=189883]: Done uploading piece. err: <nil>
Mar 18 21:54:27.068825983 (197h43m30.084495583s) [noise=true , address=145.79.227.32:4833 , piece_id=I4MJO76NNIEOOP44VSTOOY2MDNSWR2JLDSGJWQ4TSF45SYEAF3KA, seg_pos={0 0}, upload=189883]: Done uploading piece. err: <nil>
Mar 18 21:54:27.071631864 (197h43m30.087301453s) [noise=true , address=145.79.227.46:4606 , piece_id=IDZPEIF73YXZPU2U35NW2JBCIZXJ42APUZDZLBBH2QN6HP4YNRRA, seg_pos={0 0}, upload=189883]: Done uploading piece. err: <nil>
Mar 18 21:54:27.071767013 (197h43m30.087436614s) [noise=true , address=45.91.116.7:4009 , piece_id=DWR7XXGJVAAISBACLPLWVTKZXU3EJY6SXD7CNAW7XSL6O76AMLJA, seg_pos={0 0}, upload=189883]: Done uploading piece. err: <nil>
Mar 18 21:54:27.072116600 (197h43m30.0877862s) [noise=true , address=145.79.227.44:4195 , piece_id=K3YARWO7WM6N3QEC4ZJWJIGP6AWRSWGZQSBYH6UU3NP6M6QFHX3Q, seg_pos={0 0}, upload=189883]: Done uploading piece. err: <nil>
Mar 18 21:54:27.073662705 (197h43m30.089332313s) [noise=true , address=77.90.31.13:4626 , piece_id=CLILSPUBHXP3VEGE6OQNFO7QZI564POQHZNN3267TIJVODMG4N2Q, seg_pos={0 0}, upload=189883]: Done uploading piece. err: <nil>
Mar 18 21:54:27.081111494 (197h43m30.096781094s) [noise=true , address=45.91.116.4:4955 , piece_id=ZM2JV5BN5U3LKLPEQOOH4D42QHVZJFYPZCHFXU2SHXFSOJFXJVTA, seg_pos={0 0}, upload=189883]: Done uploading piece. err: <nil>
Mar 18 21:54:27.085500985 (197h43m30.101170602s) [noise=true , address=145.79.227.45:4230 , piece_id=O2X5ADE7BIIR3HHLF2PETI6ZWYPQSU4SNV3BVBCVNT5CZ722SEDQ, seg_pos={0 0}, upload=189883]: Done uploading piece. err: <nil>
Mar 18 21:54:27.085522039 (197h43m30.10119164s) [seg_pos={0 0}, upload=189883]: Satisfied cohort requirements after 13 pieces
Mar 18 21:54:27.086116447 (197h43m30.101786058s) [noise=true , address=145.79.227.22:4515 , piece_id=GYGZ7KRMHQDTPI6XZRGMPOEYXRGXMY2FURVUUYUB7AA6CFEH326Q, seg_pos={0 0}, upload=189883]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZZmn2sRUkYuNmHY3RY7skQwAYoVYth7rJ8DhmjEpzgE4Rtmxb): protocol: expected piece hash; context canceled; EOF
Mar 18 21:54:27.086202338 (197h43m30.101871943s) [noise=true , address=145.79.227.22:4515 , piece_id=GYGZ7KRMHQDTPI6XZRGMPOEYXRGXMY2FURVUUYUB7AA6CFEH326Q, seg_pos={0 0}, upload=189883]: Long tail cancellation due to optimal threshold
Mar 18 21:54:27.086610013 (197h43m30.102279622s) [noise=true , address=77.90.31.16:4738 , piece_id=E7IKMPRSNQXCACUY5HBRDPN4YZ37APUXQQZVHED4YQJGC7YPHZ4A, seg_pos={0 0}, upload=189883]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZGynTzsrzGcGzMCqbeMJkxWwhYtPc2piotu6dhkJzrr4FnXKd): protocol: expected piece hash; context canceled; EOF
Mar 18 21:54:27.086645950 (197h43m30.10231555s) [noise=true , address=77.90.31.16:4738 , piece_id=E7IKMPRSNQXCACUY5HBRDPN4YZ37APUXQQZVHED4YQJGC7YPHZ4A, seg_pos={0 0}, upload=189883]: Long tail cancellation due to optimal threshold
Mar 18 21:54:27.086923172 (197h43m30.102592787s) [noise=true , address=145.79.227.43:4162 , piece_id=HIMVFURGSWUMODFHILTPC7DM3CLYZ4NSTOWLFIKN7B257EYLZTIA, seg_pos={0 0}, upload=189883]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt): protocol: expected piece hash; context canceled; EOF
Mar 18 21:54:27.086949914 (197h43m30.102619517s) [noise=true , address=145.79.227.43:4162 , piece_id=HIMVFURGSWUMODFHILTPC7DM3CLYZ4NSTOWLFIKN7B257EYLZTIA, seg_pos={0 0}, upload=189883]: Long tail cancellation due to optimal threshold
Mar 18 21:54:27.087062900 (197h43m30.10273249s) [seg_pos={0 0}, upload=189883]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Mar 18 21:54:27.087234507 (197h43m30.102904107s) [upload=189883]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 18 21:54:27.087240800 (197h43m30.1029104s) [upload=189883]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 18 21:54:39.677837110 (197h43m42.693506704s) [upload=189883]: Done uploading segments...
Mar 18 21:54:39.677859307 (197h43m42.693528908s) [upload=189883]: Upload finished. err: <nil>
Duration: 2.558962465s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
2 out of 60 is failed (threshold=3) [.XX.........................................................] duration:
944.121543ms trace:
https://jaeger.dev.storj.io/trace/65f276fda07fe2e
FAIL (2026-03-18 22:34:54 UTC)
orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:50:55.785119820 (197h39m58.800789429s) [upload=189833]: Starting upload Mar 18 21:50:55.785202430 (197h39m58.80087204s) [upload=189833]: Done starting upload Mar 18 21:50:55.785303739 (197h39m58.800973328s) [upload=189833]: Uploading segments... Mar 18 21:50:55.785387166 (197h39m58.801056775s) [upload=189833]: Got next segment. Inline: true Mar 18 21:50:55.785393596 (197h39m58.801063197s) [upload=189833]: Waiting for error group managing segments... Mar 18 21:50:55.785466465 (197h39m58.801136064s) [upload=189833]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Mar 18 21:50:55.785472300 (197h39m58.801141901s) [upload=189833]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Mar 18 21:50:55.785497156 (197h39m58.801166757s) [upload=189833]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Mar 18 21:51:20.588512781 (197h40m23.604182387s) [upload=189833]: Done uploading segments... Mar 18 21:51:20.588537265 (197h40m23.604206873s) [upload=189833]: Upload finished. err: <nil>
Duration: 944.148162ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
3 out of 60 is failed (threshold=6) [XXX.........................................................] duration:
1.720377456s trace:
https://jaeger.dev.storj.io/trace/2c02709149f42d48
FAIL (2026-03-18 22:34:44 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.72041265s
core
uplink
us-select-1
us1-placement-20
|