Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.................X..........................................] duration: 1.658135716s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:54:27 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.658172587s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..................X.........................................] duration: 1.599420283s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:53:34 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.599449906s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...................X........................................] duration: 1.553668099s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:52:50 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.553694246s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [....................X.......................................] duration: 1.534136749s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:51:53 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.534163974s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.....................X......................................] duration: 1.677412572s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:51:36 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.677440326s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [......................X.....................................] duration: 2.061479143s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:50:30 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 2.061575242s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.......................X....................................] duration: 1.553854634s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:50:10 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.553968488s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [........................X...................................] duration: 1.675182014s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:49:11 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.675211741s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.........................X..................................] duration: 2.065511259s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:47:58 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 2.065539472s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..........................X.................................] duration: 2.084893098s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:47:07 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 2.084920916s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...........................X................................] duration: 1.588659726s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:46:42 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.588716958s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [............................X...............................] duration: 1.587860839s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:46:07 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.58789162s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.............................X..............................] duration: 2.330107678s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:44:54 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 2.33013701s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..............................X.............................] duration: 1.578841081s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:43:58 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.578866552s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...............................X............................] duration: 1.616155556s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:43:20 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.616244542s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [................................X...........................] duration: 2.198984279s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:42:18 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 2.199017167s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.................................X..........................] duration: 1.540234736s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:41:15 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.540267423s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..................................X.........................] duration: 1.72773318s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:40:34 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.727837754s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...................................X........................] duration: 1.604226905s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:39:46 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.604257254s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [....................................X.......................] duration: 2.212791913s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:38:57 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 2.213000797s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.....................................X......................] duration: 1.632401906s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:38:15 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.632446454s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [......................................X.....................] duration: 2.206861636s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:37:14 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 2.206892216s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.......................................X....................] duration: 1.638967746s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:36:16 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.639185273s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [........................................X...................] duration: 1.73761534s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:35:41 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.737644714s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.........................................X..................] duration: 1.657806188s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:34:36 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.657913722s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..........................................X.................] duration: 1.68265572s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:34:21 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.682722661s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...........................................X................] duration: 2.114174619s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:33:15 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 2.114216173s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [............................................X...............] duration: 1.535116803s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:32:27 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.535150676s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.............................................X..............] duration: 1.556069146s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:31:22 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.556134966s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..............................................X.............] duration: 2.042741238s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:30:58 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 2.042834769s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...............................................X............] duration: 2.097484589s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:29:57 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 2.097507604s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [................................................X...........] duration: 1.62794624s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:29:27 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.628037061s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.................................................X..........] duration: 1.567597755s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:28:55 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.567625978s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..................................................X.........] duration: 2.066535225s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:27:41 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 2.066562457s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...................................................X........] duration: 1.606709722s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:26:56 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.606737062s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [....................................................X.......] duration: 1.710589425s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:25:52 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.710626157s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.....................................................X......] duration: 1.717695658s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:25:30 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.717802065s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [......................................................X.....] duration: 1.638911567s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:24:46 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.639014887s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.......................................................X....] duration: 1.585075468s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:23:57 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.585155433s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [........................................................X...] duration: 1.599431118s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:23:09 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.599473795s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.........................................................X..] duration: 1.584028237s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:22:20 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.584058104s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..........................................................X.] duration: 1.674475746s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:21:06 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.674505884s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...........................................................X] duration: 5.000865566s trace: https://jaeger.dev.storj.io/trace/64d40cf45c1bd85 FAIL (2026-03-29 00:20:00 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 5.000914658s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.499021142s trace: https://jaeger.dev.storj.io/trace/409ed236b50eb53 FAIL (2026-03-28 15:26:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 28 14:45:35.315648730 (430h34m38.331318331s) [upload=413459]: Starting upload Mar 28 14:45:35.315733599 (430h34m38.331403199s) [upload=413459]: Done starting upload Mar 28 14:45:35.315794102 (430h34m38.331463702s) [upload=413459]: Uploading segments... Mar 28 14:45:35.316032212 (430h34m38.331701813s) [upload=413459]: Got next segment. Inline: false Mar 28 14:45:35.316200570 (430h34m38.331870171s) [upload=413459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 28 14:45:35.316206390 (430h34m38.33187599s) [upload=413459]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 28 14:45:35.430724217 (430h34m38.446393826s) [seg_pos={0 0}, upload=413459]: Begin upload segment... Mar 28 14:45:35.430795561 (430h34m38.446465164s) [seg_pos={0 0}, upload=413459]: Done begin upload segment. Mar 28 14:45:35.430836859 (430h34m38.446506464s) [noise=true , address=77.90.31.19:4846 , piece_id=MOMLIPX2V2BHWIU66SML35WWK6LRRDUWRUJZ6UT2HYKOA5ZWQ5IQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.430848244 (430h34m38.446517846s) [noise=true , address=145.79.227.14:4843 , piece_id=QXIASECCPGVCCC36Y6NW3NTLJMKVPNKGXWNHTCCHME2HOOW72UIQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.431376292 (430h34m38.447045883s) [noise=true , address=79.127.219.41:4925 , piece_id=UUW3LL2MKMROAJ62NW4A66AZR337DUXIUQACIGUK2XFGV6TOC67Q, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.431872685 (430h34m38.447542294s) [noise=true , address=145.79.227.40:4051 , piece_id=6SFERDADQONW3YTOVRYHDZ7VXAOLTNZ5I3QQAS2XHG62KCNRP7TQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.432614239 (430h34m38.448283849s) [noise=true , address=145.79.227.12:4768 , piece_id=Q4ISGVZD47GU3RPO55DR4JOKIJ4SQ6UWMQPKHJAKKPOFAUJ2PZ4Q, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.432736747 (430h34m38.448406359s) [noise=true , address=145.79.227.49:4341 , piece_id=DOBHYVALR2MNRJX2ABYYBHUASIGTPTBJ5POZAZYIGSJX6K5EJLTQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.433092040 (430h34m38.448761652s) [noise=true , address=77.90.31.15:4702 , piece_id=56EUQLUUJUASC4QNLK4WJIE53EG5MF2DPD3Z7IBF3Q5J3ERJE3BQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.433377489 (430h34m38.449047102s) [noise=true , address=145.79.227.16:4911 , piece_id=JYKGUHIZQ4U74SM5BCQGXFA37I42PYXKOWB5CQQRFDX3I2EV527Q, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.434098354 (430h34m38.449767944s) [noise=true , address=79.127.219.38:4818 , piece_id=YKIWBWXFSEFPB63JTXQC4BA5YIIOCZWIOKQ4GRIBXZJ3KFCZRQKA, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.434769512 (430h34m38.450439123s) [noise=true , address=145.79.227.41:4086 , piece_id=6HKTS55E7ZO6NCX45RPJD5RF5K6DRLVRQTIZMV7ATXFVR4SACDRA, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.435075797 (430h34m38.450745407s) [noise=true , address=145.79.227.25:4617 , piece_id=WSDGMWM7CASNFGW2RPAH7MDX6O7VXEXZYLSN45ZYB2S46LPHDGOA, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.435582204 (430h34m38.451251817s) [noise=true , address=145.79.227.9:4366 , piece_id=KHTUQ2PXIXRD5Y5XIB2L3X6TMQWGZCBIS3NPOY2RS3275SCAI5BA, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.436022166 (430h34m38.451691756s) [noise=true , address=45.91.116.4:4953 , piece_id=E62ZS235DKLPVBQY6FEI3DDF3DU66LDI23ISVXANBB4M5WHQEO5Q, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.436454955 (430h34m38.452124562s) [noise=true , address=145.79.227.2:4115 , piece_id=TA4GUBLNRHKJ7DSCENFIQ5L5M7NLSJFDT4S4PVN3RRBEIIPEUYNQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.436879556 (430h34m38.452549169s) [noise=true , address=145.79.227.17:4973 , piece_id=ZZMYZGBYYVWXSOHDJGVKLNG6R3NMS2AZP4XMNCDNGJDE2DYADWIQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.437343138 (430h34m38.453012746s) [noise=true , address=145.79.227.21:4478 , piece_id=Z75OK7DR2TUDRPTM2RMX5FCKF456FYTMEVJVDBUZESZYFE6QQUTQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:36.049495517 (430h34m39.065165118s) [upload=413459]: Next returned nil segment Mar 28 14:45:36.049527631 (430h34m39.065197233s) [upload=413459]: Waiting for error group managing segments... Mar 28 14:45:36.182328788 (430h34m39.197998388s) [noise=true , address=145.79.227.17:4973 , piece_id=ZZMYZGBYYVWXSOHDJGVKLNG6R3NMS2AZP4XMNCDNGJDE2DYADWIQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.182575788 (430h34m39.198245412s) [noise=true , address=145.79.227.25:4617 , piece_id=WSDGMWM7CASNFGW2RPAH7MDX6O7VXEXZYLSN45ZYB2S46LPHDGOA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.182982538 (430h34m39.198652144s) [noise=true , address=145.79.227.9:4366 , piece_id=KHTUQ2PXIXRD5Y5XIB2L3X6TMQWGZCBIS3NPOY2RS3275SCAI5BA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.183438894 (430h34m39.199108504s) [noise=true , address=45.91.116.4:4953 , piece_id=E62ZS235DKLPVBQY6FEI3DDF3DU66LDI23ISVXANBB4M5WHQEO5Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.184168119 (430h34m39.199837707s) [noise=true , address=145.79.227.12:4768 , piece_id=Q4ISGVZD47GU3RPO55DR4JOKIJ4SQ6UWMQPKHJAKKPOFAUJ2PZ4Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.184865710 (430h34m39.200535314s) [noise=true , address=145.79.227.40:4051 , piece_id=6SFERDADQONW3YTOVRYHDZ7VXAOLTNZ5I3QQAS2XHG62KCNRP7TQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.185089145 (430h34m39.20075875s) [noise=true , address=145.79.227.16:4911 , piece_id=JYKGUHIZQ4U74SM5BCQGXFA37I42PYXKOWB5CQQRFDX3I2EV527Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.188553089 (430h34m39.204222694s) [noise=true , address=145.79.227.14:4843 , piece_id=QXIASECCPGVCCC36Y6NW3NTLJMKVPNKGXWNHTCCHME2HOOW72UIQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.190225530 (430h34m39.205895143s) [noise=true , address=145.79.227.2:4115 , piece_id=TA4GUBLNRHKJ7DSCENFIQ5L5M7NLSJFDT4S4PVN3RRBEIIPEUYNQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.190551513 (430h34m39.206221103s) [noise=true , address=145.79.227.21:4478 , piece_id=Z75OK7DR2TUDRPTM2RMX5FCKF456FYTMEVJVDBUZESZYFE6QQUTQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.191113981 (430h34m39.206783572s) [noise=true , address=145.79.227.41:4086 , piece_id=6HKTS55E7ZO6NCX45RPJD5RF5K6DRLVRQTIZMV7ATXFVR4SACDRA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.289167312 (430h34m39.304836941s) [noise=true , address=145.79.227.49:4341 , piece_id=DOBHYVALR2MNRJX2ABYYBHUASIGTPTBJ5POZAZYIGSJX6K5EJLTQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:46:05.029585012 (430h35m8.045254629s) [noise=true , address=77.90.31.15:4702 , piece_id=56EUQLUUJUASC4QNLK4WJIE53EG5MF2DPD3Z7IBF3Q5J3ERJE3BQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:1ucSAZVBqRRJU7WZDKs7Cs6yKV73haX7px76sfukA6b8Q8DDsW, address:77.90.31.15:4702): protocol: write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection; write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection Mar 28 14:46:05.030074673 (430h35m8.045744285s) [noise=true , address=79.127.219.41:4925 , piece_id=UUW3LL2MKMROAJ62NW4A66AZR337DUXIUQACIGUK2XFGV6TOC67Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:1kBG5b1fGn5dLzZoD8v4KG6CHStmw1QhGpqFdtGsMsDUEXwZzs, address:79.127.219.41:4925): protocol: read tcp 139.59.144.22:33812->79.127.219.41:4925: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 28 14:46:05.030297717 (430h35m8.04596733s) [noise=true , address=77.90.31.19:4846 , piece_id=MOMLIPX2V2BHWIU66SML35WWK6LRRDUWRUJZ6UT2HYKOA5ZWQ5IQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:12V76dqXhBHet772yUXG9oRU9r1m7rtXQeARRVzTeGrcJxxzyvM, address:77.90.31.19:4846): protocol: context deadline exceeded; context deadline exceeded Mar 28 14:46:05.030604513 (430h35m8.04627423s) [noise=true , address=79.127.219.38:4818 , piece_id=YKIWBWXFSEFPB63JTXQC4BA5YIIOCZWIOKQ4GRIBXZJ3KFCZRQKA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:12ZvUdXiXPFGqmVzYffAp735GjzaSMcyUfTD61Yi6us9ETg7Ath, address:79.127.219.38:4818): protocol: read tcp 139.59.144.22:58514->79.127.219.38:4818: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 28 14:46:05.030645703 (430h35m8.046315306s) [seg_pos={0 0}, upload=413459]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 28 14:46:05.141824246 (430h35m8.157493849s) [upload=413459]: Done uploading segments... Mar 28 14:46:05.141873159 (430h35m8.157542765s) [upload=413459]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.499050345s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 2.444942531s trace: https://jaeger.dev.storj.io/trace/409ed236b50eb53 FAIL (2026-03-28 15:25:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 28 14:45:35.315648730 (430h34m38.331318331s) [upload=413459]: Starting upload Mar 28 14:45:35.315733599 (430h34m38.331403199s) [upload=413459]: Done starting upload Mar 28 14:45:35.315794102 (430h34m38.331463702s) [upload=413459]: Uploading segments... Mar 28 14:45:35.316032212 (430h34m38.331701813s) [upload=413459]: Got next segment. Inline: false Mar 28 14:45:35.316200570 (430h34m38.331870171s) [upload=413459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 28 14:45:35.316206390 (430h34m38.33187599s) [upload=413459]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 28 14:45:35.430724217 (430h34m38.446393826s) [seg_pos={0 0}, upload=413459]: Begin upload segment... Mar 28 14:45:35.430795561 (430h34m38.446465164s) [seg_pos={0 0}, upload=413459]: Done begin upload segment. Mar 28 14:45:35.430836859 (430h34m38.446506464s) [noise=true , address=77.90.31.19:4846 , piece_id=MOMLIPX2V2BHWIU66SML35WWK6LRRDUWRUJZ6UT2HYKOA5ZWQ5IQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.430848244 (430h34m38.446517846s) [noise=true , address=145.79.227.14:4843 , piece_id=QXIASECCPGVCCC36Y6NW3NTLJMKVPNKGXWNHTCCHME2HOOW72UIQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.431376292 (430h34m38.447045883s) [noise=true , address=79.127.219.41:4925 , piece_id=UUW3LL2MKMROAJ62NW4A66AZR337DUXIUQACIGUK2XFGV6TOC67Q, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.431872685 (430h34m38.447542294s) [noise=true , address=145.79.227.40:4051 , piece_id=6SFERDADQONW3YTOVRYHDZ7VXAOLTNZ5I3QQAS2XHG62KCNRP7TQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.432614239 (430h34m38.448283849s) [noise=true , address=145.79.227.12:4768 , piece_id=Q4ISGVZD47GU3RPO55DR4JOKIJ4SQ6UWMQPKHJAKKPOFAUJ2PZ4Q, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.432736747 (430h34m38.448406359s) [noise=true , address=145.79.227.49:4341 , piece_id=DOBHYVALR2MNRJX2ABYYBHUASIGTPTBJ5POZAZYIGSJX6K5EJLTQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.433092040 (430h34m38.448761652s) [noise=true , address=77.90.31.15:4702 , piece_id=56EUQLUUJUASC4QNLK4WJIE53EG5MF2DPD3Z7IBF3Q5J3ERJE3BQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.433377489 (430h34m38.449047102s) [noise=true , address=145.79.227.16:4911 , piece_id=JYKGUHIZQ4U74SM5BCQGXFA37I42PYXKOWB5CQQRFDX3I2EV527Q, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.434098354 (430h34m38.449767944s) [noise=true , address=79.127.219.38:4818 , piece_id=YKIWBWXFSEFPB63JTXQC4BA5YIIOCZWIOKQ4GRIBXZJ3KFCZRQKA, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.434769512 (430h34m38.450439123s) [noise=true , address=145.79.227.41:4086 , piece_id=6HKTS55E7ZO6NCX45RPJD5RF5K6DRLVRQTIZMV7ATXFVR4SACDRA, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.435075797 (430h34m38.450745407s) [noise=true , address=145.79.227.25:4617 , piece_id=WSDGMWM7CASNFGW2RPAH7MDX6O7VXEXZYLSN45ZYB2S46LPHDGOA, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.435582204 (430h34m38.451251817s) [noise=true , address=145.79.227.9:4366 , piece_id=KHTUQ2PXIXRD5Y5XIB2L3X6TMQWGZCBIS3NPOY2RS3275SCAI5BA, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.436022166 (430h34m38.451691756s) [noise=true , address=45.91.116.4:4953 , piece_id=E62ZS235DKLPVBQY6FEI3DDF3DU66LDI23ISVXANBB4M5WHQEO5Q, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.436454955 (430h34m38.452124562s) [noise=true , address=145.79.227.2:4115 , piece_id=TA4GUBLNRHKJ7DSCENFIQ5L5M7NLSJFDT4S4PVN3RRBEIIPEUYNQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.436879556 (430h34m38.452549169s) [noise=true , address=145.79.227.17:4973 , piece_id=ZZMYZGBYYVWXSOHDJGVKLNG6R3NMS2AZP4XMNCDNGJDE2DYADWIQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.437343138 (430h34m38.453012746s) [noise=true , address=145.79.227.21:4478 , piece_id=Z75OK7DR2TUDRPTM2RMX5FCKF456FYTMEVJVDBUZESZYFE6QQUTQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:36.049495517 (430h34m39.065165118s) [upload=413459]: Next returned nil segment Mar 28 14:45:36.049527631 (430h34m39.065197233s) [upload=413459]: Waiting for error group managing segments... Mar 28 14:45:36.182328788 (430h34m39.197998388s) [noise=true , address=145.79.227.17:4973 , piece_id=ZZMYZGBYYVWXSOHDJGVKLNG6R3NMS2AZP4XMNCDNGJDE2DYADWIQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.182575788 (430h34m39.198245412s) [noise=true , address=145.79.227.25:4617 , piece_id=WSDGMWM7CASNFGW2RPAH7MDX6O7VXEXZYLSN45ZYB2S46LPHDGOA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.182982538 (430h34m39.198652144s) [noise=true , address=145.79.227.9:4366 , piece_id=KHTUQ2PXIXRD5Y5XIB2L3X6TMQWGZCBIS3NPOY2RS3275SCAI5BA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.183438894 (430h34m39.199108504s) [noise=true , address=45.91.116.4:4953 , piece_id=E62ZS235DKLPVBQY6FEI3DDF3DU66LDI23ISVXANBB4M5WHQEO5Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.184168119 (430h34m39.199837707s) [noise=true , address=145.79.227.12:4768 , piece_id=Q4ISGVZD47GU3RPO55DR4JOKIJ4SQ6UWMQPKHJAKKPOFAUJ2PZ4Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.184865710 (430h34m39.200535314s) [noise=true , address=145.79.227.40:4051 , piece_id=6SFERDADQONW3YTOVRYHDZ7VXAOLTNZ5I3QQAS2XHG62KCNRP7TQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.185089145 (430h34m39.20075875s) [noise=true , address=145.79.227.16:4911 , piece_id=JYKGUHIZQ4U74SM5BCQGXFA37I42PYXKOWB5CQQRFDX3I2EV527Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.188553089 (430h34m39.204222694s) [noise=true , address=145.79.227.14:4843 , piece_id=QXIASECCPGVCCC36Y6NW3NTLJMKVPNKGXWNHTCCHME2HOOW72UIQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.190225530 (430h34m39.205895143s) [noise=true , address=145.79.227.2:4115 , piece_id=TA4GUBLNRHKJ7DSCENFIQ5L5M7NLSJFDT4S4PVN3RRBEIIPEUYNQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.190551513 (430h34m39.206221103s) [noise=true , address=145.79.227.21:4478 , piece_id=Z75OK7DR2TUDRPTM2RMX5FCKF456FYTMEVJVDBUZESZYFE6QQUTQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.191113981 (430h34m39.206783572s) [noise=true , address=145.79.227.41:4086 , piece_id=6HKTS55E7ZO6NCX45RPJD5RF5K6DRLVRQTIZMV7ATXFVR4SACDRA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.289167312 (430h34m39.304836941s) [noise=true , address=145.79.227.49:4341 , piece_id=DOBHYVALR2MNRJX2ABYYBHUASIGTPTBJ5POZAZYIGSJX6K5EJLTQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:46:05.029585012 (430h35m8.045254629s) [noise=true , address=77.90.31.15:4702 , piece_id=56EUQLUUJUASC4QNLK4WJIE53EG5MF2DPD3Z7IBF3Q5J3ERJE3BQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:1ucSAZVBqRRJU7WZDKs7Cs6yKV73haX7px76sfukA6b8Q8DDsW, address:77.90.31.15:4702): protocol: write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection; write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection Mar 28 14:46:05.030074673 (430h35m8.045744285s) [noise=true , address=79.127.219.41:4925 , piece_id=UUW3LL2MKMROAJ62NW4A66AZR337DUXIUQACIGUK2XFGV6TOC67Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:1kBG5b1fGn5dLzZoD8v4KG6CHStmw1QhGpqFdtGsMsDUEXwZzs, address:79.127.219.41:4925): protocol: read tcp 139.59.144.22:33812->79.127.219.41:4925: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 28 14:46:05.030297717 (430h35m8.04596733s) [noise=true , address=77.90.31.19:4846 , piece_id=MOMLIPX2V2BHWIU66SML35WWK6LRRDUWRUJZ6UT2HYKOA5ZWQ5IQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:12V76dqXhBHet772yUXG9oRU9r1m7rtXQeARRVzTeGrcJxxzyvM, address:77.90.31.19:4846): protocol: context deadline exceeded; context deadline exceeded Mar 28 14:46:05.030604513 (430h35m8.04627423s) [noise=true , address=79.127.219.38:4818 , piece_id=YKIWBWXFSEFPB63JTXQC4BA5YIIOCZWIOKQ4GRIBXZJ3KFCZRQKA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:12ZvUdXiXPFGqmVzYffAp735GjzaSMcyUfTD61Yi6us9ETg7Ath, address:79.127.219.38:4818): protocol: read tcp 139.59.144.22:58514->79.127.219.38:4818: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 28 14:46:05.030645703 (430h35m8.046315306s) [seg_pos={0 0}, upload=413459]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 28 14:46:05.141824246 (430h35m8.157493849s) [upload=413459]: Done uploading segments... Mar 28 14:46:05.141873159 (430h35m8.157542765s) [upload=413459]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.444973542s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 2.497811815s trace: https://jaeger.dev.storj.io/trace/409ed236b50eb53 FAIL (2026-03-28 15:24:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 28 14:45:35.315648730 (430h34m38.331318331s) [upload=413459]: Starting upload Mar 28 14:45:35.315733599 (430h34m38.331403199s) [upload=413459]: Done starting upload Mar 28 14:45:35.315794102 (430h34m38.331463702s) [upload=413459]: Uploading segments... Mar 28 14:45:35.316032212 (430h34m38.331701813s) [upload=413459]: Got next segment. Inline: false Mar 28 14:45:35.316200570 (430h34m38.331870171s) [upload=413459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 28 14:45:35.316206390 (430h34m38.33187599s) [upload=413459]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 28 14:45:35.430724217 (430h34m38.446393826s) [seg_pos={0 0}, upload=413459]: Begin upload segment... Mar 28 14:45:35.430795561 (430h34m38.446465164s) [seg_pos={0 0}, upload=413459]: Done begin upload segment. Mar 28 14:45:35.430836859 (430h34m38.446506464s) [noise=true , address=77.90.31.19:4846 , piece_id=MOMLIPX2V2BHWIU66SML35WWK6LRRDUWRUJZ6UT2HYKOA5ZWQ5IQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.430848244 (430h34m38.446517846s) [noise=true , address=145.79.227.14:4843 , piece_id=QXIASECCPGVCCC36Y6NW3NTLJMKVPNKGXWNHTCCHME2HOOW72UIQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.431376292 (430h34m38.447045883s) [noise=true , address=79.127.219.41:4925 , piece_id=UUW3LL2MKMROAJ62NW4A66AZR337DUXIUQACIGUK2XFGV6TOC67Q, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.431872685 (430h34m38.447542294s) [noise=true , address=145.79.227.40:4051 , piece_id=6SFERDADQONW3YTOVRYHDZ7VXAOLTNZ5I3QQAS2XHG62KCNRP7TQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.432614239 (430h34m38.448283849s) [noise=true , address=145.79.227.12:4768 , piece_id=Q4ISGVZD47GU3RPO55DR4JOKIJ4SQ6UWMQPKHJAKKPOFAUJ2PZ4Q, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.432736747 (430h34m38.448406359s) [noise=true , address=145.79.227.49:4341 , piece_id=DOBHYVALR2MNRJX2ABYYBHUASIGTPTBJ5POZAZYIGSJX6K5EJLTQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.433092040 (430h34m38.448761652s) [noise=true , address=77.90.31.15:4702 , piece_id=56EUQLUUJUASC4QNLK4WJIE53EG5MF2DPD3Z7IBF3Q5J3ERJE3BQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.433377489 (430h34m38.449047102s) [noise=true , address=145.79.227.16:4911 , piece_id=JYKGUHIZQ4U74SM5BCQGXFA37I42PYXKOWB5CQQRFDX3I2EV527Q, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.434098354 (430h34m38.449767944s) [noise=true , address=79.127.219.38:4818 , piece_id=YKIWBWXFSEFPB63JTXQC4BA5YIIOCZWIOKQ4GRIBXZJ3KFCZRQKA, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.434769512 (430h34m38.450439123s) [noise=true , address=145.79.227.41:4086 , piece_id=6HKTS55E7ZO6NCX45RPJD5RF5K6DRLVRQTIZMV7ATXFVR4SACDRA, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.435075797 (430h34m38.450745407s) [noise=true , address=145.79.227.25:4617 , piece_id=WSDGMWM7CASNFGW2RPAH7MDX6O7VXEXZYLSN45ZYB2S46LPHDGOA, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.435582204 (430h34m38.451251817s) [noise=true , address=145.79.227.9:4366 , piece_id=KHTUQ2PXIXRD5Y5XIB2L3X6TMQWGZCBIS3NPOY2RS3275SCAI5BA, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.436022166 (430h34m38.451691756s) [noise=true , address=45.91.116.4:4953 , piece_id=E62ZS235DKLPVBQY6FEI3DDF3DU66LDI23ISVXANBB4M5WHQEO5Q, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.436454955 (430h34m38.452124562s) [noise=true , address=145.79.227.2:4115 , piece_id=TA4GUBLNRHKJ7DSCENFIQ5L5M7NLSJFDT4S4PVN3RRBEIIPEUYNQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.436879556 (430h34m38.452549169s) [noise=true , address=145.79.227.17:4973 , piece_id=ZZMYZGBYYVWXSOHDJGVKLNG6R3NMS2AZP4XMNCDNGJDE2DYADWIQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.437343138 (430h34m38.453012746s) [noise=true , address=145.79.227.21:4478 , piece_id=Z75OK7DR2TUDRPTM2RMX5FCKF456FYTMEVJVDBUZESZYFE6QQUTQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:36.049495517 (430h34m39.065165118s) [upload=413459]: Next returned nil segment Mar 28 14:45:36.049527631 (430h34m39.065197233s) [upload=413459]: Waiting for error group managing segments... Mar 28 14:45:36.182328788 (430h34m39.197998388s) [noise=true , address=145.79.227.17:4973 , piece_id=ZZMYZGBYYVWXSOHDJGVKLNG6R3NMS2AZP4XMNCDNGJDE2DYADWIQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.182575788 (430h34m39.198245412s) [noise=true , address=145.79.227.25:4617 , piece_id=WSDGMWM7CASNFGW2RPAH7MDX6O7VXEXZYLSN45ZYB2S46LPHDGOA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.182982538 (430h34m39.198652144s) [noise=true , address=145.79.227.9:4366 , piece_id=KHTUQ2PXIXRD5Y5XIB2L3X6TMQWGZCBIS3NPOY2RS3275SCAI5BA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.183438894 (430h34m39.199108504s) [noise=true , address=45.91.116.4:4953 , piece_id=E62ZS235DKLPVBQY6FEI3DDF3DU66LDI23ISVXANBB4M5WHQEO5Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.184168119 (430h34m39.199837707s) [noise=true , address=145.79.227.12:4768 , piece_id=Q4ISGVZD47GU3RPO55DR4JOKIJ4SQ6UWMQPKHJAKKPOFAUJ2PZ4Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.184865710 (430h34m39.200535314s) [noise=true , address=145.79.227.40:4051 , piece_id=6SFERDADQONW3YTOVRYHDZ7VXAOLTNZ5I3QQAS2XHG62KCNRP7TQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.185089145 (430h34m39.20075875s) [noise=true , address=145.79.227.16:4911 , piece_id=JYKGUHIZQ4U74SM5BCQGXFA37I42PYXKOWB5CQQRFDX3I2EV527Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.188553089 (430h34m39.204222694s) [noise=true , address=145.79.227.14:4843 , piece_id=QXIASECCPGVCCC36Y6NW3NTLJMKVPNKGXWNHTCCHME2HOOW72UIQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.190225530 (430h34m39.205895143s) [noise=true , address=145.79.227.2:4115 , piece_id=TA4GUBLNRHKJ7DSCENFIQ5L5M7NLSJFDT4S4PVN3RRBEIIPEUYNQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.190551513 (430h34m39.206221103s) [noise=true , address=145.79.227.21:4478 , piece_id=Z75OK7DR2TUDRPTM2RMX5FCKF456FYTMEVJVDBUZESZYFE6QQUTQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.191113981 (430h34m39.206783572s) [noise=true , address=145.79.227.41:4086 , piece_id=6HKTS55E7ZO6NCX45RPJD5RF5K6DRLVRQTIZMV7ATXFVR4SACDRA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.289167312 (430h34m39.304836941s) [noise=true , address=145.79.227.49:4341 , piece_id=DOBHYVALR2MNRJX2ABYYBHUASIGTPTBJ5POZAZYIGSJX6K5EJLTQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:46:05.029585012 (430h35m8.045254629s) [noise=true , address=77.90.31.15:4702 , piece_id=56EUQLUUJUASC4QNLK4WJIE53EG5MF2DPD3Z7IBF3Q5J3ERJE3BQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:1ucSAZVBqRRJU7WZDKs7Cs6yKV73haX7px76sfukA6b8Q8DDsW, address:77.90.31.15:4702): protocol: write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection; write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection Mar 28 14:46:05.030074673 (430h35m8.045744285s) [noise=true , address=79.127.219.41:4925 , piece_id=UUW3LL2MKMROAJ62NW4A66AZR337DUXIUQACIGUK2XFGV6TOC67Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:1kBG5b1fGn5dLzZoD8v4KG6CHStmw1QhGpqFdtGsMsDUEXwZzs, address:79.127.219.41:4925): protocol: read tcp 139.59.144.22:33812->79.127.219.41:4925: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 28 14:46:05.030297717 (430h35m8.04596733s) [noise=true , address=77.90.31.19:4846 , piece_id=MOMLIPX2V2BHWIU66SML35WWK6LRRDUWRUJZ6UT2HYKOA5ZWQ5IQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:12V76dqXhBHet772yUXG9oRU9r1m7rtXQeARRVzTeGrcJxxzyvM, address:77.90.31.19:4846): protocol: context deadline exceeded; context deadline exceeded Mar 28 14:46:05.030604513 (430h35m8.04627423s) [noise=true , address=79.127.219.38:4818 , piece_id=YKIWBWXFSEFPB63JTXQC4BA5YIIOCZWIOKQ4GRIBXZJ3KFCZRQKA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:12ZvUdXiXPFGqmVzYffAp735GjzaSMcyUfTD61Yi6us9ETg7Ath, address:79.127.219.38:4818): protocol: read tcp 139.59.144.22:58514->79.127.219.38:4818: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 28 14:46:05.030645703 (430h35m8.046315306s) [seg_pos={0 0}, upload=413459]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 28 14:46:05.141824246 (430h35m8.157493849s) [upload=413459]: Done uploading segments... Mar 28 14:46:05.141873159 (430h35m8.157542765s) [upload=413459]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.497847984s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 2.424364325s trace: https://jaeger.dev.storj.io/trace/409ed236b50eb53 FAIL (2026-03-28 15:24:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 28 14:45:35.315648730 (430h34m38.331318331s) [upload=413459]: Starting upload Mar 28 14:45:35.315733599 (430h34m38.331403199s) [upload=413459]: Done starting upload Mar 28 14:45:35.315794102 (430h34m38.331463702s) [upload=413459]: Uploading segments... Mar 28 14:45:35.316032212 (430h34m38.331701813s) [upload=413459]: Got next segment. Inline: false Mar 28 14:45:35.316200570 (430h34m38.331870171s) [upload=413459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 28 14:45:35.316206390 (430h34m38.33187599s) [upload=413459]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 28 14:45:35.430724217 (430h34m38.446393826s) [seg_pos={0 0}, upload=413459]: Begin upload segment... Mar 28 14:45:35.430795561 (430h34m38.446465164s) [seg_pos={0 0}, upload=413459]: Done begin upload segment. Mar 28 14:45:35.430836859 (430h34m38.446506464s) [noise=true , address=77.90.31.19:4846 , piece_id=MOMLIPX2V2BHWIU66SML35WWK6LRRDUWRUJZ6UT2HYKOA5ZWQ5IQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.430848244 (430h34m38.446517846s) [noise=true , address=145.79.227.14:4843 , piece_id=QXIASECCPGVCCC36Y6NW3NTLJMKVPNKGXWNHTCCHME2HOOW72UIQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.431376292 (430h34m38.447045883s) [noise=true , address=79.127.219.41:4925 , piece_id=UUW3LL2MKMROAJ62NW4A66AZR337DUXIUQACIGUK2XFGV6TOC67Q, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.431872685 (430h34m38.447542294s) [noise=true , address=145.79.227.40:4051 , piece_id=6SFERDADQONW3YTOVRYHDZ7VXAOLTNZ5I3QQAS2XHG62KCNRP7TQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.432614239 (430h34m38.448283849s) [noise=true , address=145.79.227.12:4768 , piece_id=Q4ISGVZD47GU3RPO55DR4JOKIJ4SQ6UWMQPKHJAKKPOFAUJ2PZ4Q, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.432736747 (430h34m38.448406359s) [noise=true , address=145.79.227.49:4341 , piece_id=DOBHYVALR2MNRJX2ABYYBHUASIGTPTBJ5POZAZYIGSJX6K5EJLTQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.433092040 (430h34m38.448761652s) [noise=true , address=77.90.31.15:4702 , piece_id=56EUQLUUJUASC4QNLK4WJIE53EG5MF2DPD3Z7IBF3Q5J3ERJE3BQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.433377489 (430h34m38.449047102s) [noise=true , address=145.79.227.16:4911 , piece_id=JYKGUHIZQ4U74SM5BCQGXFA37I42PYXKOWB5CQQRFDX3I2EV527Q, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.434098354 (430h34m38.449767944s) [noise=true , address=79.127.219.38:4818 , piece_id=YKIWBWXFSEFPB63JTXQC4BA5YIIOCZWIOKQ4GRIBXZJ3KFCZRQKA, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.434769512 (430h34m38.450439123s) [noise=true , address=145.79.227.41:4086 , piece_id=6HKTS55E7ZO6NCX45RPJD5RF5K6DRLVRQTIZMV7ATXFVR4SACDRA, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.435075797 (430h34m38.450745407s) [noise=true , address=145.79.227.25:4617 , piece_id=WSDGMWM7CASNFGW2RPAH7MDX6O7VXEXZYLSN45ZYB2S46LPHDGOA, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.435582204 (430h34m38.451251817s) [noise=true , address=145.79.227.9:4366 , piece_id=KHTUQ2PXIXRD5Y5XIB2L3X6TMQWGZCBIS3NPOY2RS3275SCAI5BA, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.436022166 (430h34m38.451691756s) [noise=true , address=45.91.116.4:4953 , piece_id=E62ZS235DKLPVBQY6FEI3DDF3DU66LDI23ISVXANBB4M5WHQEO5Q, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.436454955 (430h34m38.452124562s) [noise=true , address=145.79.227.2:4115 , piece_id=TA4GUBLNRHKJ7DSCENFIQ5L5M7NLSJFDT4S4PVN3RRBEIIPEUYNQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.436879556 (430h34m38.452549169s) [noise=true , address=145.79.227.17:4973 , piece_id=ZZMYZGBYYVWXSOHDJGVKLNG6R3NMS2AZP4XMNCDNGJDE2DYADWIQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.437343138 (430h34m38.453012746s) [noise=true , address=145.79.227.21:4478 , piece_id=Z75OK7DR2TUDRPTM2RMX5FCKF456FYTMEVJVDBUZESZYFE6QQUTQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:36.049495517 (430h34m39.065165118s) [upload=413459]: Next returned nil segment Mar 28 14:45:36.049527631 (430h34m39.065197233s) [upload=413459]: Waiting for error group managing segments... Mar 28 14:45:36.182328788 (430h34m39.197998388s) [noise=true , address=145.79.227.17:4973 , piece_id=ZZMYZGBYYVWXSOHDJGVKLNG6R3NMS2AZP4XMNCDNGJDE2DYADWIQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.182575788 (430h34m39.198245412s) [noise=true , address=145.79.227.25:4617 , piece_id=WSDGMWM7CASNFGW2RPAH7MDX6O7VXEXZYLSN45ZYB2S46LPHDGOA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.182982538 (430h34m39.198652144s) [noise=true , address=145.79.227.9:4366 , piece_id=KHTUQ2PXIXRD5Y5XIB2L3X6TMQWGZCBIS3NPOY2RS3275SCAI5BA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.183438894 (430h34m39.199108504s) [noise=true , address=45.91.116.4:4953 , piece_id=E62ZS235DKLPVBQY6FEI3DDF3DU66LDI23ISVXANBB4M5WHQEO5Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.184168119 (430h34m39.199837707s) [noise=true , address=145.79.227.12:4768 , piece_id=Q4ISGVZD47GU3RPO55DR4JOKIJ4SQ6UWMQPKHJAKKPOFAUJ2PZ4Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.184865710 (430h34m39.200535314s) [noise=true , address=145.79.227.40:4051 , piece_id=6SFERDADQONW3YTOVRYHDZ7VXAOLTNZ5I3QQAS2XHG62KCNRP7TQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.185089145 (430h34m39.20075875s) [noise=true , address=145.79.227.16:4911 , piece_id=JYKGUHIZQ4U74SM5BCQGXFA37I42PYXKOWB5CQQRFDX3I2EV527Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.188553089 (430h34m39.204222694s) [noise=true , address=145.79.227.14:4843 , piece_id=QXIASECCPGVCCC36Y6NW3NTLJMKVPNKGXWNHTCCHME2HOOW72UIQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.190225530 (430h34m39.205895143s) [noise=true , address=145.79.227.2:4115 , piece_id=TA4GUBLNRHKJ7DSCENFIQ5L5M7NLSJFDT4S4PVN3RRBEIIPEUYNQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.190551513 (430h34m39.206221103s) [noise=true , address=145.79.227.21:4478 , piece_id=Z75OK7DR2TUDRPTM2RMX5FCKF456FYTMEVJVDBUZESZYFE6QQUTQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.191113981 (430h34m39.206783572s) [noise=true , address=145.79.227.41:4086 , piece_id=6HKTS55E7ZO6NCX45RPJD5RF5K6DRLVRQTIZMV7ATXFVR4SACDRA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.289167312 (430h34m39.304836941s) [noise=true , address=145.79.227.49:4341 , piece_id=DOBHYVALR2MNRJX2ABYYBHUASIGTPTBJ5POZAZYIGSJX6K5EJLTQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:46:05.029585012 (430h35m8.045254629s) [noise=true , address=77.90.31.15:4702 , piece_id=56EUQLUUJUASC4QNLK4WJIE53EG5MF2DPD3Z7IBF3Q5J3ERJE3BQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:1ucSAZVBqRRJU7WZDKs7Cs6yKV73haX7px76sfukA6b8Q8DDsW, address:77.90.31.15:4702): protocol: write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection; write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection Mar 28 14:46:05.030074673 (430h35m8.045744285s) [noise=true , address=79.127.219.41:4925 , piece_id=UUW3LL2MKMROAJ62NW4A66AZR337DUXIUQACIGUK2XFGV6TOC67Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:1kBG5b1fGn5dLzZoD8v4KG6CHStmw1QhGpqFdtGsMsDUEXwZzs, address:79.127.219.41:4925): protocol: read tcp 139.59.144.22:33812->79.127.219.41:4925: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 28 14:46:05.030297717 (430h35m8.04596733s) [noise=true , address=77.90.31.19:4846 , piece_id=MOMLIPX2V2BHWIU66SML35WWK6LRRDUWRUJZ6UT2HYKOA5ZWQ5IQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:12V76dqXhBHet772yUXG9oRU9r1m7rtXQeARRVzTeGrcJxxzyvM, address:77.90.31.19:4846): protocol: context deadline exceeded; context deadline exceeded Mar 28 14:46:05.030604513 (430h35m8.04627423s) [noise=true , address=79.127.219.38:4818 , piece_id=YKIWBWXFSEFPB63JTXQC4BA5YIIOCZWIOKQ4GRIBXZJ3KFCZRQKA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:12ZvUdXiXPFGqmVzYffAp735GjzaSMcyUfTD61Yi6us9ETg7Ath, address:79.127.219.38:4818): protocol: read tcp 139.59.144.22:58514->79.127.219.38:4818: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 28 14:46:05.030645703 (430h35m8.046315306s) [seg_pos={0 0}, upload=413459]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 28 14:46:05.141824246 (430h35m8.157493849s) [upload=413459]: Done uploading segments... Mar 28 14:46:05.141873159 (430h35m8.157542765s) [upload=413459]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.424403515s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 2.459391464s trace: https://jaeger.dev.storj.io/trace/409ed236b50eb53 FAIL (2026-03-28 15:23:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 28 14:45:35.315648730 (430h34m38.331318331s) [upload=413459]: Starting upload Mar 28 14:45:35.315733599 (430h34m38.331403199s) [upload=413459]: Done starting upload Mar 28 14:45:35.315794102 (430h34m38.331463702s) [upload=413459]: Uploading segments... Mar 28 14:45:35.316032212 (430h34m38.331701813s) [upload=413459]: Got next segment. Inline: false Mar 28 14:45:35.316200570 (430h34m38.331870171s) [upload=413459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 28 14:45:35.316206390 (430h34m38.33187599s) [upload=413459]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 28 14:45:35.430724217 (430h34m38.446393826s) [seg_pos={0 0}, upload=413459]: Begin upload segment... Mar 28 14:45:35.430795561 (430h34m38.446465164s) [seg_pos={0 0}, upload=413459]: Done begin upload segment. Mar 28 14:45:35.430836859 (430h34m38.446506464s) [noise=true , address=77.90.31.19:4846 , piece_id=MOMLIPX2V2BHWIU66SML35WWK6LRRDUWRUJZ6UT2HYKOA5ZWQ5IQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.430848244 (430h34m38.446517846s) [noise=true , address=145.79.227.14:4843 , piece_id=QXIASECCPGVCCC36Y6NW3NTLJMKVPNKGXWNHTCCHME2HOOW72UIQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.431376292 (430h34m38.447045883s) [noise=true , address=79.127.219.41:4925 , piece_id=UUW3LL2MKMROAJ62NW4A66AZR337DUXIUQACIGUK2XFGV6TOC67Q, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.431872685 (430h34m38.447542294s) [noise=true , address=145.79.227.40:4051 , piece_id=6SFERDADQONW3YTOVRYHDZ7VXAOLTNZ5I3QQAS2XHG62KCNRP7TQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.432614239 (430h34m38.448283849s) [noise=true , address=145.79.227.12:4768 , piece_id=Q4ISGVZD47GU3RPO55DR4JOKIJ4SQ6UWMQPKHJAKKPOFAUJ2PZ4Q, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.432736747 (430h34m38.448406359s) [noise=true , address=145.79.227.49:4341 , piece_id=DOBHYVALR2MNRJX2ABYYBHUASIGTPTBJ5POZAZYIGSJX6K5EJLTQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.433092040 (430h34m38.448761652s) [noise=true , address=77.90.31.15:4702 , piece_id=56EUQLUUJUASC4QNLK4WJIE53EG5MF2DPD3Z7IBF3Q5J3ERJE3BQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.433377489 (430h34m38.449047102s) [noise=true , address=145.79.227.16:4911 , piece_id=JYKGUHIZQ4U74SM5BCQGXFA37I42PYXKOWB5CQQRFDX3I2EV527Q, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.434098354 (430h34m38.449767944s) [noise=true , address=79.127.219.38:4818 , piece_id=YKIWBWXFSEFPB63JTXQC4BA5YIIOCZWIOKQ4GRIBXZJ3KFCZRQKA, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.434769512 (430h34m38.450439123s) [noise=true , address=145.79.227.41:4086 , piece_id=6HKTS55E7ZO6NCX45RPJD5RF5K6DRLVRQTIZMV7ATXFVR4SACDRA, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.435075797 (430h34m38.450745407s) [noise=true , address=145.79.227.25:4617 , piece_id=WSDGMWM7CASNFGW2RPAH7MDX6O7VXEXZYLSN45ZYB2S46LPHDGOA, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.435582204 (430h34m38.451251817s) [noise=true , address=145.79.227.9:4366 , piece_id=KHTUQ2PXIXRD5Y5XIB2L3X6TMQWGZCBIS3NPOY2RS3275SCAI5BA, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.436022166 (430h34m38.451691756s) [noise=true , address=45.91.116.4:4953 , piece_id=E62ZS235DKLPVBQY6FEI3DDF3DU66LDI23ISVXANBB4M5WHQEO5Q, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.436454955 (430h34m38.452124562s) [noise=true , address=145.79.227.2:4115 , piece_id=TA4GUBLNRHKJ7DSCENFIQ5L5M7NLSJFDT4S4PVN3RRBEIIPEUYNQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.436879556 (430h34m38.452549169s) [noise=true , address=145.79.227.17:4973 , piece_id=ZZMYZGBYYVWXSOHDJGVKLNG6R3NMS2AZP4XMNCDNGJDE2DYADWIQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.437343138 (430h34m38.453012746s) [noise=true , address=145.79.227.21:4478 , piece_id=Z75OK7DR2TUDRPTM2RMX5FCKF456FYTMEVJVDBUZESZYFE6QQUTQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:36.049495517 (430h34m39.065165118s) [upload=413459]: Next returned nil segment Mar 28 14:45:36.049527631 (430h34m39.065197233s) [upload=413459]: Waiting for error group managing segments... Mar 28 14:45:36.182328788 (430h34m39.197998388s) [noise=true , address=145.79.227.17:4973 , piece_id=ZZMYZGBYYVWXSOHDJGVKLNG6R3NMS2AZP4XMNCDNGJDE2DYADWIQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.182575788 (430h34m39.198245412s) [noise=true , address=145.79.227.25:4617 , piece_id=WSDGMWM7CASNFGW2RPAH7MDX6O7VXEXZYLSN45ZYB2S46LPHDGOA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.182982538 (430h34m39.198652144s) [noise=true , address=145.79.227.9:4366 , piece_id=KHTUQ2PXIXRD5Y5XIB2L3X6TMQWGZCBIS3NPOY2RS3275SCAI5BA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.183438894 (430h34m39.199108504s) [noise=true , address=45.91.116.4:4953 , piece_id=E62ZS235DKLPVBQY6FEI3DDF3DU66LDI23ISVXANBB4M5WHQEO5Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.184168119 (430h34m39.199837707s) [noise=true , address=145.79.227.12:4768 , piece_id=Q4ISGVZD47GU3RPO55DR4JOKIJ4SQ6UWMQPKHJAKKPOFAUJ2PZ4Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.184865710 (430h34m39.200535314s) [noise=true , address=145.79.227.40:4051 , piece_id=6SFERDADQONW3YTOVRYHDZ7VXAOLTNZ5I3QQAS2XHG62KCNRP7TQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.185089145 (430h34m39.20075875s) [noise=true , address=145.79.227.16:4911 , piece_id=JYKGUHIZQ4U74SM5BCQGXFA37I42PYXKOWB5CQQRFDX3I2EV527Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.188553089 (430h34m39.204222694s) [noise=true , address=145.79.227.14:4843 , piece_id=QXIASECCPGVCCC36Y6NW3NTLJMKVPNKGXWNHTCCHME2HOOW72UIQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.190225530 (430h34m39.205895143s) [noise=true , address=145.79.227.2:4115 , piece_id=TA4GUBLNRHKJ7DSCENFIQ5L5M7NLSJFDT4S4PVN3RRBEIIPEUYNQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.190551513 (430h34m39.206221103s) [noise=true , address=145.79.227.21:4478 , piece_id=Z75OK7DR2TUDRPTM2RMX5FCKF456FYTMEVJVDBUZESZYFE6QQUTQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.191113981 (430h34m39.206783572s) [noise=true , address=145.79.227.41:4086 , piece_id=6HKTS55E7ZO6NCX45RPJD5RF5K6DRLVRQTIZMV7ATXFVR4SACDRA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.289167312 (430h34m39.304836941s) [noise=true , address=145.79.227.49:4341 , piece_id=DOBHYVALR2MNRJX2ABYYBHUASIGTPTBJ5POZAZYIGSJX6K5EJLTQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:46:05.029585012 (430h35m8.045254629s) [noise=true , address=77.90.31.15:4702 , piece_id=56EUQLUUJUASC4QNLK4WJIE53EG5MF2DPD3Z7IBF3Q5J3ERJE3BQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:1ucSAZVBqRRJU7WZDKs7Cs6yKV73haX7px76sfukA6b8Q8DDsW, address:77.90.31.15:4702): protocol: write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection; write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection Mar 28 14:46:05.030074673 (430h35m8.045744285s) [noise=true , address=79.127.219.41:4925 , piece_id=UUW3LL2MKMROAJ62NW4A66AZR337DUXIUQACIGUK2XFGV6TOC67Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:1kBG5b1fGn5dLzZoD8v4KG6CHStmw1QhGpqFdtGsMsDUEXwZzs, address:79.127.219.41:4925): protocol: read tcp 139.59.144.22:33812->79.127.219.41:4925: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 28 14:46:05.030297717 (430h35m8.04596733s) [noise=true , address=77.90.31.19:4846 , piece_id=MOMLIPX2V2BHWIU66SML35WWK6LRRDUWRUJZ6UT2HYKOA5ZWQ5IQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:12V76dqXhBHet772yUXG9oRU9r1m7rtXQeARRVzTeGrcJxxzyvM, address:77.90.31.19:4846): protocol: context deadline exceeded; context deadline exceeded Mar 28 14:46:05.030604513 (430h35m8.04627423s) [noise=true , address=79.127.219.38:4818 , piece_id=YKIWBWXFSEFPB63JTXQC4BA5YIIOCZWIOKQ4GRIBXZJ3KFCZRQKA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:12ZvUdXiXPFGqmVzYffAp735GjzaSMcyUfTD61Yi6us9ETg7Ath, address:79.127.219.38:4818): protocol: read tcp 139.59.144.22:58514->79.127.219.38:4818: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 28 14:46:05.030645703 (430h35m8.046315306s) [seg_pos={0 0}, upload=413459]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 28 14:46:05.141824246 (430h35m8.157493849s) [upload=413459]: Done uploading segments... Mar 28 14:46:05.141873159 (430h35m8.157542765s) [upload=413459]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.45942836s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 2.444681606s trace: https://jaeger.dev.storj.io/trace/409ed236b50eb53 FAIL (2026-03-28 15:23:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 28 14:45:35.315648730 (430h34m38.331318331s) [upload=413459]: Starting upload Mar 28 14:45:35.315733599 (430h34m38.331403199s) [upload=413459]: Done starting upload Mar 28 14:45:35.315794102 (430h34m38.331463702s) [upload=413459]: Uploading segments... Mar 28 14:45:35.316032212 (430h34m38.331701813s) [upload=413459]: Got next segment. Inline: false Mar 28 14:45:35.316200570 (430h34m38.331870171s) [upload=413459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 28 14:45:35.316206390 (430h34m38.33187599s) [upload=413459]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 28 14:45:35.430724217 (430h34m38.446393826s) [seg_pos={0 0}, upload=413459]: Begin upload segment... Mar 28 14:45:35.430795561 (430h34m38.446465164s) [seg_pos={0 0}, upload=413459]: Done begin upload segment. Mar 28 14:45:35.430836859 (430h34m38.446506464s) [noise=true , address=77.90.31.19:4846 , piece_id=MOMLIPX2V2BHWIU66SML35WWK6LRRDUWRUJZ6UT2HYKOA5ZWQ5IQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.430848244 (430h34m38.446517846s) [noise=true , address=145.79.227.14:4843 , piece_id=QXIASECCPGVCCC36Y6NW3NTLJMKVPNKGXWNHTCCHME2HOOW72UIQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.431376292 (430h34m38.447045883s) [noise=true , address=79.127.219.41:4925 , piece_id=UUW3LL2MKMROAJ62NW4A66AZR337DUXIUQACIGUK2XFGV6TOC67Q, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.431872685 (430h34m38.447542294s) [noise=true , address=145.79.227.40:4051 , piece_id=6SFERDADQONW3YTOVRYHDZ7VXAOLTNZ5I3QQAS2XHG62KCNRP7TQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.432614239 (430h34m38.448283849s) [noise=true , address=145.79.227.12:4768 , piece_id=Q4ISGVZD47GU3RPO55DR4JOKIJ4SQ6UWMQPKHJAKKPOFAUJ2PZ4Q, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.432736747 (430h34m38.448406359s) [noise=true , address=145.79.227.49:4341 , piece_id=DOBHYVALR2MNRJX2ABYYBHUASIGTPTBJ5POZAZYIGSJX6K5EJLTQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.433092040 (430h34m38.448761652s) [noise=true , address=77.90.31.15:4702 , piece_id=56EUQLUUJUASC4QNLK4WJIE53EG5MF2DPD3Z7IBF3Q5J3ERJE3BQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.433377489 (430h34m38.449047102s) [noise=true , address=145.79.227.16:4911 , piece_id=JYKGUHIZQ4U74SM5BCQGXFA37I42PYXKOWB5CQQRFDX3I2EV527Q, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.434098354 (430h34m38.449767944s) [noise=true , address=79.127.219.38:4818 , piece_id=YKIWBWXFSEFPB63JTXQC4BA5YIIOCZWIOKQ4GRIBXZJ3KFCZRQKA, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.434769512 (430h34m38.450439123s) [noise=true , address=145.79.227.41:4086 , piece_id=6HKTS55E7ZO6NCX45RPJD5RF5K6DRLVRQTIZMV7ATXFVR4SACDRA, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.435075797 (430h34m38.450745407s) [noise=true , address=145.79.227.25:4617 , piece_id=WSDGMWM7CASNFGW2RPAH7MDX6O7VXEXZYLSN45ZYB2S46LPHDGOA, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.435582204 (430h34m38.451251817s) [noise=true , address=145.79.227.9:4366 , piece_id=KHTUQ2PXIXRD5Y5XIB2L3X6TMQWGZCBIS3NPOY2RS3275SCAI5BA, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.436022166 (430h34m38.451691756s) [noise=true , address=45.91.116.4:4953 , piece_id=E62ZS235DKLPVBQY6FEI3DDF3DU66LDI23ISVXANBB4M5WHQEO5Q, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.436454955 (430h34m38.452124562s) [noise=true , address=145.79.227.2:4115 , piece_id=TA4GUBLNRHKJ7DSCENFIQ5L5M7NLSJFDT4S4PVN3RRBEIIPEUYNQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.436879556 (430h34m38.452549169s) [noise=true , address=145.79.227.17:4973 , piece_id=ZZMYZGBYYVWXSOHDJGVKLNG6R3NMS2AZP4XMNCDNGJDE2DYADWIQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.437343138 (430h34m38.453012746s) [noise=true , address=145.79.227.21:4478 , piece_id=Z75OK7DR2TUDRPTM2RMX5FCKF456FYTMEVJVDBUZESZYFE6QQUTQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:36.049495517 (430h34m39.065165118s) [upload=413459]: Next returned nil segment Mar 28 14:45:36.049527631 (430h34m39.065197233s) [upload=413459]: Waiting for error group managing segments... Mar 28 14:45:36.182328788 (430h34m39.197998388s) [noise=true , address=145.79.227.17:4973 , piece_id=ZZMYZGBYYVWXSOHDJGVKLNG6R3NMS2AZP4XMNCDNGJDE2DYADWIQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.182575788 (430h34m39.198245412s) [noise=true , address=145.79.227.25:4617 , piece_id=WSDGMWM7CASNFGW2RPAH7MDX6O7VXEXZYLSN45ZYB2S46LPHDGOA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.182982538 (430h34m39.198652144s) [noise=true , address=145.79.227.9:4366 , piece_id=KHTUQ2PXIXRD5Y5XIB2L3X6TMQWGZCBIS3NPOY2RS3275SCAI5BA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.183438894 (430h34m39.199108504s) [noise=true , address=45.91.116.4:4953 , piece_id=E62ZS235DKLPVBQY6FEI3DDF3DU66LDI23ISVXANBB4M5WHQEO5Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.184168119 (430h34m39.199837707s) [noise=true , address=145.79.227.12:4768 , piece_id=Q4ISGVZD47GU3RPO55DR4JOKIJ4SQ6UWMQPKHJAKKPOFAUJ2PZ4Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.184865710 (430h34m39.200535314s) [noise=true , address=145.79.227.40:4051 , piece_id=6SFERDADQONW3YTOVRYHDZ7VXAOLTNZ5I3QQAS2XHG62KCNRP7TQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.185089145 (430h34m39.20075875s) [noise=true , address=145.79.227.16:4911 , piece_id=JYKGUHIZQ4U74SM5BCQGXFA37I42PYXKOWB5CQQRFDX3I2EV527Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.188553089 (430h34m39.204222694s) [noise=true , address=145.79.227.14:4843 , piece_id=QXIASECCPGVCCC36Y6NW3NTLJMKVPNKGXWNHTCCHME2HOOW72UIQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.190225530 (430h34m39.205895143s) [noise=true , address=145.79.227.2:4115 , piece_id=TA4GUBLNRHKJ7DSCENFIQ5L5M7NLSJFDT4S4PVN3RRBEIIPEUYNQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.190551513 (430h34m39.206221103s) [noise=true , address=145.79.227.21:4478 , piece_id=Z75OK7DR2TUDRPTM2RMX5FCKF456FYTMEVJVDBUZESZYFE6QQUTQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.191113981 (430h34m39.206783572s) [noise=true , address=145.79.227.41:4086 , piece_id=6HKTS55E7ZO6NCX45RPJD5RF5K6DRLVRQTIZMV7ATXFVR4SACDRA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.289167312 (430h34m39.304836941s) [noise=true , address=145.79.227.49:4341 , piece_id=DOBHYVALR2MNRJX2ABYYBHUASIGTPTBJ5POZAZYIGSJX6K5EJLTQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:46:05.029585012 (430h35m8.045254629s) [noise=true , address=77.90.31.15:4702 , piece_id=56EUQLUUJUASC4QNLK4WJIE53EG5MF2DPD3Z7IBF3Q5J3ERJE3BQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:1ucSAZVBqRRJU7WZDKs7Cs6yKV73haX7px76sfukA6b8Q8DDsW, address:77.90.31.15:4702): protocol: write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection; write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection Mar 28 14:46:05.030074673 (430h35m8.045744285s) [noise=true , address=79.127.219.41:4925 , piece_id=UUW3LL2MKMROAJ62NW4A66AZR337DUXIUQACIGUK2XFGV6TOC67Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:1kBG5b1fGn5dLzZoD8v4KG6CHStmw1QhGpqFdtGsMsDUEXwZzs, address:79.127.219.41:4925): protocol: read tcp 139.59.144.22:33812->79.127.219.41:4925: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 28 14:46:05.030297717 (430h35m8.04596733s) [noise=true , address=77.90.31.19:4846 , piece_id=MOMLIPX2V2BHWIU66SML35WWK6LRRDUWRUJZ6UT2HYKOA5ZWQ5IQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:12V76dqXhBHet772yUXG9oRU9r1m7rtXQeARRVzTeGrcJxxzyvM, address:77.90.31.19:4846): protocol: context deadline exceeded; context deadline exceeded Mar 28 14:46:05.030604513 (430h35m8.04627423s) [noise=true , address=79.127.219.38:4818 , piece_id=YKIWBWXFSEFPB63JTXQC4BA5YIIOCZWIOKQ4GRIBXZJ3KFCZRQKA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:12ZvUdXiXPFGqmVzYffAp735GjzaSMcyUfTD61Yi6us9ETg7Ath, address:79.127.219.38:4818): protocol: read tcp 139.59.144.22:58514->79.127.219.38:4818: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 28 14:46:05.030645703 (430h35m8.046315306s) [seg_pos={0 0}, upload=413459]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 28 14:46:05.141824246 (430h35m8.157493849s) [upload=413459]: Done uploading segments... Mar 28 14:46:05.141873159 (430h35m8.157542765s) [upload=413459]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.444715078s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 2.552873673s trace: https://jaeger.dev.storj.io/trace/409ed236b50eb53 FAIL (2026-03-28 15:22:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Mar 28 14:45:35.315648730 (430h34m38.331318331s) [upload=413459]: Starting upload Mar 28 14:45:35.315733599 (430h34m38.331403199s) [upload=413459]: Done starting upload Mar 28 14:45:35.315794102 (430h34m38.331463702s) [upload=413459]: Uploading segments... Mar 28 14:45:35.316032212 (430h34m38.331701813s) [upload=413459]: Got next segment. Inline: false Mar 28 14:45:35.316200570 (430h34m38.331870171s) [upload=413459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 28 14:45:35.316206390 (430h34m38.33187599s) [upload=413459]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 28 14:45:35.430724217 (430h34m38.446393826s) [seg_pos={0 0}, upload=413459]: Begin upload segment... Mar 28 14:45:35.430795561 (430h34m38.446465164s) [seg_pos={0 0}, upload=413459]: Done begin upload segment. Mar 28 14:45:35.430836859 (430h34m38.446506464s) [noise=true , address=77.90.31.19:4846 , piece_id=MOMLIPX2V2BHWIU66SML35WWK6LRRDUWRUJZ6UT2HYKOA5ZWQ5IQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.430848244 (430h34m38.446517846s) [noise=true , address=145.79.227.14:4843 , piece_id=QXIASECCPGVCCC36Y6NW3NTLJMKVPNKGXWNHTCCHME2HOOW72UIQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.431376292 (430h34m38.447045883s) [noise=true , address=79.127.219.41:4925 , piece_id=UUW3LL2MKMROAJ62NW4A66AZR337DUXIUQACIGUK2XFGV6TOC67Q, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.431872685 (430h34m38.447542294s) [noise=true , address=145.79.227.40:4051 , piece_id=6SFERDADQONW3YTOVRYHDZ7VXAOLTNZ5I3QQAS2XHG62KCNRP7TQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.432614239 (430h34m38.448283849s) [noise=true , address=145.79.227.12:4768 , piece_id=Q4ISGVZD47GU3RPO55DR4JOKIJ4SQ6UWMQPKHJAKKPOFAUJ2PZ4Q, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.432736747 (430h34m38.448406359s) [noise=true , address=145.79.227.49:4341 , piece_id=DOBHYVALR2MNRJX2ABYYBHUASIGTPTBJ5POZAZYIGSJX6K5EJLTQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.433092040 (430h34m38.448761652s) [noise=true , address=77.90.31.15:4702 , piece_id=56EUQLUUJUASC4QNLK4WJIE53EG5MF2DPD3Z7IBF3Q5J3ERJE3BQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.433377489 (430h34m38.449047102s) [noise=true , address=145.79.227.16:4911 , piece_id=JYKGUHIZQ4U74SM5BCQGXFA37I42PYXKOWB5CQQRFDX3I2EV527Q, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.434098354 (430h34m38.449767944s) [noise=true , address=79.127.219.38:4818 , piece_id=YKIWBWXFSEFPB63JTXQC4BA5YIIOCZWIOKQ4GRIBXZJ3KFCZRQKA, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.434769512 (430h34m38.450439123s) [noise=true , address=145.79.227.41:4086 , piece_id=6HKTS55E7ZO6NCX45RPJD5RF5K6DRLVRQTIZMV7ATXFVR4SACDRA, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.435075797 (430h34m38.450745407s) [noise=true , address=145.79.227.25:4617 , piece_id=WSDGMWM7CASNFGW2RPAH7MDX6O7VXEXZYLSN45ZYB2S46LPHDGOA, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.435582204 (430h34m38.451251817s) [noise=true , address=145.79.227.9:4366 , piece_id=KHTUQ2PXIXRD5Y5XIB2L3X6TMQWGZCBIS3NPOY2RS3275SCAI5BA, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.436022166 (430h34m38.451691756s) [noise=true , address=45.91.116.4:4953 , piece_id=E62ZS235DKLPVBQY6FEI3DDF3DU66LDI23ISVXANBB4M5WHQEO5Q, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.436454955 (430h34m38.452124562s) [noise=true , address=145.79.227.2:4115 , piece_id=TA4GUBLNRHKJ7DSCENFIQ5L5M7NLSJFDT4S4PVN3RRBEIIPEUYNQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.436879556 (430h34m38.452549169s) [noise=true , address=145.79.227.17:4973 , piece_id=ZZMYZGBYYVWXSOHDJGVKLNG6R3NMS2AZP4XMNCDNGJDE2DYADWIQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:35.437343138 (430h34m38.453012746s) [noise=true , address=145.79.227.21:4478 , piece_id=Z75OK7DR2TUDRPTM2RMX5FCKF456FYTMEVJVDBUZESZYFE6QQUTQ, seg_pos={0 0}, upload=413459]: Uploading piece... Mar 28 14:45:36.049495517 (430h34m39.065165118s) [upload=413459]: Next returned nil segment Mar 28 14:45:36.049527631 (430h34m39.065197233s) [upload=413459]: Waiting for error group managing segments... Mar 28 14:45:36.182328788 (430h34m39.197998388s) [noise=true , address=145.79.227.17:4973 , piece_id=ZZMYZGBYYVWXSOHDJGVKLNG6R3NMS2AZP4XMNCDNGJDE2DYADWIQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.182575788 (430h34m39.198245412s) [noise=true , address=145.79.227.25:4617 , piece_id=WSDGMWM7CASNFGW2RPAH7MDX6O7VXEXZYLSN45ZYB2S46LPHDGOA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.182982538 (430h34m39.198652144s) [noise=true , address=145.79.227.9:4366 , piece_id=KHTUQ2PXIXRD5Y5XIB2L3X6TMQWGZCBIS3NPOY2RS3275SCAI5BA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.183438894 (430h34m39.199108504s) [noise=true , address=45.91.116.4:4953 , piece_id=E62ZS235DKLPVBQY6FEI3DDF3DU66LDI23ISVXANBB4M5WHQEO5Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.184168119 (430h34m39.199837707s) [noise=true , address=145.79.227.12:4768 , piece_id=Q4ISGVZD47GU3RPO55DR4JOKIJ4SQ6UWMQPKHJAKKPOFAUJ2PZ4Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.184865710 (430h34m39.200535314s) [noise=true , address=145.79.227.40:4051 , piece_id=6SFERDADQONW3YTOVRYHDZ7VXAOLTNZ5I3QQAS2XHG62KCNRP7TQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.185089145 (430h34m39.20075875s) [noise=true , address=145.79.227.16:4911 , piece_id=JYKGUHIZQ4U74SM5BCQGXFA37I42PYXKOWB5CQQRFDX3I2EV527Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.188553089 (430h34m39.204222694s) [noise=true , address=145.79.227.14:4843 , piece_id=QXIASECCPGVCCC36Y6NW3NTLJMKVPNKGXWNHTCCHME2HOOW72UIQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.190225530 (430h34m39.205895143s) [noise=true , address=145.79.227.2:4115 , piece_id=TA4GUBLNRHKJ7DSCENFIQ5L5M7NLSJFDT4S4PVN3RRBEIIPEUYNQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.190551513 (430h34m39.206221103s) [noise=true , address=145.79.227.21:4478 , piece_id=Z75OK7DR2TUDRPTM2RMX5FCKF456FYTMEVJVDBUZESZYFE6QQUTQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.191113981 (430h34m39.206783572s) [noise=true , address=145.79.227.41:4086 , piece_id=6HKTS55E7ZO6NCX45RPJD5RF5K6DRLVRQTIZMV7ATXFVR4SACDRA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:45:36.289167312 (430h34m39.304836941s) [noise=true , address=145.79.227.49:4341 , piece_id=DOBHYVALR2MNRJX2ABYYBHUASIGTPTBJ5POZAZYIGSJX6K5EJLTQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: <nil> Mar 28 14:46:05.029585012 (430h35m8.045254629s) [noise=true , address=77.90.31.15:4702 , piece_id=56EUQLUUJUASC4QNLK4WJIE53EG5MF2DPD3Z7IBF3Q5J3ERJE3BQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:1ucSAZVBqRRJU7WZDKs7Cs6yKV73haX7px76sfukA6b8Q8DDsW, address:77.90.31.15:4702): protocol: write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection; write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50726->77.90.31.15:4702: use of closed network connection Mar 28 14:46:05.030074673 (430h35m8.045744285s) [noise=true , address=79.127.219.41:4925 , piece_id=UUW3LL2MKMROAJ62NW4A66AZR337DUXIUQACIGUK2XFGV6TOC67Q, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:1kBG5b1fGn5dLzZoD8v4KG6CHStmw1QhGpqFdtGsMsDUEXwZzs, address:79.127.219.41:4925): protocol: read tcp 139.59.144.22:33812->79.127.219.41:4925: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 28 14:46:05.030297717 (430h35m8.04596733s) [noise=true , address=77.90.31.19:4846 , piece_id=MOMLIPX2V2BHWIU66SML35WWK6LRRDUWRUJZ6UT2HYKOA5ZWQ5IQ, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:12V76dqXhBHet772yUXG9oRU9r1m7rtXQeARRVzTeGrcJxxzyvM, address:77.90.31.19:4846): protocol: context deadline exceeded; context deadline exceeded Mar 28 14:46:05.030604513 (430h35m8.04627423s) [noise=true , address=79.127.219.38:4818 , piece_id=YKIWBWXFSEFPB63JTXQC4BA5YIIOCZWIOKQ4GRIBXZJ3KFCZRQKA, seg_pos={0 0}, upload=413459]: Done uploading piece. err: ecclient: upload failed (node:12ZvUdXiXPFGqmVzYffAp735GjzaSMcyUfTD61Yi6us9ETg7Ath, address:79.127.219.38:4818): protocol: read tcp 139.59.144.22:58514->79.127.219.38:4818: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Mar 28 14:46:05.030645703 (430h35m8.046315306s) [seg_pos={0 0}, upload=413459]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Mar 28 14:46:05.141824246 (430h35m8.157493849s) [upload=413459]: Done uploading segments... Mar 28 14:46:05.141873159 (430h35m8.157542765s) [upload=413459]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.552912619s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)