Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
2.330092028s trace:
https://jaeger.dev.storj.io/trace/2fe3acd100d740fd
FAIL (2026-02-09 00:32:42 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Feb 9 00:14:22.628823254 (584h19m0.512777443s) [upload=514373]: Starting upload Feb 9 00:14:22.628881807 (584h19m0.512836015s) [upload=514373]: Done starting upload Feb 9 00:14:22.629148558 (584h19m0.513102751s) [upload=514373]: Uploading segments... Feb 9 00:14:22.629188448 (584h19m0.51314265s) [upload=514373]: Got next segment. Inline: false Feb 9 00:14:22.629227334 (584h19m0.513181534s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Feb 9 00:14:22.629232569 (584h19m0.513186769s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Feb 9 00:14:52.340184039 (584h19m30.224138241s) [upload=514373]: Next segment err: context deadline exceeded Feb 9 00:14:52.340437904 (584h19m30.224392094s) [upload=514373]: Done uploading segments... Feb 9 00:14:52.340476462 (584h19m30.224430647s) [upload=514373]: Upload finished. err: context deadline exceeded
Duration: 2.330133802s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
2.402812905s trace:
https://jaeger.dev.storj.io/trace/2fe3acd100d740fd
FAIL (2026-02-09 00:32:23 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Feb 9 00:14:22.628823254 (584h19m0.512777443s) [upload=514373]: Starting upload Feb 9 00:14:22.628881807 (584h19m0.512836015s) [upload=514373]: Done starting upload Feb 9 00:14:22.629148558 (584h19m0.513102751s) [upload=514373]: Uploading segments... Feb 9 00:14:22.629188448 (584h19m0.51314265s) [upload=514373]: Got next segment. Inline: false Feb 9 00:14:22.629227334 (584h19m0.513181534s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Feb 9 00:14:22.629232569 (584h19m0.513186769s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Feb 9 00:14:52.340184039 (584h19m30.224138241s) [upload=514373]: Next segment err: context deadline exceeded Feb 9 00:14:52.340437904 (584h19m30.224392094s) [upload=514373]: Done uploading segments... Feb 9 00:14:52.340476462 (584h19m30.224430647s) [upload=514373]: Upload finished. err: context deadline exceeded
Duration: 2.402925065s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
2.522175391s trace:
https://jaeger.dev.storj.io/trace/2fe3acd100d740fd
FAIL (2026-02-09 00:31:46 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Feb 9 00:14:22.628823254 (584h19m0.512777443s) [upload=514373]: Starting upload Feb 9 00:14:22.628881807 (584h19m0.512836015s) [upload=514373]: Done starting upload Feb 9 00:14:22.629148558 (584h19m0.513102751s) [upload=514373]: Uploading segments... Feb 9 00:14:22.629188448 (584h19m0.51314265s) [upload=514373]: Got next segment. Inline: false Feb 9 00:14:22.629227334 (584h19m0.513181534s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Feb 9 00:14:22.629232569 (584h19m0.513186769s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Feb 9 00:14:52.340184039 (584h19m30.224138241s) [upload=514373]: Next segment err: context deadline exceeded Feb 9 00:14:52.340437904 (584h19m30.224392094s) [upload=514373]: Done uploading segments... Feb 9 00:14:52.340476462 (584h19m30.224430647s) [upload=514373]: Upload finished. err: context deadline exceeded
Duration: 2.52220302s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
2.372232263s trace:
https://jaeger.dev.storj.io/trace/2fe3acd100d740fd
FAIL (2026-02-09 00:31:12 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Feb 9 00:14:22.628823254 (584h19m0.512777443s) [upload=514373]: Starting upload Feb 9 00:14:22.628881807 (584h19m0.512836015s) [upload=514373]: Done starting upload Feb 9 00:14:22.629148558 (584h19m0.513102751s) [upload=514373]: Uploading segments... Feb 9 00:14:22.629188448 (584h19m0.51314265s) [upload=514373]: Got next segment. Inline: false Feb 9 00:14:22.629227334 (584h19m0.513181534s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Feb 9 00:14:22.629232569 (584h19m0.513186769s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Feb 9 00:14:52.340184039 (584h19m30.224138241s) [upload=514373]: Next segment err: context deadline exceeded Feb 9 00:14:52.340437904 (584h19m30.224392094s) [upload=514373]: Done uploading segments... Feb 9 00:14:52.340476462 (584h19m30.224430647s) [upload=514373]: Upload finished. err: context deadline exceeded
Duration: 2.372254311s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
2.343885214s trace:
https://jaeger.dev.storj.io/trace/2fe3acd100d740fd
FAIL (2026-02-09 00:30:46 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Feb 9 00:14:22.628823254 (584h19m0.512777443s) [upload=514373]: Starting upload Feb 9 00:14:22.628881807 (584h19m0.512836015s) [upload=514373]: Done starting upload Feb 9 00:14:22.629148558 (584h19m0.513102751s) [upload=514373]: Uploading segments... Feb 9 00:14:22.629188448 (584h19m0.51314265s) [upload=514373]: Got next segment. Inline: false Feb 9 00:14:22.629227334 (584h19m0.513181534s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Feb 9 00:14:22.629232569 (584h19m0.513186769s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Feb 9 00:14:52.340184039 (584h19m30.224138241s) [upload=514373]: Next segment err: context deadline exceeded Feb 9 00:14:52.340437904 (584h19m30.224392094s) [upload=514373]: Done uploading segments... Feb 9 00:14:52.340476462 (584h19m30.224430647s) [upload=514373]: Upload finished. err: context deadline exceeded
Duration: 2.34390782s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
2.287758348s trace:
https://jaeger.dev.storj.io/trace/2fe3acd100d740fd
FAIL (2026-02-09 00:29:38 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Feb 9 00:14:22.628823254 (584h19m0.512777443s) [upload=514373]: Starting upload Feb 9 00:14:22.628881807 (584h19m0.512836015s) [upload=514373]: Done starting upload Feb 9 00:14:22.629148558 (584h19m0.513102751s) [upload=514373]: Uploading segments... Feb 9 00:14:22.629188448 (584h19m0.51314265s) [upload=514373]: Got next segment. Inline: false Feb 9 00:14:22.629227334 (584h19m0.513181534s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Feb 9 00:14:22.629232569 (584h19m0.513186769s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Feb 9 00:14:52.340184039 (584h19m30.224138241s) [upload=514373]: Next segment err: context deadline exceeded Feb 9 00:14:52.340437904 (584h19m30.224392094s) [upload=514373]: Done uploading segments... Feb 9 00:14:52.340476462 (584h19m30.224430647s) [upload=514373]: Upload finished. err: context deadline exceeded
Duration: 2.287793386s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
2.359235963s trace:
https://jaeger.dev.storj.io/trace/2fe3acd100d740fd
FAIL (2026-02-09 00:29:03 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Feb 9 00:14:22.628823254 (584h19m0.512777443s) [upload=514373]: Starting upload Feb 9 00:14:22.628881807 (584h19m0.512836015s) [upload=514373]: Done starting upload Feb 9 00:14:22.629148558 (584h19m0.513102751s) [upload=514373]: Uploading segments... Feb 9 00:14:22.629188448 (584h19m0.51314265s) [upload=514373]: Got next segment. Inline: false Feb 9 00:14:22.629227334 (584h19m0.513181534s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Feb 9 00:14:22.629232569 (584h19m0.513186769s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Feb 9 00:14:52.340184039 (584h19m30.224138241s) [upload=514373]: Next segment err: context deadline exceeded Feb 9 00:14:52.340437904 (584h19m30.224392094s) [upload=514373]: Done uploading segments... Feb 9 00:14:52.340476462 (584h19m30.224430647s) [upload=514373]: Upload finished. err: context deadline exceeded
Duration: 2.359268417s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
2.373507759s trace:
https://jaeger.dev.storj.io/trace/2fe3acd100d740fd
FAIL (2026-02-09 00:27:58 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Feb 9 00:14:22.628823254 (584h19m0.512777443s) [upload=514373]: Starting upload Feb 9 00:14:22.628881807 (584h19m0.512836015s) [upload=514373]: Done starting upload Feb 9 00:14:22.629148558 (584h19m0.513102751s) [upload=514373]: Uploading segments... Feb 9 00:14:22.629188448 (584h19m0.51314265s) [upload=514373]: Got next segment. Inline: false Feb 9 00:14:22.629227334 (584h19m0.513181534s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Feb 9 00:14:22.629232569 (584h19m0.513186769s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Feb 9 00:14:52.340184039 (584h19m30.224138241s) [upload=514373]: Next segment err: context deadline exceeded Feb 9 00:14:52.340437904 (584h19m30.224392094s) [upload=514373]: Done uploading segments... Feb 9 00:14:52.340476462 (584h19m30.224430647s) [upload=514373]: Upload finished. err: context deadline exceeded
Duration: 2.373535108s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
2.428158168s trace:
https://jaeger.dev.storj.io/trace/2fe3acd100d740fd
FAIL (2026-02-09 00:27:22 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Feb 9 00:14:22.628823254 (584h19m0.512777443s) [upload=514373]: Starting upload Feb 9 00:14:22.628881807 (584h19m0.512836015s) [upload=514373]: Done starting upload Feb 9 00:14:22.629148558 (584h19m0.513102751s) [upload=514373]: Uploading segments... Feb 9 00:14:22.629188448 (584h19m0.51314265s) [upload=514373]: Got next segment. Inline: false Feb 9 00:14:22.629227334 (584h19m0.513181534s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Feb 9 00:14:22.629232569 (584h19m0.513186769s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Feb 9 00:14:52.340184039 (584h19m30.224138241s) [upload=514373]: Next segment err: context deadline exceeded Feb 9 00:14:52.340437904 (584h19m30.224392094s) [upload=514373]: Done uploading segments... Feb 9 00:14:52.340476462 (584h19m30.224430647s) [upload=514373]: Upload finished. err: context deadline exceeded
Duration: 2.42818867s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
2.421428134s trace:
https://jaeger.dev.storj.io/trace/2fe3acd100d740fd
FAIL (2026-02-09 00:27:04 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Feb 9 00:14:22.628823254 (584h19m0.512777443s) [upload=514373]: Starting upload Feb 9 00:14:22.628881807 (584h19m0.512836015s) [upload=514373]: Done starting upload Feb 9 00:14:22.629148558 (584h19m0.513102751s) [upload=514373]: Uploading segments... Feb 9 00:14:22.629188448 (584h19m0.51314265s) [upload=514373]: Got next segment. Inline: false Feb 9 00:14:22.629227334 (584h19m0.513181534s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Feb 9 00:14:22.629232569 (584h19m0.513186769s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Feb 9 00:14:52.340184039 (584h19m30.224138241s) [upload=514373]: Next segment err: context deadline exceeded Feb 9 00:14:52.340437904 (584h19m30.224392094s) [upload=514373]: Done uploading segments... Feb 9 00:14:52.340476462 (584h19m30.224430647s) [upload=514373]: Upload finished. err: context deadline exceeded
Duration: 2.421460051s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
2.438716719s trace:
https://jaeger.dev.storj.io/trace/2fe3acd100d740fd
FAIL (2026-02-09 00:26:00 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Feb 9 00:14:22.628823254 (584h19m0.512777443s) [upload=514373]: Starting upload Feb 9 00:14:22.628881807 (584h19m0.512836015s) [upload=514373]: Done starting upload Feb 9 00:14:22.629148558 (584h19m0.513102751s) [upload=514373]: Uploading segments... Feb 9 00:14:22.629188448 (584h19m0.51314265s) [upload=514373]: Got next segment. Inline: false Feb 9 00:14:22.629227334 (584h19m0.513181534s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Feb 9 00:14:22.629232569 (584h19m0.513186769s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Feb 9 00:14:52.340184039 (584h19m30.224138241s) [upload=514373]: Next segment err: context deadline exceeded Feb 9 00:14:52.340437904 (584h19m30.224392094s) [upload=514373]: Done uploading segments... Feb 9 00:14:52.340476462 (584h19m30.224430647s) [upload=514373]: Upload finished. err: context deadline exceeded
Duration: 2.43875025s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
2.454694176s trace:
https://jaeger.dev.storj.io/trace/2fe3acd100d740fd
FAIL (2026-02-09 00:24:53 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Feb 9 00:14:22.628823254 (584h19m0.512777443s) [upload=514373]: Starting upload Feb 9 00:14:22.628881807 (584h19m0.512836015s) [upload=514373]: Done starting upload Feb 9 00:14:22.629148558 (584h19m0.513102751s) [upload=514373]: Uploading segments... Feb 9 00:14:22.629188448 (584h19m0.51314265s) [upload=514373]: Got next segment. Inline: false Feb 9 00:14:22.629227334 (584h19m0.513181534s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Feb 9 00:14:22.629232569 (584h19m0.513186769s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Feb 9 00:14:52.340184039 (584h19m30.224138241s) [upload=514373]: Next segment err: context deadline exceeded Feb 9 00:14:52.340437904 (584h19m30.224392094s) [upload=514373]: Done uploading segments... Feb 9 00:14:52.340476462 (584h19m30.224430647s) [upload=514373]: Upload finished. err: context deadline exceeded
Duration: 2.454721152s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
2.388603654s trace:
https://jaeger.dev.storj.io/trace/2fe3acd100d740fd
FAIL (2026-02-09 00:23:52 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Feb 9 00:14:22.628823254 (584h19m0.512777443s) [upload=514373]: Starting upload Feb 9 00:14:22.628881807 (584h19m0.512836015s) [upload=514373]: Done starting upload Feb 9 00:14:22.629148558 (584h19m0.513102751s) [upload=514373]: Uploading segments... Feb 9 00:14:22.629188448 (584h19m0.51314265s) [upload=514373]: Got next segment. Inline: false Feb 9 00:14:22.629227334 (584h19m0.513181534s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Feb 9 00:14:22.629232569 (584h19m0.513186769s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Feb 9 00:14:52.340184039 (584h19m30.224138241s) [upload=514373]: Next segment err: context deadline exceeded Feb 9 00:14:52.340437904 (584h19m30.224392094s) [upload=514373]: Done uploading segments... Feb 9 00:14:52.340476462 (584h19m30.224430647s) [upload=514373]: Upload finished. err: context deadline exceeded
Duration: 2.388726749s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
2.409619974s trace:
https://jaeger.dev.storj.io/trace/2fe3acd100d740fd
FAIL (2026-02-09 00:23:00 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Feb 9 00:14:22.628823254 (584h19m0.512777443s) [upload=514373]: Starting upload Feb 9 00:14:22.628881807 (584h19m0.512836015s) [upload=514373]: Done starting upload Feb 9 00:14:22.629148558 (584h19m0.513102751s) [upload=514373]: Uploading segments... Feb 9 00:14:22.629188448 (584h19m0.51314265s) [upload=514373]: Got next segment. Inline: false Feb 9 00:14:22.629227334 (584h19m0.513181534s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Feb 9 00:14:22.629232569 (584h19m0.513186769s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Feb 9 00:14:52.340184039 (584h19m30.224138241s) [upload=514373]: Next segment err: context deadline exceeded Feb 9 00:14:52.340437904 (584h19m30.224392094s) [upload=514373]: Done uploading segments... Feb 9 00:14:52.340476462 (584h19m30.224430647s) [upload=514373]: Upload finished. err: context deadline exceeded
Duration: 2.409650027s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
2.251402426s trace:
https://jaeger.dev.storj.io/trace/2fe3acd100d740fd
FAIL (2026-02-09 00:22:30 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Feb 9 00:14:22.628823254 (584h19m0.512777443s) [upload=514373]: Starting upload Feb 9 00:14:22.628881807 (584h19m0.512836015s) [upload=514373]: Done starting upload Feb 9 00:14:22.629148558 (584h19m0.513102751s) [upload=514373]: Uploading segments... Feb 9 00:14:22.629188448 (584h19m0.51314265s) [upload=514373]: Got next segment. Inline: false Feb 9 00:14:22.629227334 (584h19m0.513181534s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Feb 9 00:14:22.629232569 (584h19m0.513186769s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Feb 9 00:14:52.340184039 (584h19m30.224138241s) [upload=514373]: Next segment err: context deadline exceeded Feb 9 00:14:52.340437904 (584h19m30.224392094s) [upload=514373]: Done uploading segments... Feb 9 00:14:52.340476462 (584h19m30.224430647s) [upload=514373]: Upload finished. err: context deadline exceeded
Duration: 2.251450818s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
2.393902659s trace:
https://jaeger.dev.storj.io/trace/2fe3acd100d740fd
FAIL (2026-02-09 00:21:20 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Feb 9 00:14:22.628823254 (584h19m0.512777443s) [upload=514373]: Starting upload Feb 9 00:14:22.628881807 (584h19m0.512836015s) [upload=514373]: Done starting upload Feb 9 00:14:22.629148558 (584h19m0.513102751s) [upload=514373]: Uploading segments... Feb 9 00:14:22.629188448 (584h19m0.51314265s) [upload=514373]: Got next segment. Inline: false Feb 9 00:14:22.629227334 (584h19m0.513181534s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Feb 9 00:14:22.629232569 (584h19m0.513186769s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Feb 9 00:14:52.340184039 (584h19m30.224138241s) [upload=514373]: Next segment err: context deadline exceeded Feb 9 00:14:52.340437904 (584h19m30.224392094s) [upload=514373]: Done uploading segments... Feb 9 00:14:52.340476462 (584h19m30.224430647s) [upload=514373]: Upload finished. err: context deadline exceeded
Duration: 2.393932169s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
2.249109992s trace:
https://jaeger.dev.storj.io/trace/2fe3acd100d740fd
FAIL (2026-02-09 00:20:57 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Feb 9 00:14:22.628823254 (584h19m0.512777443s) [upload=514373]: Starting upload Feb 9 00:14:22.628881807 (584h19m0.512836015s) [upload=514373]: Done starting upload Feb 9 00:14:22.629148558 (584h19m0.513102751s) [upload=514373]: Uploading segments... Feb 9 00:14:22.629188448 (584h19m0.51314265s) [upload=514373]: Got next segment. Inline: false Feb 9 00:14:22.629227334 (584h19m0.513181534s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Feb 9 00:14:22.629232569 (584h19m0.513186769s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Feb 9 00:14:52.340184039 (584h19m30.224138241s) [upload=514373]: Next segment err: context deadline exceeded Feb 9 00:14:52.340437904 (584h19m30.224392094s) [upload=514373]: Done uploading segments... Feb 9 00:14:52.340476462 (584h19m30.224430647s) [upload=514373]: Upload finished. err: context deadline exceeded
Duration: 2.249136289s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
2.318103985s trace:
https://jaeger.dev.storj.io/trace/2fe3acd100d740fd
FAIL (2026-02-09 00:19:53 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Feb 9 00:14:22.628823254 (584h19m0.512777443s) [upload=514373]: Starting upload Feb 9 00:14:22.628881807 (584h19m0.512836015s) [upload=514373]: Done starting upload Feb 9 00:14:22.629148558 (584h19m0.513102751s) [upload=514373]: Uploading segments... Feb 9 00:14:22.629188448 (584h19m0.51314265s) [upload=514373]: Got next segment. Inline: false Feb 9 00:14:22.629227334 (584h19m0.513181534s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Feb 9 00:14:22.629232569 (584h19m0.513186769s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Feb 9 00:14:52.340184039 (584h19m30.224138241s) [upload=514373]: Next segment err: context deadline exceeded Feb 9 00:14:52.340437904 (584h19m30.224392094s) [upload=514373]: Done uploading segments... Feb 9 00:14:52.340476462 (584h19m30.224430647s) [upload=514373]: Upload finished. err: context deadline exceeded
Duration: 2.318131835s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
2.348070789s trace:
https://jaeger.dev.storj.io/trace/2fe3acd100d740fd
FAIL (2026-02-09 00:19:23 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Feb 9 00:14:22.628823254 (584h19m0.512777443s) [upload=514373]: Starting upload Feb 9 00:14:22.628881807 (584h19m0.512836015s) [upload=514373]: Done starting upload Feb 9 00:14:22.629148558 (584h19m0.513102751s) [upload=514373]: Uploading segments... Feb 9 00:14:22.629188448 (584h19m0.51314265s) [upload=514373]: Got next segment. Inline: false Feb 9 00:14:22.629227334 (584h19m0.513181534s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Feb 9 00:14:22.629232569 (584h19m0.513186769s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Feb 9 00:14:52.340184039 (584h19m30.224138241s) [upload=514373]: Next segment err: context deadline exceeded Feb 9 00:14:52.340437904 (584h19m30.224392094s) [upload=514373]: Done uploading segments... Feb 9 00:14:52.340476462 (584h19m30.224430647s) [upload=514373]: Upload finished. err: context deadline exceeded
Duration: 2.348098269s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
2.361332348s trace:
https://jaeger.dev.storj.io/trace/2fe3acd100d740fd
FAIL (2026-02-09 00:18:28 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Feb 9 00:14:22.628823254 (584h19m0.512777443s) [upload=514373]: Starting upload Feb 9 00:14:22.628881807 (584h19m0.512836015s) [upload=514373]: Done starting upload Feb 9 00:14:22.629148558 (584h19m0.513102751s) [upload=514373]: Uploading segments... Feb 9 00:14:22.629188448 (584h19m0.51314265s) [upload=514373]: Got next segment. Inline: false Feb 9 00:14:22.629227334 (584h19m0.513181534s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Feb 9 00:14:22.629232569 (584h19m0.513186769s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Feb 9 00:14:52.340184039 (584h19m30.224138241s) [upload=514373]: Next segment err: context deadline exceeded Feb 9 00:14:52.340437904 (584h19m30.224392094s) [upload=514373]: Done uploading segments... Feb 9 00:14:52.340476462 (584h19m30.224430647s) [upload=514373]: Upload finished. err: context deadline exceeded
Duration: 2.361358514s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
2.356884369s trace:
https://jaeger.dev.storj.io/trace/2fe3acd100d740fd
FAIL (2026-02-09 00:17:20 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Feb 9 00:14:22.628823254 (584h19m0.512777443s) [upload=514373]: Starting upload Feb 9 00:14:22.628881807 (584h19m0.512836015s) [upload=514373]: Done starting upload Feb 9 00:14:22.629148558 (584h19m0.513102751s) [upload=514373]: Uploading segments... Feb 9 00:14:22.629188448 (584h19m0.51314265s) [upload=514373]: Got next segment. Inline: false Feb 9 00:14:22.629227334 (584h19m0.513181534s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Feb 9 00:14:22.629232569 (584h19m0.513186769s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Feb 9 00:14:52.340184039 (584h19m30.224138241s) [upload=514373]: Next segment err: context deadline exceeded Feb 9 00:14:52.340437904 (584h19m30.224392094s) [upload=514373]: Done uploading segments... Feb 9 00:14:52.340476462 (584h19m30.224430647s) [upload=514373]: Upload finished. err: context deadline exceeded
Duration: 2.356908085s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
2.264842694s trace:
https://jaeger.dev.storj.io/trace/2fe3acd100d740fd
FAIL (2026-02-09 00:16:45 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Feb 9 00:14:22.628823254 (584h19m0.512777443s) [upload=514373]: Starting upload Feb 9 00:14:22.628881807 (584h19m0.512836015s) [upload=514373]: Done starting upload Feb 9 00:14:22.629148558 (584h19m0.513102751s) [upload=514373]: Uploading segments... Feb 9 00:14:22.629188448 (584h19m0.51314265s) [upload=514373]: Got next segment. Inline: false Feb 9 00:14:22.629227334 (584h19m0.513181534s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Feb 9 00:14:22.629232569 (584h19m0.513186769s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Feb 9 00:14:52.340184039 (584h19m30.224138241s) [upload=514373]: Next segment err: context deadline exceeded Feb 9 00:14:52.340437904 (584h19m30.224392094s) [upload=514373]: Done uploading segments... Feb 9 00:14:52.340476462 (584h19m30.224430647s) [upload=514373]: Upload finished. err: context deadline exceeded
Duration: 2.264947358s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
2.342634475s trace:
https://jaeger.dev.storj.io/trace/2fe3acd100d740fd
FAIL (2026-02-09 00:16:10 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Feb 9 00:14:22.628823254 (584h19m0.512777443s) [upload=514373]: Starting upload Feb 9 00:14:22.628881807 (584h19m0.512836015s) [upload=514373]: Done starting upload Feb 9 00:14:22.629148558 (584h19m0.513102751s) [upload=514373]: Uploading segments... Feb 9 00:14:22.629188448 (584h19m0.51314265s) [upload=514373]: Got next segment. Inline: false Feb 9 00:14:22.629227334 (584h19m0.513181534s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Feb 9 00:14:22.629232569 (584h19m0.513186769s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Feb 9 00:14:52.340184039 (584h19m30.224138241s) [upload=514373]: Next segment err: context deadline exceeded Feb 9 00:14:52.340437904 (584h19m30.224392094s) [upload=514373]: Done uploading segments... Feb 9 00:14:52.340476462 (584h19m30.224430647s) [upload=514373]: Upload finished. err: context deadline exceeded
Duration: 2.342669466s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
2.383594019s trace:
https://jaeger.dev.storj.io/trace/2fe3acd100d740fd
FAIL (2026-02-09 00:14:57 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Feb 9 00:14:22.628823254 (584h19m0.512777443s) [upload=514373]: Starting upload Feb 9 00:14:22.628881807 (584h19m0.512836015s) [upload=514373]: Done starting upload Feb 9 00:14:22.629148558 (584h19m0.513102751s) [upload=514373]: Uploading segments... Feb 9 00:14:22.629188448 (584h19m0.51314265s) [upload=514373]: Got next segment. Inline: false Feb 9 00:14:22.629227334 (584h19m0.513181534s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Feb 9 00:14:22.629232569 (584h19m0.513186769s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Feb 9 00:14:52.340184039 (584h19m30.224138241s) [upload=514373]: Next segment err: context deadline exceeded Feb 9 00:14:52.340437904 (584h19m30.224392094s) [upload=514373]: Done uploading segments... Feb 9 00:14:52.340476462 (584h19m30.224430647s) [upload=514373]: Upload finished. err: context deadline exceeded
Duration: 2.383612569s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
30.001590293s trace:
https://jaeger.dev.storj.io/trace/2fe3acd100d740fd
FAIL (2026-02-09 00:14:52 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Feb 9 00:14:22.628823254 (584h19m0.512777443s) [upload=514373]: Starting upload Feb 9 00:14:22.628881807 (584h19m0.512836015s) [upload=514373]: Done starting upload Feb 9 00:14:22.629148558 (584h19m0.513102751s) [upload=514373]: Uploading segments... Feb 9 00:14:22.629188448 (584h19m0.51314265s) [upload=514373]: Got next segment. Inline: false Feb 9 00:14:22.629227334 (584h19m0.513181534s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Feb 9 00:14:22.629232569 (584h19m0.513186769s) [upload=514373]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Feb 9 00:14:52.340184039 (584h19m30.224138241s) [upload=514373]: Next segment err: context deadline exceeded Feb 9 00:14:52.340437904 (584h19m30.224392094s) [upload=514373]: Done uploading segments... Feb 9 00:14:52.340476462 (584h19m30.224430647s) [upload=514373]: Upload finished. err: context deadline exceeded
Duration: 30.001613299s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
2.4764408s trace:
https://jaeger.dev.storj.io/trace/3c8a03916dd3efb0
FAIL (2026-02-08 21:19:50 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Feb 8 20:35:15.284031765 (580h39m53.167985964s) [upload=511155]: Starting upload
Feb 8 20:35:15.284112500 (580h39m53.168066701s) [upload=511155]: Done starting upload
Feb 8 20:35:15.284149599 (580h39m53.168103791s) [upload=511155]: Uploading segments...
Feb 8 20:35:15.284601793 (580h39m53.168555992s) [upload=511155]: Got next segment. Inline: false
Feb 8 20:35:15.284721869 (580h39m53.16867607s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 20:35:15.284728431 (580h39m53.16868263s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 20:35:15.397389353 (580h39m53.281343559s) [seg_pos={0 0}, upload=511155]: Begin upload segment...
Feb 8 20:35:15.397453134 (580h39m53.281407318s) [seg_pos={0 0}, upload=511155]: Done begin upload segment.
Feb 8 20:35:15.397517573 (580h39m53.281471782s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.397533666 (580h39m53.281487892s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398120348 (580h39m53.282074556s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398572052 (580h39m53.282526262s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399001740 (580h39m53.282955955s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399492847 (580h39m53.283447038s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399954482 (580h39m53.283908691s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.400483442 (580h39m53.284437649s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.402650352 (580h39m53.286604556s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.404412781 (580h39m53.288366984s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.406266549 (580h39m53.290220752s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.407902210 (580h39m53.291856418s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.409594415 (580h39m53.293548626s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.410340922 (580h39m53.294295132s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411911194 (580h39m53.2958654s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411928878 (580h39m53.29588307s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:16.207177374 (580h39m54.091131559s) [upload=511155]: Next returned nil segment
Feb 8 20:35:16.207191375 (580h39m54.091145561s) [upload=511155]: Waiting for error group managing segments...
Feb 8 20:35:16.334354958 (580h39m54.21830916s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.335358031 (580h39m54.21931223s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.337810738 (580h39m54.221764931s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.338164047 (580h39m54.222118238s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.342858708 (580h39m54.226812913s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.343378259 (580h39m54.227332458s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.345609935 (580h39m54.229564141s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.350042706 (580h39m54.233996915s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.379091195 (580h39m54.263045398s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.387202016 (580h39m54.271156202s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.390136524 (580h39m54.274090711s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:44.095174165 (580h40m21.979128365s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:45.002467024 (580h40m22.886421233s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:1JgPWqe81FwGvfduteWKs6QAmxY3Av3Nbd4RBu4fcZh5rH3RMM, address:79.127.219.41:4927): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002509190 (580h40m22.886463383s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002708091 (580h40m22.886662296s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:19MoR7r7wEgpnEpEesf4oJzFsWZmygg7mxg4jCtGf9rhJDw56B, address:45.91.116.4:4947): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002855562 (580h40m22.88680977s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12A39mM8Cc7tERvweTDoj4wkGgNTNn2QafEkMhn8PV1rwREj8WP, address:79.127.219.41:4930): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002888009 (580h40m22.886842201s) [seg_pos={0 0}, upload=511155]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Feb 8 20:35:45.110856986 (580h40m22.994811201s) [upload=511155]: Done uploading segments...
Feb 8 20:35:45.110918692 (580h40m22.994872902s) [upload=511155]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.476551688s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
2.371632704s trace:
https://jaeger.dev.storj.io/trace/3c8a03916dd3efb0
FAIL (2026-02-08 21:18:41 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Feb 8 20:35:15.284031765 (580h39m53.167985964s) [upload=511155]: Starting upload
Feb 8 20:35:15.284112500 (580h39m53.168066701s) [upload=511155]: Done starting upload
Feb 8 20:35:15.284149599 (580h39m53.168103791s) [upload=511155]: Uploading segments...
Feb 8 20:35:15.284601793 (580h39m53.168555992s) [upload=511155]: Got next segment. Inline: false
Feb 8 20:35:15.284721869 (580h39m53.16867607s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 20:35:15.284728431 (580h39m53.16868263s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 20:35:15.397389353 (580h39m53.281343559s) [seg_pos={0 0}, upload=511155]: Begin upload segment...
Feb 8 20:35:15.397453134 (580h39m53.281407318s) [seg_pos={0 0}, upload=511155]: Done begin upload segment.
Feb 8 20:35:15.397517573 (580h39m53.281471782s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.397533666 (580h39m53.281487892s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398120348 (580h39m53.282074556s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398572052 (580h39m53.282526262s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399001740 (580h39m53.282955955s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399492847 (580h39m53.283447038s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399954482 (580h39m53.283908691s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.400483442 (580h39m53.284437649s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.402650352 (580h39m53.286604556s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.404412781 (580h39m53.288366984s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.406266549 (580h39m53.290220752s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.407902210 (580h39m53.291856418s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.409594415 (580h39m53.293548626s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.410340922 (580h39m53.294295132s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411911194 (580h39m53.2958654s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411928878 (580h39m53.29588307s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:16.207177374 (580h39m54.091131559s) [upload=511155]: Next returned nil segment
Feb 8 20:35:16.207191375 (580h39m54.091145561s) [upload=511155]: Waiting for error group managing segments...
Feb 8 20:35:16.334354958 (580h39m54.21830916s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.335358031 (580h39m54.21931223s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.337810738 (580h39m54.221764931s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.338164047 (580h39m54.222118238s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.342858708 (580h39m54.226812913s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.343378259 (580h39m54.227332458s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.345609935 (580h39m54.229564141s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.350042706 (580h39m54.233996915s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.379091195 (580h39m54.263045398s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.387202016 (580h39m54.271156202s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.390136524 (580h39m54.274090711s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:44.095174165 (580h40m21.979128365s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:45.002467024 (580h40m22.886421233s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:1JgPWqe81FwGvfduteWKs6QAmxY3Av3Nbd4RBu4fcZh5rH3RMM, address:79.127.219.41:4927): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002509190 (580h40m22.886463383s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002708091 (580h40m22.886662296s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:19MoR7r7wEgpnEpEesf4oJzFsWZmygg7mxg4jCtGf9rhJDw56B, address:45.91.116.4:4947): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002855562 (580h40m22.88680977s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12A39mM8Cc7tERvweTDoj4wkGgNTNn2QafEkMhn8PV1rwREj8WP, address:79.127.219.41:4930): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002888009 (580h40m22.886842201s) [seg_pos={0 0}, upload=511155]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Feb 8 20:35:45.110856986 (580h40m22.994811201s) [upload=511155]: Done uploading segments...
Feb 8 20:35:45.110918692 (580h40m22.994872902s) [upload=511155]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.371663885s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
2.374090182s trace:
https://jaeger.dev.storj.io/trace/3c8a03916dd3efb0
FAIL (2026-02-08 21:18:22 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Feb 8 20:35:15.284031765 (580h39m53.167985964s) [upload=511155]: Starting upload
Feb 8 20:35:15.284112500 (580h39m53.168066701s) [upload=511155]: Done starting upload
Feb 8 20:35:15.284149599 (580h39m53.168103791s) [upload=511155]: Uploading segments...
Feb 8 20:35:15.284601793 (580h39m53.168555992s) [upload=511155]: Got next segment. Inline: false
Feb 8 20:35:15.284721869 (580h39m53.16867607s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 20:35:15.284728431 (580h39m53.16868263s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 20:35:15.397389353 (580h39m53.281343559s) [seg_pos={0 0}, upload=511155]: Begin upload segment...
Feb 8 20:35:15.397453134 (580h39m53.281407318s) [seg_pos={0 0}, upload=511155]: Done begin upload segment.
Feb 8 20:35:15.397517573 (580h39m53.281471782s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.397533666 (580h39m53.281487892s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398120348 (580h39m53.282074556s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398572052 (580h39m53.282526262s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399001740 (580h39m53.282955955s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399492847 (580h39m53.283447038s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399954482 (580h39m53.283908691s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.400483442 (580h39m53.284437649s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.402650352 (580h39m53.286604556s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.404412781 (580h39m53.288366984s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.406266549 (580h39m53.290220752s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.407902210 (580h39m53.291856418s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.409594415 (580h39m53.293548626s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.410340922 (580h39m53.294295132s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411911194 (580h39m53.2958654s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411928878 (580h39m53.29588307s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:16.207177374 (580h39m54.091131559s) [upload=511155]: Next returned nil segment
Feb 8 20:35:16.207191375 (580h39m54.091145561s) [upload=511155]: Waiting for error group managing segments...
Feb 8 20:35:16.334354958 (580h39m54.21830916s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.335358031 (580h39m54.21931223s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.337810738 (580h39m54.221764931s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.338164047 (580h39m54.222118238s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.342858708 (580h39m54.226812913s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.343378259 (580h39m54.227332458s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.345609935 (580h39m54.229564141s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.350042706 (580h39m54.233996915s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.379091195 (580h39m54.263045398s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.387202016 (580h39m54.271156202s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.390136524 (580h39m54.274090711s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:44.095174165 (580h40m21.979128365s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:45.002467024 (580h40m22.886421233s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:1JgPWqe81FwGvfduteWKs6QAmxY3Av3Nbd4RBu4fcZh5rH3RMM, address:79.127.219.41:4927): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002509190 (580h40m22.886463383s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002708091 (580h40m22.886662296s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:19MoR7r7wEgpnEpEesf4oJzFsWZmygg7mxg4jCtGf9rhJDw56B, address:45.91.116.4:4947): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002855562 (580h40m22.88680977s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12A39mM8Cc7tERvweTDoj4wkGgNTNn2QafEkMhn8PV1rwREj8WP, address:79.127.219.41:4930): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002888009 (580h40m22.886842201s) [seg_pos={0 0}, upload=511155]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Feb 8 20:35:45.110856986 (580h40m22.994811201s) [upload=511155]: Done uploading segments...
Feb 8 20:35:45.110918692 (580h40m22.994872902s) [upload=511155]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.374118366s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
2.553919699s trace:
https://jaeger.dev.storj.io/trace/3c8a03916dd3efb0
FAIL (2026-02-08 21:17:35 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Feb 8 20:35:15.284031765 (580h39m53.167985964s) [upload=511155]: Starting upload
Feb 8 20:35:15.284112500 (580h39m53.168066701s) [upload=511155]: Done starting upload
Feb 8 20:35:15.284149599 (580h39m53.168103791s) [upload=511155]: Uploading segments...
Feb 8 20:35:15.284601793 (580h39m53.168555992s) [upload=511155]: Got next segment. Inline: false
Feb 8 20:35:15.284721869 (580h39m53.16867607s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 20:35:15.284728431 (580h39m53.16868263s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 20:35:15.397389353 (580h39m53.281343559s) [seg_pos={0 0}, upload=511155]: Begin upload segment...
Feb 8 20:35:15.397453134 (580h39m53.281407318s) [seg_pos={0 0}, upload=511155]: Done begin upload segment.
Feb 8 20:35:15.397517573 (580h39m53.281471782s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.397533666 (580h39m53.281487892s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398120348 (580h39m53.282074556s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398572052 (580h39m53.282526262s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399001740 (580h39m53.282955955s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399492847 (580h39m53.283447038s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399954482 (580h39m53.283908691s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.400483442 (580h39m53.284437649s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.402650352 (580h39m53.286604556s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.404412781 (580h39m53.288366984s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.406266549 (580h39m53.290220752s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.407902210 (580h39m53.291856418s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.409594415 (580h39m53.293548626s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.410340922 (580h39m53.294295132s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411911194 (580h39m53.2958654s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411928878 (580h39m53.29588307s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:16.207177374 (580h39m54.091131559s) [upload=511155]: Next returned nil segment
Feb 8 20:35:16.207191375 (580h39m54.091145561s) [upload=511155]: Waiting for error group managing segments...
Feb 8 20:35:16.334354958 (580h39m54.21830916s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.335358031 (580h39m54.21931223s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.337810738 (580h39m54.221764931s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.338164047 (580h39m54.222118238s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.342858708 (580h39m54.226812913s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.343378259 (580h39m54.227332458s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.345609935 (580h39m54.229564141s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.350042706 (580h39m54.233996915s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.379091195 (580h39m54.263045398s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.387202016 (580h39m54.271156202s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.390136524 (580h39m54.274090711s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:44.095174165 (580h40m21.979128365s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:45.002467024 (580h40m22.886421233s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:1JgPWqe81FwGvfduteWKs6QAmxY3Av3Nbd4RBu4fcZh5rH3RMM, address:79.127.219.41:4927): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002509190 (580h40m22.886463383s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002708091 (580h40m22.886662296s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:19MoR7r7wEgpnEpEesf4oJzFsWZmygg7mxg4jCtGf9rhJDw56B, address:45.91.116.4:4947): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002855562 (580h40m22.88680977s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12A39mM8Cc7tERvweTDoj4wkGgNTNn2QafEkMhn8PV1rwREj8WP, address:79.127.219.41:4930): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002888009 (580h40m22.886842201s) [seg_pos={0 0}, upload=511155]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Feb 8 20:35:45.110856986 (580h40m22.994811201s) [upload=511155]: Done uploading segments...
Feb 8 20:35:45.110918692 (580h40m22.994872902s) [upload=511155]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.553960334s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
2.333401902s trace:
https://jaeger.dev.storj.io/trace/3c8a03916dd3efb0
FAIL (2026-02-08 21:16:59 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Feb 8 20:35:15.284031765 (580h39m53.167985964s) [upload=511155]: Starting upload
Feb 8 20:35:15.284112500 (580h39m53.168066701s) [upload=511155]: Done starting upload
Feb 8 20:35:15.284149599 (580h39m53.168103791s) [upload=511155]: Uploading segments...
Feb 8 20:35:15.284601793 (580h39m53.168555992s) [upload=511155]: Got next segment. Inline: false
Feb 8 20:35:15.284721869 (580h39m53.16867607s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 20:35:15.284728431 (580h39m53.16868263s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 20:35:15.397389353 (580h39m53.281343559s) [seg_pos={0 0}, upload=511155]: Begin upload segment...
Feb 8 20:35:15.397453134 (580h39m53.281407318s) [seg_pos={0 0}, upload=511155]: Done begin upload segment.
Feb 8 20:35:15.397517573 (580h39m53.281471782s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.397533666 (580h39m53.281487892s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398120348 (580h39m53.282074556s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398572052 (580h39m53.282526262s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399001740 (580h39m53.282955955s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399492847 (580h39m53.283447038s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399954482 (580h39m53.283908691s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.400483442 (580h39m53.284437649s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.402650352 (580h39m53.286604556s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.404412781 (580h39m53.288366984s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.406266549 (580h39m53.290220752s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.407902210 (580h39m53.291856418s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.409594415 (580h39m53.293548626s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.410340922 (580h39m53.294295132s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411911194 (580h39m53.2958654s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411928878 (580h39m53.29588307s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:16.207177374 (580h39m54.091131559s) [upload=511155]: Next returned nil segment
Feb 8 20:35:16.207191375 (580h39m54.091145561s) [upload=511155]: Waiting for error group managing segments...
Feb 8 20:35:16.334354958 (580h39m54.21830916s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.335358031 (580h39m54.21931223s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.337810738 (580h39m54.221764931s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.338164047 (580h39m54.222118238s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.342858708 (580h39m54.226812913s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.343378259 (580h39m54.227332458s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.345609935 (580h39m54.229564141s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.350042706 (580h39m54.233996915s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.379091195 (580h39m54.263045398s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.387202016 (580h39m54.271156202s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.390136524 (580h39m54.274090711s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:44.095174165 (580h40m21.979128365s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:45.002467024 (580h40m22.886421233s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:1JgPWqe81FwGvfduteWKs6QAmxY3Av3Nbd4RBu4fcZh5rH3RMM, address:79.127.219.41:4927): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002509190 (580h40m22.886463383s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002708091 (580h40m22.886662296s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:19MoR7r7wEgpnEpEesf4oJzFsWZmygg7mxg4jCtGf9rhJDw56B, address:45.91.116.4:4947): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002855562 (580h40m22.88680977s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12A39mM8Cc7tERvweTDoj4wkGgNTNn2QafEkMhn8PV1rwREj8WP, address:79.127.219.41:4930): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002888009 (580h40m22.886842201s) [seg_pos={0 0}, upload=511155]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Feb 8 20:35:45.110856986 (580h40m22.994811201s) [upload=511155]: Done uploading segments...
Feb 8 20:35:45.110918692 (580h40m22.994872902s) [upload=511155]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.333430504s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
2.662280998s trace:
https://jaeger.dev.storj.io/trace/3c8a03916dd3efb0
FAIL (2026-02-08 21:16:04 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Feb 8 20:35:15.284031765 (580h39m53.167985964s) [upload=511155]: Starting upload
Feb 8 20:35:15.284112500 (580h39m53.168066701s) [upload=511155]: Done starting upload
Feb 8 20:35:15.284149599 (580h39m53.168103791s) [upload=511155]: Uploading segments...
Feb 8 20:35:15.284601793 (580h39m53.168555992s) [upload=511155]: Got next segment. Inline: false
Feb 8 20:35:15.284721869 (580h39m53.16867607s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 20:35:15.284728431 (580h39m53.16868263s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 20:35:15.397389353 (580h39m53.281343559s) [seg_pos={0 0}, upload=511155]: Begin upload segment...
Feb 8 20:35:15.397453134 (580h39m53.281407318s) [seg_pos={0 0}, upload=511155]: Done begin upload segment.
Feb 8 20:35:15.397517573 (580h39m53.281471782s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.397533666 (580h39m53.281487892s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398120348 (580h39m53.282074556s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398572052 (580h39m53.282526262s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399001740 (580h39m53.282955955s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399492847 (580h39m53.283447038s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399954482 (580h39m53.283908691s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.400483442 (580h39m53.284437649s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.402650352 (580h39m53.286604556s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.404412781 (580h39m53.288366984s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.406266549 (580h39m53.290220752s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.407902210 (580h39m53.291856418s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.409594415 (580h39m53.293548626s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.410340922 (580h39m53.294295132s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411911194 (580h39m53.2958654s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411928878 (580h39m53.29588307s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:16.207177374 (580h39m54.091131559s) [upload=511155]: Next returned nil segment
Feb 8 20:35:16.207191375 (580h39m54.091145561s) [upload=511155]: Waiting for error group managing segments...
Feb 8 20:35:16.334354958 (580h39m54.21830916s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.335358031 (580h39m54.21931223s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.337810738 (580h39m54.221764931s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.338164047 (580h39m54.222118238s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.342858708 (580h39m54.226812913s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.343378259 (580h39m54.227332458s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.345609935 (580h39m54.229564141s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.350042706 (580h39m54.233996915s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.379091195 (580h39m54.263045398s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.387202016 (580h39m54.271156202s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.390136524 (580h39m54.274090711s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:44.095174165 (580h40m21.979128365s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:45.002467024 (580h40m22.886421233s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:1JgPWqe81FwGvfduteWKs6QAmxY3Av3Nbd4RBu4fcZh5rH3RMM, address:79.127.219.41:4927): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002509190 (580h40m22.886463383s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002708091 (580h40m22.886662296s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:19MoR7r7wEgpnEpEesf4oJzFsWZmygg7mxg4jCtGf9rhJDw56B, address:45.91.116.4:4947): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002855562 (580h40m22.88680977s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12A39mM8Cc7tERvweTDoj4wkGgNTNn2QafEkMhn8PV1rwREj8WP, address:79.127.219.41:4930): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002888009 (580h40m22.886842201s) [seg_pos={0 0}, upload=511155]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Feb 8 20:35:45.110856986 (580h40m22.994811201s) [upload=511155]: Done uploading segments...
Feb 8 20:35:45.110918692 (580h40m22.994872902s) [upload=511155]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.662394437s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
2.280353724s trace:
https://jaeger.dev.storj.io/trace/3c8a03916dd3efb0
FAIL (2026-02-08 21:14:59 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Feb 8 20:35:15.284031765 (580h39m53.167985964s) [upload=511155]: Starting upload
Feb 8 20:35:15.284112500 (580h39m53.168066701s) [upload=511155]: Done starting upload
Feb 8 20:35:15.284149599 (580h39m53.168103791s) [upload=511155]: Uploading segments...
Feb 8 20:35:15.284601793 (580h39m53.168555992s) [upload=511155]: Got next segment. Inline: false
Feb 8 20:35:15.284721869 (580h39m53.16867607s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 20:35:15.284728431 (580h39m53.16868263s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 20:35:15.397389353 (580h39m53.281343559s) [seg_pos={0 0}, upload=511155]: Begin upload segment...
Feb 8 20:35:15.397453134 (580h39m53.281407318s) [seg_pos={0 0}, upload=511155]: Done begin upload segment.
Feb 8 20:35:15.397517573 (580h39m53.281471782s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.397533666 (580h39m53.281487892s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398120348 (580h39m53.282074556s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398572052 (580h39m53.282526262s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399001740 (580h39m53.282955955s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399492847 (580h39m53.283447038s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399954482 (580h39m53.283908691s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.400483442 (580h39m53.284437649s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.402650352 (580h39m53.286604556s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.404412781 (580h39m53.288366984s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.406266549 (580h39m53.290220752s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.407902210 (580h39m53.291856418s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.409594415 (580h39m53.293548626s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.410340922 (580h39m53.294295132s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411911194 (580h39m53.2958654s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411928878 (580h39m53.29588307s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:16.207177374 (580h39m54.091131559s) [upload=511155]: Next returned nil segment
Feb 8 20:35:16.207191375 (580h39m54.091145561s) [upload=511155]: Waiting for error group managing segments...
Feb 8 20:35:16.334354958 (580h39m54.21830916s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.335358031 (580h39m54.21931223s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.337810738 (580h39m54.221764931s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.338164047 (580h39m54.222118238s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.342858708 (580h39m54.226812913s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.343378259 (580h39m54.227332458s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.345609935 (580h39m54.229564141s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.350042706 (580h39m54.233996915s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.379091195 (580h39m54.263045398s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.387202016 (580h39m54.271156202s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.390136524 (580h39m54.274090711s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:44.095174165 (580h40m21.979128365s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:45.002467024 (580h40m22.886421233s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:1JgPWqe81FwGvfduteWKs6QAmxY3Av3Nbd4RBu4fcZh5rH3RMM, address:79.127.219.41:4927): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002509190 (580h40m22.886463383s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002708091 (580h40m22.886662296s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:19MoR7r7wEgpnEpEesf4oJzFsWZmygg7mxg4jCtGf9rhJDw56B, address:45.91.116.4:4947): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002855562 (580h40m22.88680977s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12A39mM8Cc7tERvweTDoj4wkGgNTNn2QafEkMhn8PV1rwREj8WP, address:79.127.219.41:4930): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002888009 (580h40m22.886842201s) [seg_pos={0 0}, upload=511155]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Feb 8 20:35:45.110856986 (580h40m22.994811201s) [upload=511155]: Done uploading segments...
Feb 8 20:35:45.110918692 (580h40m22.994872902s) [upload=511155]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.280386028s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
2.364072365s trace:
https://jaeger.dev.storj.io/trace/3c8a03916dd3efb0
FAIL (2026-02-08 21:13:51 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Feb 8 20:35:15.284031765 (580h39m53.167985964s) [upload=511155]: Starting upload
Feb 8 20:35:15.284112500 (580h39m53.168066701s) [upload=511155]: Done starting upload
Feb 8 20:35:15.284149599 (580h39m53.168103791s) [upload=511155]: Uploading segments...
Feb 8 20:35:15.284601793 (580h39m53.168555992s) [upload=511155]: Got next segment. Inline: false
Feb 8 20:35:15.284721869 (580h39m53.16867607s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 20:35:15.284728431 (580h39m53.16868263s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 20:35:15.397389353 (580h39m53.281343559s) [seg_pos={0 0}, upload=511155]: Begin upload segment...
Feb 8 20:35:15.397453134 (580h39m53.281407318s) [seg_pos={0 0}, upload=511155]: Done begin upload segment.
Feb 8 20:35:15.397517573 (580h39m53.281471782s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.397533666 (580h39m53.281487892s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398120348 (580h39m53.282074556s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398572052 (580h39m53.282526262s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399001740 (580h39m53.282955955s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399492847 (580h39m53.283447038s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399954482 (580h39m53.283908691s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.400483442 (580h39m53.284437649s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.402650352 (580h39m53.286604556s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.404412781 (580h39m53.288366984s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.406266549 (580h39m53.290220752s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.407902210 (580h39m53.291856418s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.409594415 (580h39m53.293548626s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.410340922 (580h39m53.294295132s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411911194 (580h39m53.2958654s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411928878 (580h39m53.29588307s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:16.207177374 (580h39m54.091131559s) [upload=511155]: Next returned nil segment
Feb 8 20:35:16.207191375 (580h39m54.091145561s) [upload=511155]: Waiting for error group managing segments...
Feb 8 20:35:16.334354958 (580h39m54.21830916s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.335358031 (580h39m54.21931223s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.337810738 (580h39m54.221764931s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.338164047 (580h39m54.222118238s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.342858708 (580h39m54.226812913s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.343378259 (580h39m54.227332458s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.345609935 (580h39m54.229564141s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.350042706 (580h39m54.233996915s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.379091195 (580h39m54.263045398s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.387202016 (580h39m54.271156202s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.390136524 (580h39m54.274090711s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:44.095174165 (580h40m21.979128365s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:45.002467024 (580h40m22.886421233s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:1JgPWqe81FwGvfduteWKs6QAmxY3Av3Nbd4RBu4fcZh5rH3RMM, address:79.127.219.41:4927): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002509190 (580h40m22.886463383s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002708091 (580h40m22.886662296s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:19MoR7r7wEgpnEpEesf4oJzFsWZmygg7mxg4jCtGf9rhJDw56B, address:45.91.116.4:4947): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002855562 (580h40m22.88680977s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12A39mM8Cc7tERvweTDoj4wkGgNTNn2QafEkMhn8PV1rwREj8WP, address:79.127.219.41:4930): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002888009 (580h40m22.886842201s) [seg_pos={0 0}, upload=511155]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Feb 8 20:35:45.110856986 (580h40m22.994811201s) [upload=511155]: Done uploading segments...
Feb 8 20:35:45.110918692 (580h40m22.994872902s) [upload=511155]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.364123176s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
2.337421721s trace:
https://jaeger.dev.storj.io/trace/3c8a03916dd3efb0
FAIL (2026-02-08 21:12:52 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Feb 8 20:35:15.284031765 (580h39m53.167985964s) [upload=511155]: Starting upload
Feb 8 20:35:15.284112500 (580h39m53.168066701s) [upload=511155]: Done starting upload
Feb 8 20:35:15.284149599 (580h39m53.168103791s) [upload=511155]: Uploading segments...
Feb 8 20:35:15.284601793 (580h39m53.168555992s) [upload=511155]: Got next segment. Inline: false
Feb 8 20:35:15.284721869 (580h39m53.16867607s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 20:35:15.284728431 (580h39m53.16868263s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 20:35:15.397389353 (580h39m53.281343559s) [seg_pos={0 0}, upload=511155]: Begin upload segment...
Feb 8 20:35:15.397453134 (580h39m53.281407318s) [seg_pos={0 0}, upload=511155]: Done begin upload segment.
Feb 8 20:35:15.397517573 (580h39m53.281471782s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.397533666 (580h39m53.281487892s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398120348 (580h39m53.282074556s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398572052 (580h39m53.282526262s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399001740 (580h39m53.282955955s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399492847 (580h39m53.283447038s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399954482 (580h39m53.283908691s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.400483442 (580h39m53.284437649s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.402650352 (580h39m53.286604556s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.404412781 (580h39m53.288366984s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.406266549 (580h39m53.290220752s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.407902210 (580h39m53.291856418s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.409594415 (580h39m53.293548626s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.410340922 (580h39m53.294295132s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411911194 (580h39m53.2958654s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411928878 (580h39m53.29588307s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:16.207177374 (580h39m54.091131559s) [upload=511155]: Next returned nil segment
Feb 8 20:35:16.207191375 (580h39m54.091145561s) [upload=511155]: Waiting for error group managing segments...
Feb 8 20:35:16.334354958 (580h39m54.21830916s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.335358031 (580h39m54.21931223s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.337810738 (580h39m54.221764931s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.338164047 (580h39m54.222118238s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.342858708 (580h39m54.226812913s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.343378259 (580h39m54.227332458s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.345609935 (580h39m54.229564141s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.350042706 (580h39m54.233996915s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.379091195 (580h39m54.263045398s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.387202016 (580h39m54.271156202s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.390136524 (580h39m54.274090711s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:44.095174165 (580h40m21.979128365s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:45.002467024 (580h40m22.886421233s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:1JgPWqe81FwGvfduteWKs6QAmxY3Av3Nbd4RBu4fcZh5rH3RMM, address:79.127.219.41:4927): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002509190 (580h40m22.886463383s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002708091 (580h40m22.886662296s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:19MoR7r7wEgpnEpEesf4oJzFsWZmygg7mxg4jCtGf9rhJDw56B, address:45.91.116.4:4947): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002855562 (580h40m22.88680977s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12A39mM8Cc7tERvweTDoj4wkGgNTNn2QafEkMhn8PV1rwREj8WP, address:79.127.219.41:4930): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002888009 (580h40m22.886842201s) [seg_pos={0 0}, upload=511155]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Feb 8 20:35:45.110856986 (580h40m22.994811201s) [upload=511155]: Done uploading segments...
Feb 8 20:35:45.110918692 (580h40m22.994872902s) [upload=511155]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.337454593s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
2.327422124s trace:
https://jaeger.dev.storj.io/trace/3c8a03916dd3efb0
FAIL (2026-02-08 21:12:24 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Feb 8 20:35:15.284031765 (580h39m53.167985964s) [upload=511155]: Starting upload
Feb 8 20:35:15.284112500 (580h39m53.168066701s) [upload=511155]: Done starting upload
Feb 8 20:35:15.284149599 (580h39m53.168103791s) [upload=511155]: Uploading segments...
Feb 8 20:35:15.284601793 (580h39m53.168555992s) [upload=511155]: Got next segment. Inline: false
Feb 8 20:35:15.284721869 (580h39m53.16867607s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 20:35:15.284728431 (580h39m53.16868263s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 20:35:15.397389353 (580h39m53.281343559s) [seg_pos={0 0}, upload=511155]: Begin upload segment...
Feb 8 20:35:15.397453134 (580h39m53.281407318s) [seg_pos={0 0}, upload=511155]: Done begin upload segment.
Feb 8 20:35:15.397517573 (580h39m53.281471782s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.397533666 (580h39m53.281487892s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398120348 (580h39m53.282074556s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398572052 (580h39m53.282526262s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399001740 (580h39m53.282955955s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399492847 (580h39m53.283447038s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399954482 (580h39m53.283908691s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.400483442 (580h39m53.284437649s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.402650352 (580h39m53.286604556s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.404412781 (580h39m53.288366984s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.406266549 (580h39m53.290220752s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.407902210 (580h39m53.291856418s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.409594415 (580h39m53.293548626s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.410340922 (580h39m53.294295132s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411911194 (580h39m53.2958654s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411928878 (580h39m53.29588307s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:16.207177374 (580h39m54.091131559s) [upload=511155]: Next returned nil segment
Feb 8 20:35:16.207191375 (580h39m54.091145561s) [upload=511155]: Waiting for error group managing segments...
Feb 8 20:35:16.334354958 (580h39m54.21830916s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.335358031 (580h39m54.21931223s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.337810738 (580h39m54.221764931s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.338164047 (580h39m54.222118238s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.342858708 (580h39m54.226812913s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.343378259 (580h39m54.227332458s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.345609935 (580h39m54.229564141s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.350042706 (580h39m54.233996915s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.379091195 (580h39m54.263045398s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.387202016 (580h39m54.271156202s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.390136524 (580h39m54.274090711s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:44.095174165 (580h40m21.979128365s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:45.002467024 (580h40m22.886421233s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:1JgPWqe81FwGvfduteWKs6QAmxY3Av3Nbd4RBu4fcZh5rH3RMM, address:79.127.219.41:4927): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002509190 (580h40m22.886463383s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002708091 (580h40m22.886662296s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:19MoR7r7wEgpnEpEesf4oJzFsWZmygg7mxg4jCtGf9rhJDw56B, address:45.91.116.4:4947): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002855562 (580h40m22.88680977s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12A39mM8Cc7tERvweTDoj4wkGgNTNn2QafEkMhn8PV1rwREj8WP, address:79.127.219.41:4930): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002888009 (580h40m22.886842201s) [seg_pos={0 0}, upload=511155]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Feb 8 20:35:45.110856986 (580h40m22.994811201s) [upload=511155]: Done uploading segments...
Feb 8 20:35:45.110918692 (580h40m22.994872902s) [upload=511155]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.327448271s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
2.396055452s trace:
https://jaeger.dev.storj.io/trace/3c8a03916dd3efb0
FAIL (2026-02-08 21:11:30 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Feb 8 20:35:15.284031765 (580h39m53.167985964s) [upload=511155]: Starting upload
Feb 8 20:35:15.284112500 (580h39m53.168066701s) [upload=511155]: Done starting upload
Feb 8 20:35:15.284149599 (580h39m53.168103791s) [upload=511155]: Uploading segments...
Feb 8 20:35:15.284601793 (580h39m53.168555992s) [upload=511155]: Got next segment. Inline: false
Feb 8 20:35:15.284721869 (580h39m53.16867607s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 20:35:15.284728431 (580h39m53.16868263s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 20:35:15.397389353 (580h39m53.281343559s) [seg_pos={0 0}, upload=511155]: Begin upload segment...
Feb 8 20:35:15.397453134 (580h39m53.281407318s) [seg_pos={0 0}, upload=511155]: Done begin upload segment.
Feb 8 20:35:15.397517573 (580h39m53.281471782s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.397533666 (580h39m53.281487892s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398120348 (580h39m53.282074556s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398572052 (580h39m53.282526262s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399001740 (580h39m53.282955955s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399492847 (580h39m53.283447038s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399954482 (580h39m53.283908691s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.400483442 (580h39m53.284437649s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.402650352 (580h39m53.286604556s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.404412781 (580h39m53.288366984s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.406266549 (580h39m53.290220752s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.407902210 (580h39m53.291856418s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.409594415 (580h39m53.293548626s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.410340922 (580h39m53.294295132s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411911194 (580h39m53.2958654s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411928878 (580h39m53.29588307s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:16.207177374 (580h39m54.091131559s) [upload=511155]: Next returned nil segment
Feb 8 20:35:16.207191375 (580h39m54.091145561s) [upload=511155]: Waiting for error group managing segments...
Feb 8 20:35:16.334354958 (580h39m54.21830916s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.335358031 (580h39m54.21931223s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.337810738 (580h39m54.221764931s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.338164047 (580h39m54.222118238s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.342858708 (580h39m54.226812913s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.343378259 (580h39m54.227332458s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.345609935 (580h39m54.229564141s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.350042706 (580h39m54.233996915s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.379091195 (580h39m54.263045398s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.387202016 (580h39m54.271156202s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.390136524 (580h39m54.274090711s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:44.095174165 (580h40m21.979128365s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:45.002467024 (580h40m22.886421233s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:1JgPWqe81FwGvfduteWKs6QAmxY3Av3Nbd4RBu4fcZh5rH3RMM, address:79.127.219.41:4927): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002509190 (580h40m22.886463383s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002708091 (580h40m22.886662296s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:19MoR7r7wEgpnEpEesf4oJzFsWZmygg7mxg4jCtGf9rhJDw56B, address:45.91.116.4:4947): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002855562 (580h40m22.88680977s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12A39mM8Cc7tERvweTDoj4wkGgNTNn2QafEkMhn8PV1rwREj8WP, address:79.127.219.41:4930): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002888009 (580h40m22.886842201s) [seg_pos={0 0}, upload=511155]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Feb 8 20:35:45.110856986 (580h40m22.994811201s) [upload=511155]: Done uploading segments...
Feb 8 20:35:45.110918692 (580h40m22.994872902s) [upload=511155]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.396080777s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
2.348908208s trace:
https://jaeger.dev.storj.io/trace/3c8a03916dd3efb0
FAIL (2026-02-08 21:10:43 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Feb 8 20:35:15.284031765 (580h39m53.167985964s) [upload=511155]: Starting upload
Feb 8 20:35:15.284112500 (580h39m53.168066701s) [upload=511155]: Done starting upload
Feb 8 20:35:15.284149599 (580h39m53.168103791s) [upload=511155]: Uploading segments...
Feb 8 20:35:15.284601793 (580h39m53.168555992s) [upload=511155]: Got next segment. Inline: false
Feb 8 20:35:15.284721869 (580h39m53.16867607s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 20:35:15.284728431 (580h39m53.16868263s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 20:35:15.397389353 (580h39m53.281343559s) [seg_pos={0 0}, upload=511155]: Begin upload segment...
Feb 8 20:35:15.397453134 (580h39m53.281407318s) [seg_pos={0 0}, upload=511155]: Done begin upload segment.
Feb 8 20:35:15.397517573 (580h39m53.281471782s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.397533666 (580h39m53.281487892s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398120348 (580h39m53.282074556s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398572052 (580h39m53.282526262s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399001740 (580h39m53.282955955s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399492847 (580h39m53.283447038s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399954482 (580h39m53.283908691s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.400483442 (580h39m53.284437649s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.402650352 (580h39m53.286604556s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.404412781 (580h39m53.288366984s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.406266549 (580h39m53.290220752s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.407902210 (580h39m53.291856418s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.409594415 (580h39m53.293548626s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.410340922 (580h39m53.294295132s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411911194 (580h39m53.2958654s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411928878 (580h39m53.29588307s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:16.207177374 (580h39m54.091131559s) [upload=511155]: Next returned nil segment
Feb 8 20:35:16.207191375 (580h39m54.091145561s) [upload=511155]: Waiting for error group managing segments...
Feb 8 20:35:16.334354958 (580h39m54.21830916s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.335358031 (580h39m54.21931223s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.337810738 (580h39m54.221764931s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.338164047 (580h39m54.222118238s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.342858708 (580h39m54.226812913s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.343378259 (580h39m54.227332458s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.345609935 (580h39m54.229564141s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.350042706 (580h39m54.233996915s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.379091195 (580h39m54.263045398s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.387202016 (580h39m54.271156202s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.390136524 (580h39m54.274090711s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:44.095174165 (580h40m21.979128365s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:45.002467024 (580h40m22.886421233s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:1JgPWqe81FwGvfduteWKs6QAmxY3Av3Nbd4RBu4fcZh5rH3RMM, address:79.127.219.41:4927): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002509190 (580h40m22.886463383s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002708091 (580h40m22.886662296s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:19MoR7r7wEgpnEpEesf4oJzFsWZmygg7mxg4jCtGf9rhJDw56B, address:45.91.116.4:4947): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002855562 (580h40m22.88680977s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12A39mM8Cc7tERvweTDoj4wkGgNTNn2QafEkMhn8PV1rwREj8WP, address:79.127.219.41:4930): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002888009 (580h40m22.886842201s) [seg_pos={0 0}, upload=511155]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Feb 8 20:35:45.110856986 (580h40m22.994811201s) [upload=511155]: Done uploading segments...
Feb 8 20:35:45.110918692 (580h40m22.994872902s) [upload=511155]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.348942749s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
2.392976894s trace:
https://jaeger.dev.storj.io/trace/3c8a03916dd3efb0
FAIL (2026-02-08 21:10:11 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Feb 8 20:35:15.284031765 (580h39m53.167985964s) [upload=511155]: Starting upload
Feb 8 20:35:15.284112500 (580h39m53.168066701s) [upload=511155]: Done starting upload
Feb 8 20:35:15.284149599 (580h39m53.168103791s) [upload=511155]: Uploading segments...
Feb 8 20:35:15.284601793 (580h39m53.168555992s) [upload=511155]: Got next segment. Inline: false
Feb 8 20:35:15.284721869 (580h39m53.16867607s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 20:35:15.284728431 (580h39m53.16868263s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 20:35:15.397389353 (580h39m53.281343559s) [seg_pos={0 0}, upload=511155]: Begin upload segment...
Feb 8 20:35:15.397453134 (580h39m53.281407318s) [seg_pos={0 0}, upload=511155]: Done begin upload segment.
Feb 8 20:35:15.397517573 (580h39m53.281471782s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.397533666 (580h39m53.281487892s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398120348 (580h39m53.282074556s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398572052 (580h39m53.282526262s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399001740 (580h39m53.282955955s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399492847 (580h39m53.283447038s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399954482 (580h39m53.283908691s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.400483442 (580h39m53.284437649s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.402650352 (580h39m53.286604556s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.404412781 (580h39m53.288366984s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.406266549 (580h39m53.290220752s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.407902210 (580h39m53.291856418s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.409594415 (580h39m53.293548626s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.410340922 (580h39m53.294295132s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411911194 (580h39m53.2958654s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411928878 (580h39m53.29588307s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:16.207177374 (580h39m54.091131559s) [upload=511155]: Next returned nil segment
Feb 8 20:35:16.207191375 (580h39m54.091145561s) [upload=511155]: Waiting for error group managing segments...
Feb 8 20:35:16.334354958 (580h39m54.21830916s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.335358031 (580h39m54.21931223s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.337810738 (580h39m54.221764931s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.338164047 (580h39m54.222118238s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.342858708 (580h39m54.226812913s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.343378259 (580h39m54.227332458s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.345609935 (580h39m54.229564141s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.350042706 (580h39m54.233996915s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.379091195 (580h39m54.263045398s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.387202016 (580h39m54.271156202s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.390136524 (580h39m54.274090711s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:44.095174165 (580h40m21.979128365s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:45.002467024 (580h40m22.886421233s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:1JgPWqe81FwGvfduteWKs6QAmxY3Av3Nbd4RBu4fcZh5rH3RMM, address:79.127.219.41:4927): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002509190 (580h40m22.886463383s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002708091 (580h40m22.886662296s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:19MoR7r7wEgpnEpEesf4oJzFsWZmygg7mxg4jCtGf9rhJDw56B, address:45.91.116.4:4947): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002855562 (580h40m22.88680977s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12A39mM8Cc7tERvweTDoj4wkGgNTNn2QafEkMhn8PV1rwREj8WP, address:79.127.219.41:4930): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002888009 (580h40m22.886842201s) [seg_pos={0 0}, upload=511155]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Feb 8 20:35:45.110856986 (580h40m22.994811201s) [upload=511155]: Done uploading segments...
Feb 8 20:35:45.110918692 (580h40m22.994872902s) [upload=511155]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.393020333s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
2.425305974s trace:
https://jaeger.dev.storj.io/trace/3c8a03916dd3efb0
FAIL (2026-02-08 21:09:54 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Feb 8 20:35:15.284031765 (580h39m53.167985964s) [upload=511155]: Starting upload
Feb 8 20:35:15.284112500 (580h39m53.168066701s) [upload=511155]: Done starting upload
Feb 8 20:35:15.284149599 (580h39m53.168103791s) [upload=511155]: Uploading segments...
Feb 8 20:35:15.284601793 (580h39m53.168555992s) [upload=511155]: Got next segment. Inline: false
Feb 8 20:35:15.284721869 (580h39m53.16867607s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 20:35:15.284728431 (580h39m53.16868263s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 20:35:15.397389353 (580h39m53.281343559s) [seg_pos={0 0}, upload=511155]: Begin upload segment...
Feb 8 20:35:15.397453134 (580h39m53.281407318s) [seg_pos={0 0}, upload=511155]: Done begin upload segment.
Feb 8 20:35:15.397517573 (580h39m53.281471782s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.397533666 (580h39m53.281487892s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398120348 (580h39m53.282074556s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398572052 (580h39m53.282526262s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399001740 (580h39m53.282955955s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399492847 (580h39m53.283447038s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399954482 (580h39m53.283908691s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.400483442 (580h39m53.284437649s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.402650352 (580h39m53.286604556s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.404412781 (580h39m53.288366984s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.406266549 (580h39m53.290220752s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.407902210 (580h39m53.291856418s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.409594415 (580h39m53.293548626s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.410340922 (580h39m53.294295132s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411911194 (580h39m53.2958654s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411928878 (580h39m53.29588307s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:16.207177374 (580h39m54.091131559s) [upload=511155]: Next returned nil segment
Feb 8 20:35:16.207191375 (580h39m54.091145561s) [upload=511155]: Waiting for error group managing segments...
Feb 8 20:35:16.334354958 (580h39m54.21830916s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.335358031 (580h39m54.21931223s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.337810738 (580h39m54.221764931s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.338164047 (580h39m54.222118238s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.342858708 (580h39m54.226812913s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.343378259 (580h39m54.227332458s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.345609935 (580h39m54.229564141s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.350042706 (580h39m54.233996915s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.379091195 (580h39m54.263045398s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.387202016 (580h39m54.271156202s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.390136524 (580h39m54.274090711s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:44.095174165 (580h40m21.979128365s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:45.002467024 (580h40m22.886421233s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:1JgPWqe81FwGvfduteWKs6QAmxY3Av3Nbd4RBu4fcZh5rH3RMM, address:79.127.219.41:4927): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002509190 (580h40m22.886463383s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002708091 (580h40m22.886662296s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:19MoR7r7wEgpnEpEesf4oJzFsWZmygg7mxg4jCtGf9rhJDw56B, address:45.91.116.4:4947): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002855562 (580h40m22.88680977s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12A39mM8Cc7tERvweTDoj4wkGgNTNn2QafEkMhn8PV1rwREj8WP, address:79.127.219.41:4930): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002888009 (580h40m22.886842201s) [seg_pos={0 0}, upload=511155]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Feb 8 20:35:45.110856986 (580h40m22.994811201s) [upload=511155]: Done uploading segments...
Feb 8 20:35:45.110918692 (580h40m22.994872902s) [upload=511155]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.425355383s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
2.368731797s trace:
https://jaeger.dev.storj.io/trace/3c8a03916dd3efb0
FAIL (2026-02-08 21:09:15 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Feb 8 20:35:15.284031765 (580h39m53.167985964s) [upload=511155]: Starting upload
Feb 8 20:35:15.284112500 (580h39m53.168066701s) [upload=511155]: Done starting upload
Feb 8 20:35:15.284149599 (580h39m53.168103791s) [upload=511155]: Uploading segments...
Feb 8 20:35:15.284601793 (580h39m53.168555992s) [upload=511155]: Got next segment. Inline: false
Feb 8 20:35:15.284721869 (580h39m53.16867607s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 20:35:15.284728431 (580h39m53.16868263s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 20:35:15.397389353 (580h39m53.281343559s) [seg_pos={0 0}, upload=511155]: Begin upload segment...
Feb 8 20:35:15.397453134 (580h39m53.281407318s) [seg_pos={0 0}, upload=511155]: Done begin upload segment.
Feb 8 20:35:15.397517573 (580h39m53.281471782s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.397533666 (580h39m53.281487892s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398120348 (580h39m53.282074556s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398572052 (580h39m53.282526262s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399001740 (580h39m53.282955955s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399492847 (580h39m53.283447038s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399954482 (580h39m53.283908691s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.400483442 (580h39m53.284437649s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.402650352 (580h39m53.286604556s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.404412781 (580h39m53.288366984s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.406266549 (580h39m53.290220752s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.407902210 (580h39m53.291856418s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.409594415 (580h39m53.293548626s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.410340922 (580h39m53.294295132s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411911194 (580h39m53.2958654s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411928878 (580h39m53.29588307s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:16.207177374 (580h39m54.091131559s) [upload=511155]: Next returned nil segment
Feb 8 20:35:16.207191375 (580h39m54.091145561s) [upload=511155]: Waiting for error group managing segments...
Feb 8 20:35:16.334354958 (580h39m54.21830916s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.335358031 (580h39m54.21931223s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.337810738 (580h39m54.221764931s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.338164047 (580h39m54.222118238s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.342858708 (580h39m54.226812913s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.343378259 (580h39m54.227332458s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.345609935 (580h39m54.229564141s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.350042706 (580h39m54.233996915s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.379091195 (580h39m54.263045398s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.387202016 (580h39m54.271156202s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.390136524 (580h39m54.274090711s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:44.095174165 (580h40m21.979128365s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:45.002467024 (580h40m22.886421233s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:1JgPWqe81FwGvfduteWKs6QAmxY3Av3Nbd4RBu4fcZh5rH3RMM, address:79.127.219.41:4927): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002509190 (580h40m22.886463383s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002708091 (580h40m22.886662296s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:19MoR7r7wEgpnEpEesf4oJzFsWZmygg7mxg4jCtGf9rhJDw56B, address:45.91.116.4:4947): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002855562 (580h40m22.88680977s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12A39mM8Cc7tERvweTDoj4wkGgNTNn2QafEkMhn8PV1rwREj8WP, address:79.127.219.41:4930): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002888009 (580h40m22.886842201s) [seg_pos={0 0}, upload=511155]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Feb 8 20:35:45.110856986 (580h40m22.994811201s) [upload=511155]: Done uploading segments...
Feb 8 20:35:45.110918692 (580h40m22.994872902s) [upload=511155]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.368824043s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
2.356989383s trace:
https://jaeger.dev.storj.io/trace/3c8a03916dd3efb0
FAIL (2026-02-08 21:08:50 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Feb 8 20:35:15.284031765 (580h39m53.167985964s) [upload=511155]: Starting upload
Feb 8 20:35:15.284112500 (580h39m53.168066701s) [upload=511155]: Done starting upload
Feb 8 20:35:15.284149599 (580h39m53.168103791s) [upload=511155]: Uploading segments...
Feb 8 20:35:15.284601793 (580h39m53.168555992s) [upload=511155]: Got next segment. Inline: false
Feb 8 20:35:15.284721869 (580h39m53.16867607s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 20:35:15.284728431 (580h39m53.16868263s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 20:35:15.397389353 (580h39m53.281343559s) [seg_pos={0 0}, upload=511155]: Begin upload segment...
Feb 8 20:35:15.397453134 (580h39m53.281407318s) [seg_pos={0 0}, upload=511155]: Done begin upload segment.
Feb 8 20:35:15.397517573 (580h39m53.281471782s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.397533666 (580h39m53.281487892s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398120348 (580h39m53.282074556s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398572052 (580h39m53.282526262s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399001740 (580h39m53.282955955s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399492847 (580h39m53.283447038s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399954482 (580h39m53.283908691s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.400483442 (580h39m53.284437649s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.402650352 (580h39m53.286604556s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.404412781 (580h39m53.288366984s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.406266549 (580h39m53.290220752s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.407902210 (580h39m53.291856418s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.409594415 (580h39m53.293548626s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.410340922 (580h39m53.294295132s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411911194 (580h39m53.2958654s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411928878 (580h39m53.29588307s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:16.207177374 (580h39m54.091131559s) [upload=511155]: Next returned nil segment
Feb 8 20:35:16.207191375 (580h39m54.091145561s) [upload=511155]: Waiting for error group managing segments...
Feb 8 20:35:16.334354958 (580h39m54.21830916s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.335358031 (580h39m54.21931223s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.337810738 (580h39m54.221764931s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.338164047 (580h39m54.222118238s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.342858708 (580h39m54.226812913s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.343378259 (580h39m54.227332458s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.345609935 (580h39m54.229564141s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.350042706 (580h39m54.233996915s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.379091195 (580h39m54.263045398s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.387202016 (580h39m54.271156202s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.390136524 (580h39m54.274090711s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:44.095174165 (580h40m21.979128365s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:45.002467024 (580h40m22.886421233s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:1JgPWqe81FwGvfduteWKs6QAmxY3Av3Nbd4RBu4fcZh5rH3RMM, address:79.127.219.41:4927): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002509190 (580h40m22.886463383s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002708091 (580h40m22.886662296s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:19MoR7r7wEgpnEpEesf4oJzFsWZmygg7mxg4jCtGf9rhJDw56B, address:45.91.116.4:4947): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002855562 (580h40m22.88680977s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12A39mM8Cc7tERvweTDoj4wkGgNTNn2QafEkMhn8PV1rwREj8WP, address:79.127.219.41:4930): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002888009 (580h40m22.886842201s) [seg_pos={0 0}, upload=511155]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Feb 8 20:35:45.110856986 (580h40m22.994811201s) [upload=511155]: Done uploading segments...
Feb 8 20:35:45.110918692 (580h40m22.994872902s) [upload=511155]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.357017156s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
2.353682624s trace:
https://jaeger.dev.storj.io/trace/3c8a03916dd3efb0
FAIL (2026-02-08 21:08:14 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Feb 8 20:35:15.284031765 (580h39m53.167985964s) [upload=511155]: Starting upload
Feb 8 20:35:15.284112500 (580h39m53.168066701s) [upload=511155]: Done starting upload
Feb 8 20:35:15.284149599 (580h39m53.168103791s) [upload=511155]: Uploading segments...
Feb 8 20:35:15.284601793 (580h39m53.168555992s) [upload=511155]: Got next segment. Inline: false
Feb 8 20:35:15.284721869 (580h39m53.16867607s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 20:35:15.284728431 (580h39m53.16868263s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 20:35:15.397389353 (580h39m53.281343559s) [seg_pos={0 0}, upload=511155]: Begin upload segment...
Feb 8 20:35:15.397453134 (580h39m53.281407318s) [seg_pos={0 0}, upload=511155]: Done begin upload segment.
Feb 8 20:35:15.397517573 (580h39m53.281471782s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.397533666 (580h39m53.281487892s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398120348 (580h39m53.282074556s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398572052 (580h39m53.282526262s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399001740 (580h39m53.282955955s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399492847 (580h39m53.283447038s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399954482 (580h39m53.283908691s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.400483442 (580h39m53.284437649s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.402650352 (580h39m53.286604556s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.404412781 (580h39m53.288366984s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.406266549 (580h39m53.290220752s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.407902210 (580h39m53.291856418s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.409594415 (580h39m53.293548626s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.410340922 (580h39m53.294295132s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411911194 (580h39m53.2958654s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411928878 (580h39m53.29588307s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:16.207177374 (580h39m54.091131559s) [upload=511155]: Next returned nil segment
Feb 8 20:35:16.207191375 (580h39m54.091145561s) [upload=511155]: Waiting for error group managing segments...
Feb 8 20:35:16.334354958 (580h39m54.21830916s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.335358031 (580h39m54.21931223s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.337810738 (580h39m54.221764931s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.338164047 (580h39m54.222118238s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.342858708 (580h39m54.226812913s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.343378259 (580h39m54.227332458s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.345609935 (580h39m54.229564141s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.350042706 (580h39m54.233996915s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.379091195 (580h39m54.263045398s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.387202016 (580h39m54.271156202s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.390136524 (580h39m54.274090711s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:44.095174165 (580h40m21.979128365s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:45.002467024 (580h40m22.886421233s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:1JgPWqe81FwGvfduteWKs6QAmxY3Av3Nbd4RBu4fcZh5rH3RMM, address:79.127.219.41:4927): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002509190 (580h40m22.886463383s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002708091 (580h40m22.886662296s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:19MoR7r7wEgpnEpEesf4oJzFsWZmygg7mxg4jCtGf9rhJDw56B, address:45.91.116.4:4947): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002855562 (580h40m22.88680977s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12A39mM8Cc7tERvweTDoj4wkGgNTNn2QafEkMhn8PV1rwREj8WP, address:79.127.219.41:4930): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002888009 (580h40m22.886842201s) [seg_pos={0 0}, upload=511155]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Feb 8 20:35:45.110856986 (580h40m22.994811201s) [upload=511155]: Done uploading segments...
Feb 8 20:35:45.110918692 (580h40m22.994872902s) [upload=511155]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.353717601s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
2.404237438s trace:
https://jaeger.dev.storj.io/trace/3c8a03916dd3efb0
FAIL (2026-02-08 21:07:26 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Feb 8 20:35:15.284031765 (580h39m53.167985964s) [upload=511155]: Starting upload
Feb 8 20:35:15.284112500 (580h39m53.168066701s) [upload=511155]: Done starting upload
Feb 8 20:35:15.284149599 (580h39m53.168103791s) [upload=511155]: Uploading segments...
Feb 8 20:35:15.284601793 (580h39m53.168555992s) [upload=511155]: Got next segment. Inline: false
Feb 8 20:35:15.284721869 (580h39m53.16867607s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 20:35:15.284728431 (580h39m53.16868263s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 20:35:15.397389353 (580h39m53.281343559s) [seg_pos={0 0}, upload=511155]: Begin upload segment...
Feb 8 20:35:15.397453134 (580h39m53.281407318s) [seg_pos={0 0}, upload=511155]: Done begin upload segment.
Feb 8 20:35:15.397517573 (580h39m53.281471782s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.397533666 (580h39m53.281487892s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398120348 (580h39m53.282074556s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398572052 (580h39m53.282526262s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399001740 (580h39m53.282955955s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399492847 (580h39m53.283447038s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399954482 (580h39m53.283908691s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.400483442 (580h39m53.284437649s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.402650352 (580h39m53.286604556s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.404412781 (580h39m53.288366984s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.406266549 (580h39m53.290220752s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.407902210 (580h39m53.291856418s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.409594415 (580h39m53.293548626s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.410340922 (580h39m53.294295132s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411911194 (580h39m53.2958654s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411928878 (580h39m53.29588307s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:16.207177374 (580h39m54.091131559s) [upload=511155]: Next returned nil segment
Feb 8 20:35:16.207191375 (580h39m54.091145561s) [upload=511155]: Waiting for error group managing segments...
Feb 8 20:35:16.334354958 (580h39m54.21830916s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.335358031 (580h39m54.21931223s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.337810738 (580h39m54.221764931s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.338164047 (580h39m54.222118238s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.342858708 (580h39m54.226812913s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.343378259 (580h39m54.227332458s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.345609935 (580h39m54.229564141s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.350042706 (580h39m54.233996915s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.379091195 (580h39m54.263045398s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.387202016 (580h39m54.271156202s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.390136524 (580h39m54.274090711s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:44.095174165 (580h40m21.979128365s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:45.002467024 (580h40m22.886421233s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:1JgPWqe81FwGvfduteWKs6QAmxY3Av3Nbd4RBu4fcZh5rH3RMM, address:79.127.219.41:4927): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002509190 (580h40m22.886463383s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002708091 (580h40m22.886662296s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:19MoR7r7wEgpnEpEesf4oJzFsWZmygg7mxg4jCtGf9rhJDw56B, address:45.91.116.4:4947): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002855562 (580h40m22.88680977s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12A39mM8Cc7tERvweTDoj4wkGgNTNn2QafEkMhn8PV1rwREj8WP, address:79.127.219.41:4930): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002888009 (580h40m22.886842201s) [seg_pos={0 0}, upload=511155]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Feb 8 20:35:45.110856986 (580h40m22.994811201s) [upload=511155]: Done uploading segments...
Feb 8 20:35:45.110918692 (580h40m22.994872902s) [upload=511155]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.404304525s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
2.358379953s trace:
https://jaeger.dev.storj.io/trace/3c8a03916dd3efb0
FAIL (2026-02-08 21:06:18 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Feb 8 20:35:15.284031765 (580h39m53.167985964s) [upload=511155]: Starting upload
Feb 8 20:35:15.284112500 (580h39m53.168066701s) [upload=511155]: Done starting upload
Feb 8 20:35:15.284149599 (580h39m53.168103791s) [upload=511155]: Uploading segments...
Feb 8 20:35:15.284601793 (580h39m53.168555992s) [upload=511155]: Got next segment. Inline: false
Feb 8 20:35:15.284721869 (580h39m53.16867607s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 20:35:15.284728431 (580h39m53.16868263s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 20:35:15.397389353 (580h39m53.281343559s) [seg_pos={0 0}, upload=511155]: Begin upload segment...
Feb 8 20:35:15.397453134 (580h39m53.281407318s) [seg_pos={0 0}, upload=511155]: Done begin upload segment.
Feb 8 20:35:15.397517573 (580h39m53.281471782s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.397533666 (580h39m53.281487892s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398120348 (580h39m53.282074556s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398572052 (580h39m53.282526262s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399001740 (580h39m53.282955955s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399492847 (580h39m53.283447038s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399954482 (580h39m53.283908691s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.400483442 (580h39m53.284437649s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.402650352 (580h39m53.286604556s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.404412781 (580h39m53.288366984s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.406266549 (580h39m53.290220752s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.407902210 (580h39m53.291856418s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.409594415 (580h39m53.293548626s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.410340922 (580h39m53.294295132s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411911194 (580h39m53.2958654s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411928878 (580h39m53.29588307s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:16.207177374 (580h39m54.091131559s) [upload=511155]: Next returned nil segment
Feb 8 20:35:16.207191375 (580h39m54.091145561s) [upload=511155]: Waiting for error group managing segments...
Feb 8 20:35:16.334354958 (580h39m54.21830916s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.335358031 (580h39m54.21931223s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.337810738 (580h39m54.221764931s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.338164047 (580h39m54.222118238s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.342858708 (580h39m54.226812913s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.343378259 (580h39m54.227332458s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.345609935 (580h39m54.229564141s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.350042706 (580h39m54.233996915s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.379091195 (580h39m54.263045398s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.387202016 (580h39m54.271156202s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.390136524 (580h39m54.274090711s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:44.095174165 (580h40m21.979128365s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:45.002467024 (580h40m22.886421233s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:1JgPWqe81FwGvfduteWKs6QAmxY3Av3Nbd4RBu4fcZh5rH3RMM, address:79.127.219.41:4927): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002509190 (580h40m22.886463383s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002708091 (580h40m22.886662296s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:19MoR7r7wEgpnEpEesf4oJzFsWZmygg7mxg4jCtGf9rhJDw56B, address:45.91.116.4:4947): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002855562 (580h40m22.88680977s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12A39mM8Cc7tERvweTDoj4wkGgNTNn2QafEkMhn8PV1rwREj8WP, address:79.127.219.41:4930): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002888009 (580h40m22.886842201s) [seg_pos={0 0}, upload=511155]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Feb 8 20:35:45.110856986 (580h40m22.994811201s) [upload=511155]: Done uploading segments...
Feb 8 20:35:45.110918692 (580h40m22.994872902s) [upload=511155]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.358409103s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
2.409317323s trace:
https://jaeger.dev.storj.io/trace/3c8a03916dd3efb0
FAIL (2026-02-08 21:05:42 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Feb 8 20:35:15.284031765 (580h39m53.167985964s) [upload=511155]: Starting upload
Feb 8 20:35:15.284112500 (580h39m53.168066701s) [upload=511155]: Done starting upload
Feb 8 20:35:15.284149599 (580h39m53.168103791s) [upload=511155]: Uploading segments...
Feb 8 20:35:15.284601793 (580h39m53.168555992s) [upload=511155]: Got next segment. Inline: false
Feb 8 20:35:15.284721869 (580h39m53.16867607s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 20:35:15.284728431 (580h39m53.16868263s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 20:35:15.397389353 (580h39m53.281343559s) [seg_pos={0 0}, upload=511155]: Begin upload segment...
Feb 8 20:35:15.397453134 (580h39m53.281407318s) [seg_pos={0 0}, upload=511155]: Done begin upload segment.
Feb 8 20:35:15.397517573 (580h39m53.281471782s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.397533666 (580h39m53.281487892s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398120348 (580h39m53.282074556s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398572052 (580h39m53.282526262s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399001740 (580h39m53.282955955s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399492847 (580h39m53.283447038s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399954482 (580h39m53.283908691s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.400483442 (580h39m53.284437649s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.402650352 (580h39m53.286604556s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.404412781 (580h39m53.288366984s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.406266549 (580h39m53.290220752s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.407902210 (580h39m53.291856418s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.409594415 (580h39m53.293548626s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.410340922 (580h39m53.294295132s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411911194 (580h39m53.2958654s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411928878 (580h39m53.29588307s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:16.207177374 (580h39m54.091131559s) [upload=511155]: Next returned nil segment
Feb 8 20:35:16.207191375 (580h39m54.091145561s) [upload=511155]: Waiting for error group managing segments...
Feb 8 20:35:16.334354958 (580h39m54.21830916s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.335358031 (580h39m54.21931223s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.337810738 (580h39m54.221764931s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.338164047 (580h39m54.222118238s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.342858708 (580h39m54.226812913s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.343378259 (580h39m54.227332458s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.345609935 (580h39m54.229564141s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.350042706 (580h39m54.233996915s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.379091195 (580h39m54.263045398s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.387202016 (580h39m54.271156202s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.390136524 (580h39m54.274090711s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:44.095174165 (580h40m21.979128365s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:45.002467024 (580h40m22.886421233s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:1JgPWqe81FwGvfduteWKs6QAmxY3Av3Nbd4RBu4fcZh5rH3RMM, address:79.127.219.41:4927): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002509190 (580h40m22.886463383s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002708091 (580h40m22.886662296s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:19MoR7r7wEgpnEpEesf4oJzFsWZmygg7mxg4jCtGf9rhJDw56B, address:45.91.116.4:4947): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002855562 (580h40m22.88680977s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12A39mM8Cc7tERvweTDoj4wkGgNTNn2QafEkMhn8PV1rwREj8WP, address:79.127.219.41:4930): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002888009 (580h40m22.886842201s) [seg_pos={0 0}, upload=511155]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Feb 8 20:35:45.110856986 (580h40m22.994811201s) [upload=511155]: Done uploading segments...
Feb 8 20:35:45.110918692 (580h40m22.994872902s) [upload=511155]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.40939858s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
2.4572665s trace:
https://jaeger.dev.storj.io/trace/3c8a03916dd3efb0
FAIL (2026-02-08 21:05:05 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Feb 8 20:35:15.284031765 (580h39m53.167985964s) [upload=511155]: Starting upload
Feb 8 20:35:15.284112500 (580h39m53.168066701s) [upload=511155]: Done starting upload
Feb 8 20:35:15.284149599 (580h39m53.168103791s) [upload=511155]: Uploading segments...
Feb 8 20:35:15.284601793 (580h39m53.168555992s) [upload=511155]: Got next segment. Inline: false
Feb 8 20:35:15.284721869 (580h39m53.16867607s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 20:35:15.284728431 (580h39m53.16868263s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 20:35:15.397389353 (580h39m53.281343559s) [seg_pos={0 0}, upload=511155]: Begin upload segment...
Feb 8 20:35:15.397453134 (580h39m53.281407318s) [seg_pos={0 0}, upload=511155]: Done begin upload segment.
Feb 8 20:35:15.397517573 (580h39m53.281471782s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.397533666 (580h39m53.281487892s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398120348 (580h39m53.282074556s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398572052 (580h39m53.282526262s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399001740 (580h39m53.282955955s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399492847 (580h39m53.283447038s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399954482 (580h39m53.283908691s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.400483442 (580h39m53.284437649s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.402650352 (580h39m53.286604556s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.404412781 (580h39m53.288366984s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.406266549 (580h39m53.290220752s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.407902210 (580h39m53.291856418s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.409594415 (580h39m53.293548626s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.410340922 (580h39m53.294295132s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411911194 (580h39m53.2958654s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411928878 (580h39m53.29588307s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:16.207177374 (580h39m54.091131559s) [upload=511155]: Next returned nil segment
Feb 8 20:35:16.207191375 (580h39m54.091145561s) [upload=511155]: Waiting for error group managing segments...
Feb 8 20:35:16.334354958 (580h39m54.21830916s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.335358031 (580h39m54.21931223s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.337810738 (580h39m54.221764931s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.338164047 (580h39m54.222118238s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.342858708 (580h39m54.226812913s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.343378259 (580h39m54.227332458s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.345609935 (580h39m54.229564141s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.350042706 (580h39m54.233996915s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.379091195 (580h39m54.263045398s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.387202016 (580h39m54.271156202s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.390136524 (580h39m54.274090711s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:44.095174165 (580h40m21.979128365s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:45.002467024 (580h40m22.886421233s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:1JgPWqe81FwGvfduteWKs6QAmxY3Av3Nbd4RBu4fcZh5rH3RMM, address:79.127.219.41:4927): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002509190 (580h40m22.886463383s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002708091 (580h40m22.886662296s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:19MoR7r7wEgpnEpEesf4oJzFsWZmygg7mxg4jCtGf9rhJDw56B, address:45.91.116.4:4947): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002855562 (580h40m22.88680977s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12A39mM8Cc7tERvweTDoj4wkGgNTNn2QafEkMhn8PV1rwREj8WP, address:79.127.219.41:4930): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002888009 (580h40m22.886842201s) [seg_pos={0 0}, upload=511155]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Feb 8 20:35:45.110856986 (580h40m22.994811201s) [upload=511155]: Done uploading segments...
Feb 8 20:35:45.110918692 (580h40m22.994872902s) [upload=511155]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.457333336s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
2.33931223s trace:
https://jaeger.dev.storj.io/trace/3c8a03916dd3efb0
FAIL (2026-02-08 21:04:34 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Feb 8 20:35:15.284031765 (580h39m53.167985964s) [upload=511155]: Starting upload
Feb 8 20:35:15.284112500 (580h39m53.168066701s) [upload=511155]: Done starting upload
Feb 8 20:35:15.284149599 (580h39m53.168103791s) [upload=511155]: Uploading segments...
Feb 8 20:35:15.284601793 (580h39m53.168555992s) [upload=511155]: Got next segment. Inline: false
Feb 8 20:35:15.284721869 (580h39m53.16867607s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 20:35:15.284728431 (580h39m53.16868263s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 20:35:15.397389353 (580h39m53.281343559s) [seg_pos={0 0}, upload=511155]: Begin upload segment...
Feb 8 20:35:15.397453134 (580h39m53.281407318s) [seg_pos={0 0}, upload=511155]: Done begin upload segment.
Feb 8 20:35:15.397517573 (580h39m53.281471782s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.397533666 (580h39m53.281487892s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398120348 (580h39m53.282074556s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398572052 (580h39m53.282526262s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399001740 (580h39m53.282955955s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399492847 (580h39m53.283447038s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399954482 (580h39m53.283908691s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.400483442 (580h39m53.284437649s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.402650352 (580h39m53.286604556s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.404412781 (580h39m53.288366984s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.406266549 (580h39m53.290220752s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.407902210 (580h39m53.291856418s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.409594415 (580h39m53.293548626s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.410340922 (580h39m53.294295132s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411911194 (580h39m53.2958654s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411928878 (580h39m53.29588307s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:16.207177374 (580h39m54.091131559s) [upload=511155]: Next returned nil segment
Feb 8 20:35:16.207191375 (580h39m54.091145561s) [upload=511155]: Waiting for error group managing segments...
Feb 8 20:35:16.334354958 (580h39m54.21830916s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.335358031 (580h39m54.21931223s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.337810738 (580h39m54.221764931s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.338164047 (580h39m54.222118238s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.342858708 (580h39m54.226812913s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.343378259 (580h39m54.227332458s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.345609935 (580h39m54.229564141s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.350042706 (580h39m54.233996915s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.379091195 (580h39m54.263045398s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.387202016 (580h39m54.271156202s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.390136524 (580h39m54.274090711s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:44.095174165 (580h40m21.979128365s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:45.002467024 (580h40m22.886421233s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:1JgPWqe81FwGvfduteWKs6QAmxY3Av3Nbd4RBu4fcZh5rH3RMM, address:79.127.219.41:4927): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002509190 (580h40m22.886463383s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002708091 (580h40m22.886662296s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:19MoR7r7wEgpnEpEesf4oJzFsWZmygg7mxg4jCtGf9rhJDw56B, address:45.91.116.4:4947): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002855562 (580h40m22.88680977s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12A39mM8Cc7tERvweTDoj4wkGgNTNn2QafEkMhn8PV1rwREj8WP, address:79.127.219.41:4930): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002888009 (580h40m22.886842201s) [seg_pos={0 0}, upload=511155]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Feb 8 20:35:45.110856986 (580h40m22.994811201s) [upload=511155]: Done uploading segments...
Feb 8 20:35:45.110918692 (580h40m22.994872902s) [upload=511155]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.339340649s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
2.31889726s trace:
https://jaeger.dev.storj.io/trace/3c8a03916dd3efb0
FAIL (2026-02-08 21:03:20 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Feb 8 20:35:15.284031765 (580h39m53.167985964s) [upload=511155]: Starting upload
Feb 8 20:35:15.284112500 (580h39m53.168066701s) [upload=511155]: Done starting upload
Feb 8 20:35:15.284149599 (580h39m53.168103791s) [upload=511155]: Uploading segments...
Feb 8 20:35:15.284601793 (580h39m53.168555992s) [upload=511155]: Got next segment. Inline: false
Feb 8 20:35:15.284721869 (580h39m53.16867607s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 20:35:15.284728431 (580h39m53.16868263s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 20:35:15.397389353 (580h39m53.281343559s) [seg_pos={0 0}, upload=511155]: Begin upload segment...
Feb 8 20:35:15.397453134 (580h39m53.281407318s) [seg_pos={0 0}, upload=511155]: Done begin upload segment.
Feb 8 20:35:15.397517573 (580h39m53.281471782s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.397533666 (580h39m53.281487892s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398120348 (580h39m53.282074556s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398572052 (580h39m53.282526262s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399001740 (580h39m53.282955955s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399492847 (580h39m53.283447038s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399954482 (580h39m53.283908691s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.400483442 (580h39m53.284437649s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.402650352 (580h39m53.286604556s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.404412781 (580h39m53.288366984s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.406266549 (580h39m53.290220752s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.407902210 (580h39m53.291856418s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.409594415 (580h39m53.293548626s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.410340922 (580h39m53.294295132s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411911194 (580h39m53.2958654s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411928878 (580h39m53.29588307s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:16.207177374 (580h39m54.091131559s) [upload=511155]: Next returned nil segment
Feb 8 20:35:16.207191375 (580h39m54.091145561s) [upload=511155]: Waiting for error group managing segments...
Feb 8 20:35:16.334354958 (580h39m54.21830916s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.335358031 (580h39m54.21931223s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.337810738 (580h39m54.221764931s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.338164047 (580h39m54.222118238s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.342858708 (580h39m54.226812913s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.343378259 (580h39m54.227332458s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.345609935 (580h39m54.229564141s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.350042706 (580h39m54.233996915s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.379091195 (580h39m54.263045398s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.387202016 (580h39m54.271156202s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.390136524 (580h39m54.274090711s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:44.095174165 (580h40m21.979128365s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:45.002467024 (580h40m22.886421233s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:1JgPWqe81FwGvfduteWKs6QAmxY3Av3Nbd4RBu4fcZh5rH3RMM, address:79.127.219.41:4927): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002509190 (580h40m22.886463383s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002708091 (580h40m22.886662296s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:19MoR7r7wEgpnEpEesf4oJzFsWZmygg7mxg4jCtGf9rhJDw56B, address:45.91.116.4:4947): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002855562 (580h40m22.88680977s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12A39mM8Cc7tERvweTDoj4wkGgNTNn2QafEkMhn8PV1rwREj8WP, address:79.127.219.41:4930): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002888009 (580h40m22.886842201s) [seg_pos={0 0}, upload=511155]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Feb 8 20:35:45.110856986 (580h40m22.994811201s) [upload=511155]: Done uploading segments...
Feb 8 20:35:45.110918692 (580h40m22.994872902s) [upload=511155]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.31898907s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
2.392014009s trace:
https://jaeger.dev.storj.io/trace/3c8a03916dd3efb0
FAIL (2026-02-08 21:02:14 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Feb 8 20:35:15.284031765 (580h39m53.167985964s) [upload=511155]: Starting upload
Feb 8 20:35:15.284112500 (580h39m53.168066701s) [upload=511155]: Done starting upload
Feb 8 20:35:15.284149599 (580h39m53.168103791s) [upload=511155]: Uploading segments...
Feb 8 20:35:15.284601793 (580h39m53.168555992s) [upload=511155]: Got next segment. Inline: false
Feb 8 20:35:15.284721869 (580h39m53.16867607s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 20:35:15.284728431 (580h39m53.16868263s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 20:35:15.397389353 (580h39m53.281343559s) [seg_pos={0 0}, upload=511155]: Begin upload segment...
Feb 8 20:35:15.397453134 (580h39m53.281407318s) [seg_pos={0 0}, upload=511155]: Done begin upload segment.
Feb 8 20:35:15.397517573 (580h39m53.281471782s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.397533666 (580h39m53.281487892s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398120348 (580h39m53.282074556s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398572052 (580h39m53.282526262s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399001740 (580h39m53.282955955s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399492847 (580h39m53.283447038s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399954482 (580h39m53.283908691s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.400483442 (580h39m53.284437649s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.402650352 (580h39m53.286604556s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.404412781 (580h39m53.288366984s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.406266549 (580h39m53.290220752s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.407902210 (580h39m53.291856418s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.409594415 (580h39m53.293548626s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.410340922 (580h39m53.294295132s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411911194 (580h39m53.2958654s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411928878 (580h39m53.29588307s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:16.207177374 (580h39m54.091131559s) [upload=511155]: Next returned nil segment
Feb 8 20:35:16.207191375 (580h39m54.091145561s) [upload=511155]: Waiting for error group managing segments...
Feb 8 20:35:16.334354958 (580h39m54.21830916s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.335358031 (580h39m54.21931223s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.337810738 (580h39m54.221764931s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.338164047 (580h39m54.222118238s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.342858708 (580h39m54.226812913s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.343378259 (580h39m54.227332458s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.345609935 (580h39m54.229564141s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.350042706 (580h39m54.233996915s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.379091195 (580h39m54.263045398s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.387202016 (580h39m54.271156202s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.390136524 (580h39m54.274090711s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:44.095174165 (580h40m21.979128365s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:45.002467024 (580h40m22.886421233s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:1JgPWqe81FwGvfduteWKs6QAmxY3Av3Nbd4RBu4fcZh5rH3RMM, address:79.127.219.41:4927): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002509190 (580h40m22.886463383s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002708091 (580h40m22.886662296s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:19MoR7r7wEgpnEpEesf4oJzFsWZmygg7mxg4jCtGf9rhJDw56B, address:45.91.116.4:4947): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002855562 (580h40m22.88680977s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12A39mM8Cc7tERvweTDoj4wkGgNTNn2QafEkMhn8PV1rwREj8WP, address:79.127.219.41:4930): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002888009 (580h40m22.886842201s) [seg_pos={0 0}, upload=511155]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Feb 8 20:35:45.110856986 (580h40m22.994811201s) [upload=511155]: Done uploading segments...
Feb 8 20:35:45.110918692 (580h40m22.994872902s) [upload=511155]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.392110729s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
2.449154687s trace:
https://jaeger.dev.storj.io/trace/3c8a03916dd3efb0
FAIL (2026-02-08 21:01:12 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Feb 8 20:35:15.284031765 (580h39m53.167985964s) [upload=511155]: Starting upload
Feb 8 20:35:15.284112500 (580h39m53.168066701s) [upload=511155]: Done starting upload
Feb 8 20:35:15.284149599 (580h39m53.168103791s) [upload=511155]: Uploading segments...
Feb 8 20:35:15.284601793 (580h39m53.168555992s) [upload=511155]: Got next segment. Inline: false
Feb 8 20:35:15.284721869 (580h39m53.16867607s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 8 20:35:15.284728431 (580h39m53.16868263s) [upload=511155]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 8 20:35:15.397389353 (580h39m53.281343559s) [seg_pos={0 0}, upload=511155]: Begin upload segment...
Feb 8 20:35:15.397453134 (580h39m53.281407318s) [seg_pos={0 0}, upload=511155]: Done begin upload segment.
Feb 8 20:35:15.397517573 (580h39m53.281471782s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.397533666 (580h39m53.281487892s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398120348 (580h39m53.282074556s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.398572052 (580h39m53.282526262s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399001740 (580h39m53.282955955s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399492847 (580h39m53.283447038s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.399954482 (580h39m53.283908691s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.400483442 (580h39m53.284437649s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.402650352 (580h39m53.286604556s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.404412781 (580h39m53.288366984s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.406266549 (580h39m53.290220752s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.407902210 (580h39m53.291856418s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.409594415 (580h39m53.293548626s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.410340922 (580h39m53.294295132s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411911194 (580h39m53.2958654s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:15.411928878 (580h39m53.29588307s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Uploading piece...
Feb 8 20:35:16.207177374 (580h39m54.091131559s) [upload=511155]: Next returned nil segment
Feb 8 20:35:16.207191375 (580h39m54.091145561s) [upload=511155]: Waiting for error group managing segments...
Feb 8 20:35:16.334354958 (580h39m54.21830916s) [noise=true , address=77.90.31.12:4589 , piece_id=PNNGHWTVU2Q5MKHVVHSDTNBC67PCNAS6RB2UDAVILBC72EWI3LOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.335358031 (580h39m54.21931223s) [noise=true , address=77.90.31.16:4735 , piece_id=LOXMIJRR3E3V36HUELF3BAM4TG5DCNL37TK6WP54X3OL4A7PHSFQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.337810738 (580h39m54.221764931s) [noise=true , address=77.90.31.14:4663 , piece_id=LX32SOLAV54LL37GSXEUQYEFU5UV4YXZULFOSE4TON43HJZVHQCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.338164047 (580h39m54.222118238s) [noise=true , address=77.90.31.19:4841 , piece_id=CSPTVQ54Z3MMEOC7ZA4QXQEXNHS6WDGC5DSBH5O5G65BTJK4WOOQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.342858708 (580h39m54.226812913s) [noise=true , address=145.79.227.42:4123 , piece_id=NPPJ2GEXVWE5OFCUBZVJP3XSZOXTOVHQK5IWDC3DV653DU2KSUHQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.343378259 (580h39m54.227332458s) [noise=true , address=45.91.116.12:4244 , piece_id=RCA4ATQF22KSJ2JEET4XXMAAQK5S3KVYSY4UOJEX2VC6MTYJKDZA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.345609935 (580h39m54.229564141s) [noise=true , address=77.90.31.17:4773 , piece_id=ULHWAAU4FL3VIF3ELDVHDMLXQ77U3F4LLDW4ZYXWKEJVU3ILXZEA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.350042706 (580h39m54.233996915s) [noise=true , address=45.91.116.4:4952 , piece_id=4MFOEWCPDPR6WDQBRWWYLFUDKQDO6BGXXCY7VPBUB7XDSEN25YOA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.379091195 (580h39m54.263045398s) [noise=true , address=77.90.31.15:4700 , piece_id=SH4DGURZPM4E7W3PXW6LAXAPSBPCZHVDPPP43K3FZVJN5OQRXR6A, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.387202016 (580h39m54.271156202s) [noise=true , address=77.90.31.10:4519 , piece_id=IVSFMBTNVCUIJWJGC4DWHKQEC6HGGTXKBLVCKAQLNHE3CEUSDOGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:16.390136524 (580h39m54.274090711s) [noise=true , address=45.91.116.7:4024 , piece_id=52MSPAT5O2G2XTLNNWMD4PTBGRCL6UMXZAO6CKP2C4CKJYEP25MQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:44.095174165 (580h40m21.979128365s) [noise=true , address=79.127.219.46:4893 , piece_id=G3JMHW4TTJBUGZXFQ6JMYSWZGG54YFSLAUO5NMJ64KZCPLNESIAA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: <nil>
Feb 8 20:35:45.002467024 (580h40m22.886421233s) [noise=true , address=79.127.219.41:4927 , piece_id=DC4VW465KZXLINYS7AFC7CKH2M5M7UN2KHPIGEVPTQQFLJWHJQXQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:1JgPWqe81FwGvfduteWKs6QAmxY3Av3Nbd4RBu4fcZh5rH3RMM, address:79.127.219.41:4927): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002509190 (580h40m22.886463383s) [noise=true , address=79.127.219.38:4820 , piece_id=D4YYXV3FUYB6CWOZFBMBVCOEOQQHC4F4YBOYNW2WDEKR5FB6NCGA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12ZvmVWpER8Z5N2RtEYx7dpBYnCX8GRsU4nvy5HRi11Hyk7GGf7, address:79.127.219.38:4820): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002708091 (580h40m22.886662296s) [noise=true , address=45.91.116.4:4947 , piece_id=OO27KHBM6HQWUWWPYUDGFFCG2HX3KI2NYIYO7O6YFLKS4B5Z3DCQ, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:19MoR7r7wEgpnEpEesf4oJzFsWZmygg7mxg4jCtGf9rhJDw56B, address:45.91.116.4:4947): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002855562 (580h40m22.88680977s) [noise=true , address=79.127.219.41:4930 , piece_id=DAI2WRJQOH6TYDBINB42V6SQWWXX6Z6UUDYTOVKZM3T3N3HKNTCA, seg_pos={0 0}, upload=511155]: Done uploading piece. err: ecclient: upload failed (node:12A39mM8Cc7tERvweTDoj4wkGgNTNn2QafEkMhn8PV1rwREj8WP, address:79.127.219.41:4930): protocol: expected piece hash; context deadline exceeded; EOF
Feb 8 20:35:45.002888009 (580h40m22.886842201s) [seg_pos={0 0}, upload=511155]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Feb 8 20:35:45.110856986 (580h40m22.994811201s) [upload=511155]: Done uploading segments...
Feb 8 20:35:45.110918692 (580h40m22.994872902s) [upload=511155]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 2.449182716s
core
uplink
us-select-1
|