Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 2.996993007s trace: https://jaeger.dev.storj.io/trace/1b4db29464d5897c FAIL (2025-11-05 10:00:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.997020453s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 2.454769535s trace: https://jaeger.dev.storj.io/trace/1b4db29464d5897c FAIL (2025-11-05 09:59:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.45480131s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 2.84642225s trace: https://jaeger.dev.storj.io/trace/1b4db29464d5897c FAIL (2025-11-05 09:59:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.846458402s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 2.571817316s trace: https://jaeger.dev.storj.io/trace/1b4db29464d5897c FAIL (2025-11-05 09:58:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.571836558s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 2.744297685s trace: https://jaeger.dev.storj.io/trace/1b4db29464d5897c FAIL (2025-11-05 09:57:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.744323763s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 2.725950445s trace: https://jaeger.dev.storj.io/trace/1b4db29464d5897c FAIL (2025-11-05 09:56:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.725982527s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 2.713299247s trace: https://jaeger.dev.storj.io/trace/1b4db29464d5897c FAIL (2025-11-05 09:55:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.713322897s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 2.67527892s trace: https://jaeger.dev.storj.io/trace/1b4db29464d5897c FAIL (2025-11-05 09:54:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.675339604s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 2.799805762s trace: https://jaeger.dev.storj.io/trace/1b4db29464d5897c FAIL (2025-11-05 09:53:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.799899211s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 2.704742218s trace: https://jaeger.dev.storj.io/trace/1b4db29464d5897c FAIL (2025-11-05 09:52:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.704771793s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 2.682245128s trace: https://jaeger.dev.storj.io/trace/1b4db29464d5897c FAIL (2025-11-05 09:52:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.682275883s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 2.781320224s trace: https://jaeger.dev.storj.io/trace/1b4db29464d5897c FAIL (2025-11-05 09:51:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.781362065s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 2.861765658s trace: https://jaeger.dev.storj.io/trace/1b4db29464d5897c FAIL (2025-11-05 09:50:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.861876279s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 2.570312241s trace: https://jaeger.dev.storj.io/trace/1b4db29464d5897c FAIL (2025-11-05 09:49:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.570342663s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 2.536207605s trace: https://jaeger.dev.storj.io/trace/1b4db29464d5897c FAIL (2025-11-05 09:48:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.536248309s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 2.679660063s trace: https://jaeger.dev.storj.io/trace/1b4db29464d5897c FAIL (2025-11-05 09:47:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.679703366s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 2.887214232s trace: https://jaeger.dev.storj.io/trace/1b4db29464d5897c FAIL (2025-11-05 09:47:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.887259688s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 2.533598771s trace: https://jaeger.dev.storj.io/trace/1b4db29464d5897c FAIL (2025-11-05 09:46:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.533675366s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 2.687954249s trace: https://jaeger.dev.storj.io/trace/1b4db29464d5897c FAIL (2025-11-05 09:45:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.687984607s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 2.71955185s trace: https://jaeger.dev.storj.io/trace/1b4db29464d5897c FAIL (2025-11-05 09:45:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.719584861s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 2.922203191s trace: https://jaeger.dev.storj.io/trace/1b4db29464d5897c FAIL (2025-11-05 09:44:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.922247197s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 2.755118822s trace: https://jaeger.dev.storj.io/trace/1b4db29464d5897c FAIL (2025-11-05 09:44:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.75526604s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 2.670283796s trace: https://jaeger.dev.storj.io/trace/1b4db29464d5897c FAIL (2025-11-05 09:42:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.670308988s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 2.625417179s trace: https://jaeger.dev.storj.io/trace/1b4db29464d5897c FAIL (2025-11-05 09:41:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.625449062s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 2.683347919s trace: https://jaeger.dev.storj.io/trace/1b4db29464d5897c FAIL (2025-11-05 09:41:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.6834487s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 2.858168497s trace: https://jaeger.dev.storj.io/trace/1b4db29464d5897c FAIL (2025-11-05 09:40:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.858197936s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 2.707080732s trace: https://jaeger.dev.storj.io/trace/1b4db29464d5897c FAIL (2025-11-05 09:39:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.707105295s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 2.750420332s trace: https://jaeger.dev.storj.io/trace/1b4db29464d5897c FAIL (2025-11-05 09:39:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.750515405s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 2.561504631s trace: https://jaeger.dev.storj.io/trace/1b4db29464d5897c FAIL (2025-11-05 09:38:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.561536432s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 2.715019263s trace: https://jaeger.dev.storj.io/trace/1b4db29464d5897c FAIL (2025-11-05 09:37:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.715111576s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 2.677851741s trace: https://jaeger.dev.storj.io/trace/1b4db29464d5897c FAIL (2025-11-05 09:37:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.677882508s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.000606865s trace: https://jaeger.dev.storj.io/trace/1b4db29464d5897c FAIL (2025-11-05 09:37:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000647227s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.358948414s trace: https://jaeger.dev.storj.io/trace/73a29de9c809b554 FAIL (2025-11-05 07:53:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 53.391572844 (139h27m56.093165284s) [upload=122653]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 5 07:08:53.511496551 (139h27m56.213088992s) [seg_pos={0 0}, upload=122653]: Begin upload segment... Nov 5 07:08:53.511554279 (139h27m56.21314672s) [seg_pos={0 0}, upload=122653]: Done begin upload segment. Nov 5 07:08:53.511581616 (139h27m56.213174066s) [noise=true , address=145.79.227.13:4802 , piece_id=PQEQCX24FLF7WNZL6NVRSBH2E4IESGGMEESF77XLCL2DZXTML6DA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.513588552 (139h27m56.215181007s) [noise=true , address=145.79.227.18:4005 , piece_id=5AKQXEJY72OR6NAVFSE5S6EX2TQUURE6R4JCYFWOZNNULR5SS6FQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.513752850 (139h27m56.215345301s) [noise=true , address=77.90.31.13:4628 , piece_id=L3WCZYDMEM2W2552M7LG2A5PBP4HU64JNQMCJAENRC46BUAOG3EQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.514354806 (139h27m56.215947257s) [noise=true , address=77.90.31.18:4807 , piece_id=2OGMRA7RUWBJOKEN3MNZLTW6VIXPOLHRR3O5H3CVNT4AUEKWBXBQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.515244711 (139h27m56.216837159s) [noise=true , address=77.90.31.12:4590 , piece_id=DEMANEV6ZWO62EVVT6FXPJOTNYQLPJIC6ABQCTJCF2MPI2PF2MCA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.515784410 (139h27m56.217376865s) [noise=true , address=77.90.31.21:4879 , piece_id=DTTPJA34K7VYRBAGQ6LYIPKNR4ZUFWNU3XICCXFSIS5VCOCFRGRA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516021191 (139h27m56.217613644s) [noise=true , address=145.79.227.15:4876 , piece_id=P65E3AEIWXVODXYXRLYI6CCUCL37W4LVPVZFE7RYG3L6KJ7SMJVQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516273977 (139h27m56.217866426s) [noise=true , address=145.79.227.31:4797 , piece_id=4IXOFVR7EBRFFWAQKCLNWGE7J5ZARFXBJQ67A3H7FUEP2IQYKNBA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516440796 (139h27m56.218033244s) [noise=true , address=145.79.227.21:4475 , piece_id=42BQHCUFBK2SODY3PIVRYD4N3CQRXDZ6CHXZP3OM2BGRPYBDTI7Q, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516682821 (139h27m56.218275269s) [noise=true , address=145.79.227.14:4840 , piece_id=JVAE4ZXJ6MX4KFVAXDYKSXDYJ5EMNCGMBXRK3WVIPD6NGWB2GQDA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516799834 (139h27m56.218392285s) [noise=true , address=77.90.31.14:4663 , piece_id=TBLQUWZALDARTQIBY3TYWHK24QQYSRM47NCDFDYI3ROL2P2WFIIA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.517334963 (139h27m56.218927412s) [noise=true , address=145.79.227.27:4690 , piece_id=3ZJPN6QKPUH4RIVRTPOJOV6Y4FD4YKYQHZJGAV3CUNWOUTLUY3ZA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.517377298 (139h27m56.21896976s) [noise=true , address=145.79.227.32:4834 , piece_id=NIF4YDGRWHNM734KXYVCGWWHNQPVPZIWZD5UGJKEWGMKWUYN634A, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.517781951 (139h27m56.2193744s) [noise=true , address=145.79.227.10:4401 , piece_id=E3TWGH2FG7QBTIQMEZ3TVTHRQCRUPVJGF65RSPUPF3M6NKRNZHHA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.518271127 (139h27m56.219863578s) [noise=true , address=77.90.31.18:4805 , piece_id=VUQEVDQ3UPZFATAGOQ4QSMTCSWFPMBEK4MRZLF4VOKPLFPYIMMBA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.518768591 (139h27m56.220361047s) [noise=true , address=145.79.227.24:4582 , piece_id=SM2SI6H445LDLWKN4433SZ2JWNKW7VWCUZD6QI5DC72FFKXIZKGA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:54.140079499 (139h27m56.841671947s) [upload=122653]: Next returned nil segment Nov 5 07:08:54.140092295 (139h27m56.84168473s) [upload=122653]: Waiting for error group managing segments... Nov 5 07:08:54.275546767 (139h27m56.977139218s) [noise=true , address=77.90.31.18:4807 , piece_id=2OGMRA7RUWBJOKEN3MNZLTW6VIXPOLHRR3O5H3CVNT4AUEKWBXBQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.285318656 (139h27m56.98691112s) [noise=true , address=77.90.31.13:4628 , piece_id=L3WCZYDMEM2W2552M7LG2A5PBP4HU64JNQMCJAENRC46BUAOG3EQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.285563903 (139h27m56.987156362s) [noise=true , address=77.90.31.18:4805 , piece_id=VUQEVDQ3UPZFATAGOQ4QSMTCSWFPMBEK4MRZLF4VOKPLFPYIMMBA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.307243468 (139h27m57.008835919s) [noise=true , address=77.90.31.21:4879 , piece_id=DTTPJA34K7VYRBAGQ6LYIPKNR4ZUFWNU3XICCXFSIS5VCOCFRGRA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.311638805 (139h27m57.013231286s) [noise=true , address=77.90.31.14:4663 , piece_id=TBLQUWZALDARTQIBY3TYWHK24QQYSRM47NCDFDYI3ROL2P2WFIIA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.403790160 (139h27m57.105382594s) [noise=true , address=77.90.31.12:4590 , piece_id=DEMANEV6ZWO62EVVT6FXPJOTNYQLPJIC6ABQCTJCF2MPI2PF2MCA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:09:23.097144333 (139h28m25.7987368s) [noise=true , address=145.79.227.21:4475 , piece_id=42BQHCUFBK2SODY3PIVRYD4N3CQRXDZ6CHXZP3OM2BGRPYBDTI7Q, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:128SC59hAFBUVutPFym3iZv9qNYtgiqrk6q9RpfWL2ftyTt5Tzk, address:145.79.227.21:4475): protocol: rpc: dial tcp 145.79.227.21:4475: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097392075 (139h28m25.798984538s) [noise=true , address=145.79.227.27:4690 , piece_id=3ZJPN6QKPUH4RIVRTPOJOV6Y4FD4YKYQHZJGAV3CUNWOUTLUY3ZA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: rpc: dial tcp 145.79.227.27:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097504503 (139h28m25.799096966s) [noise=true , address=145.79.227.24:4582 , piece_id=SM2SI6H445LDLWKN4433SZ2JWNKW7VWCUZD6QI5DC72FFKXIZKGA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1YLhczXWKCwzCQxM97Q77HpiWNMe5E15Nv679FZo5uHw6MRXa, address:145.79.227.24:4582): protocol: rpc: dial tcp 145.79.227.24:4582: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097732944 (139h28m25.799325406s) [noise=true , address=145.79.227.32:4834 , piece_id=NIF4YDGRWHNM734KXYVCGWWHNQPVPZIWZD5UGJKEWGMKWUYN634A, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: rpc: dial tcp 145.79.227.32:4834: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097846391 (139h28m25.799438853s) [noise=true , address=145.79.227.13:4802 , piece_id=PQEQCX24FLF7WNZL6NVRSBH2E4IESGGMEESF77XLCL2DZXTML6DA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:145.79.227.13:4802): protocol: rpc: dial tcp 145.79.227.13:4802: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098108991 (139h28m25.799701452s) [noise=true , address=145.79.227.10:4401 , piece_id=E3TWGH2FG7QBTIQMEZ3TVTHRQCRUPVJGF65RSPUPF3M6NKRNZHHA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: rpc: dial tcp 145.79.227.10:4401: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098205549 (139h28m25.799798009s) [noise=true , address=145.79.227.15:4876 , piece_id=P65E3AEIWXVODXYXRLYI6CCUCL37W4LVPVZFE7RYG3L6KJ7SMJVQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: rpc: dial tcp 145.79.227.15:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098453405 (139h28m25.80004587s) [noise=true , address=145.79.227.18:4005 , piece_id=5AKQXEJY72OR6NAVFSE5S6EX2TQUURE6R4JCYFWOZNNULR5SS6FQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098482863 (139h28m25.800075324s) [noise=true , address=145.79.227.31:4797 , piece_id=4IXOFVR7EBRFFWAQKCLNWGE7J5ZARFXBJQ67A3H7FUEP2IQYKNBA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: rpc: dial tcp 145.79.227.31:4797: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098774913 (139h28m25.800367375s) [noise=true , address=145.79.227.14:4840 , piece_id=JVAE4ZXJ6MX4KFVAXDYKSXDYJ5EMNCGMBXRK3WVIPD6NGWB2GQDA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: rpc: dial tcp 145.79.227.14:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098823983 (139h28m25.800416425s) [seg_pos={0 0}, upload=122653]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Nov 5 07:09:23.210562291 (139h28m25.912154746s) [upload=122653]: Done uploading segments... Nov 5 07:09:23.210617929 (139h28m25.912210379s) [upload=122653]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration:* 2.358981997s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 2.379012212s trace: https://jaeger.dev.storj.io/trace/73a29de9c809b554 FAIL (2025-11-05 07:52:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 53.391572844 (139h27m56.093165284s) [upload=122653]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 5 07:08:53.511496551 (139h27m56.213088992s) [seg_pos={0 0}, upload=122653]: Begin upload segment... Nov 5 07:08:53.511554279 (139h27m56.21314672s) [seg_pos={0 0}, upload=122653]: Done begin upload segment. Nov 5 07:08:53.511581616 (139h27m56.213174066s) [noise=true , address=145.79.227.13:4802 , piece_id=PQEQCX24FLF7WNZL6NVRSBH2E4IESGGMEESF77XLCL2DZXTML6DA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.513588552 (139h27m56.215181007s) [noise=true , address=145.79.227.18:4005 , piece_id=5AKQXEJY72OR6NAVFSE5S6EX2TQUURE6R4JCYFWOZNNULR5SS6FQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.513752850 (139h27m56.215345301s) [noise=true , address=77.90.31.13:4628 , piece_id=L3WCZYDMEM2W2552M7LG2A5PBP4HU64JNQMCJAENRC46BUAOG3EQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.514354806 (139h27m56.215947257s) [noise=true , address=77.90.31.18:4807 , piece_id=2OGMRA7RUWBJOKEN3MNZLTW6VIXPOLHRR3O5H3CVNT4AUEKWBXBQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.515244711 (139h27m56.216837159s) [noise=true , address=77.90.31.12:4590 , piece_id=DEMANEV6ZWO62EVVT6FXPJOTNYQLPJIC6ABQCTJCF2MPI2PF2MCA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.515784410 (139h27m56.217376865s) [noise=true , address=77.90.31.21:4879 , piece_id=DTTPJA34K7VYRBAGQ6LYIPKNR4ZUFWNU3XICCXFSIS5VCOCFRGRA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516021191 (139h27m56.217613644s) [noise=true , address=145.79.227.15:4876 , piece_id=P65E3AEIWXVODXYXRLYI6CCUCL37W4LVPVZFE7RYG3L6KJ7SMJVQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516273977 (139h27m56.217866426s) [noise=true , address=145.79.227.31:4797 , piece_id=4IXOFVR7EBRFFWAQKCLNWGE7J5ZARFXBJQ67A3H7FUEP2IQYKNBA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516440796 (139h27m56.218033244s) [noise=true , address=145.79.227.21:4475 , piece_id=42BQHCUFBK2SODY3PIVRYD4N3CQRXDZ6CHXZP3OM2BGRPYBDTI7Q, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516682821 (139h27m56.218275269s) [noise=true , address=145.79.227.14:4840 , piece_id=JVAE4ZXJ6MX4KFVAXDYKSXDYJ5EMNCGMBXRK3WVIPD6NGWB2GQDA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516799834 (139h27m56.218392285s) [noise=true , address=77.90.31.14:4663 , piece_id=TBLQUWZALDARTQIBY3TYWHK24QQYSRM47NCDFDYI3ROL2P2WFIIA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.517334963 (139h27m56.218927412s) [noise=true , address=145.79.227.27:4690 , piece_id=3ZJPN6QKPUH4RIVRTPOJOV6Y4FD4YKYQHZJGAV3CUNWOUTLUY3ZA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.517377298 (139h27m56.21896976s) [noise=true , address=145.79.227.32:4834 , piece_id=NIF4YDGRWHNM734KXYVCGWWHNQPVPZIWZD5UGJKEWGMKWUYN634A, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.517781951 (139h27m56.2193744s) [noise=true , address=145.79.227.10:4401 , piece_id=E3TWGH2FG7QBTIQMEZ3TVTHRQCRUPVJGF65RSPUPF3M6NKRNZHHA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.518271127 (139h27m56.219863578s) [noise=true , address=77.90.31.18:4805 , piece_id=VUQEVDQ3UPZFATAGOQ4QSMTCSWFPMBEK4MRZLF4VOKPLFPYIMMBA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.518768591 (139h27m56.220361047s) [noise=true , address=145.79.227.24:4582 , piece_id=SM2SI6H445LDLWKN4433SZ2JWNKW7VWCUZD6QI5DC72FFKXIZKGA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:54.140079499 (139h27m56.841671947s) [upload=122653]: Next returned nil segment Nov 5 07:08:54.140092295 (139h27m56.84168473s) [upload=122653]: Waiting for error group managing segments... Nov 5 07:08:54.275546767 (139h27m56.977139218s) [noise=true , address=77.90.31.18:4807 , piece_id=2OGMRA7RUWBJOKEN3MNZLTW6VIXPOLHRR3O5H3CVNT4AUEKWBXBQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.285318656 (139h27m56.98691112s) [noise=true , address=77.90.31.13:4628 , piece_id=L3WCZYDMEM2W2552M7LG2A5PBP4HU64JNQMCJAENRC46BUAOG3EQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.285563903 (139h27m56.987156362s) [noise=true , address=77.90.31.18:4805 , piece_id=VUQEVDQ3UPZFATAGOQ4QSMTCSWFPMBEK4MRZLF4VOKPLFPYIMMBA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.307243468 (139h27m57.008835919s) [noise=true , address=77.90.31.21:4879 , piece_id=DTTPJA34K7VYRBAGQ6LYIPKNR4ZUFWNU3XICCXFSIS5VCOCFRGRA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.311638805 (139h27m57.013231286s) [noise=true , address=77.90.31.14:4663 , piece_id=TBLQUWZALDARTQIBY3TYWHK24QQYSRM47NCDFDYI3ROL2P2WFIIA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.403790160 (139h27m57.105382594s) [noise=true , address=77.90.31.12:4590 , piece_id=DEMANEV6ZWO62EVVT6FXPJOTNYQLPJIC6ABQCTJCF2MPI2PF2MCA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:09:23.097144333 (139h28m25.7987368s) [noise=true , address=145.79.227.21:4475 , piece_id=42BQHCUFBK2SODY3PIVRYD4N3CQRXDZ6CHXZP3OM2BGRPYBDTI7Q, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:128SC59hAFBUVutPFym3iZv9qNYtgiqrk6q9RpfWL2ftyTt5Tzk, address:145.79.227.21:4475): protocol: rpc: dial tcp 145.79.227.21:4475: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097392075 (139h28m25.798984538s) [noise=true , address=145.79.227.27:4690 , piece_id=3ZJPN6QKPUH4RIVRTPOJOV6Y4FD4YKYQHZJGAV3CUNWOUTLUY3ZA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: rpc: dial tcp 145.79.227.27:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097504503 (139h28m25.799096966s) [noise=true , address=145.79.227.24:4582 , piece_id=SM2SI6H445LDLWKN4433SZ2JWNKW7VWCUZD6QI5DC72FFKXIZKGA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1YLhczXWKCwzCQxM97Q77HpiWNMe5E15Nv679FZo5uHw6MRXa, address:145.79.227.24:4582): protocol: rpc: dial tcp 145.79.227.24:4582: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097732944 (139h28m25.799325406s) [noise=true , address=145.79.227.32:4834 , piece_id=NIF4YDGRWHNM734KXYVCGWWHNQPVPZIWZD5UGJKEWGMKWUYN634A, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: rpc: dial tcp 145.79.227.32:4834: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097846391 (139h28m25.799438853s) [noise=true , address=145.79.227.13:4802 , piece_id=PQEQCX24FLF7WNZL6NVRSBH2E4IESGGMEESF77XLCL2DZXTML6DA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:145.79.227.13:4802): protocol: rpc: dial tcp 145.79.227.13:4802: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098108991 (139h28m25.799701452s) [noise=true , address=145.79.227.10:4401 , piece_id=E3TWGH2FG7QBTIQMEZ3TVTHRQCRUPVJGF65RSPUPF3M6NKRNZHHA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: rpc: dial tcp 145.79.227.10:4401: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098205549 (139h28m25.799798009s) [noise=true , address=145.79.227.15:4876 , piece_id=P65E3AEIWXVODXYXRLYI6CCUCL37W4LVPVZFE7RYG3L6KJ7SMJVQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: rpc: dial tcp 145.79.227.15:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098453405 (139h28m25.80004587s) [noise=true , address=145.79.227.18:4005 , piece_id=5AKQXEJY72OR6NAVFSE5S6EX2TQUURE6R4JCYFWOZNNULR5SS6FQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098482863 (139h28m25.800075324s) [noise=true , address=145.79.227.31:4797 , piece_id=4IXOFVR7EBRFFWAQKCLNWGE7J5ZARFXBJQ67A3H7FUEP2IQYKNBA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: rpc: dial tcp 145.79.227.31:4797: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098774913 (139h28m25.800367375s) [noise=true , address=145.79.227.14:4840 , piece_id=JVAE4ZXJ6MX4KFVAXDYKSXDYJ5EMNCGMBXRK3WVIPD6NGWB2GQDA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: rpc: dial tcp 145.79.227.14:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098823983 (139h28m25.800416425s) [seg_pos={0 0}, upload=122653]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Nov 5 07:09:23.210562291 (139h28m25.912154746s) [upload=122653]: Done uploading segments... Nov 5 07:09:23.210617929 (139h28m25.912210379s) [upload=122653]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration:* 2.379047097s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 2.382576105s trace: https://jaeger.dev.storj.io/trace/73a29de9c809b554 FAIL (2025-11-05 07:52:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 53.391572844 (139h27m56.093165284s) [upload=122653]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 5 07:08:53.511496551 (139h27m56.213088992s) [seg_pos={0 0}, upload=122653]: Begin upload segment... Nov 5 07:08:53.511554279 (139h27m56.21314672s) [seg_pos={0 0}, upload=122653]: Done begin upload segment. Nov 5 07:08:53.511581616 (139h27m56.213174066s) [noise=true , address=145.79.227.13:4802 , piece_id=PQEQCX24FLF7WNZL6NVRSBH2E4IESGGMEESF77XLCL2DZXTML6DA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.513588552 (139h27m56.215181007s) [noise=true , address=145.79.227.18:4005 , piece_id=5AKQXEJY72OR6NAVFSE5S6EX2TQUURE6R4JCYFWOZNNULR5SS6FQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.513752850 (139h27m56.215345301s) [noise=true , address=77.90.31.13:4628 , piece_id=L3WCZYDMEM2W2552M7LG2A5PBP4HU64JNQMCJAENRC46BUAOG3EQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.514354806 (139h27m56.215947257s) [noise=true , address=77.90.31.18:4807 , piece_id=2OGMRA7RUWBJOKEN3MNZLTW6VIXPOLHRR3O5H3CVNT4AUEKWBXBQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.515244711 (139h27m56.216837159s) [noise=true , address=77.90.31.12:4590 , piece_id=DEMANEV6ZWO62EVVT6FXPJOTNYQLPJIC6ABQCTJCF2MPI2PF2MCA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.515784410 (139h27m56.217376865s) [noise=true , address=77.90.31.21:4879 , piece_id=DTTPJA34K7VYRBAGQ6LYIPKNR4ZUFWNU3XICCXFSIS5VCOCFRGRA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516021191 (139h27m56.217613644s) [noise=true , address=145.79.227.15:4876 , piece_id=P65E3AEIWXVODXYXRLYI6CCUCL37W4LVPVZFE7RYG3L6KJ7SMJVQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516273977 (139h27m56.217866426s) [noise=true , address=145.79.227.31:4797 , piece_id=4IXOFVR7EBRFFWAQKCLNWGE7J5ZARFXBJQ67A3H7FUEP2IQYKNBA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516440796 (139h27m56.218033244s) [noise=true , address=145.79.227.21:4475 , piece_id=42BQHCUFBK2SODY3PIVRYD4N3CQRXDZ6CHXZP3OM2BGRPYBDTI7Q, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516682821 (139h27m56.218275269s) [noise=true , address=145.79.227.14:4840 , piece_id=JVAE4ZXJ6MX4KFVAXDYKSXDYJ5EMNCGMBXRK3WVIPD6NGWB2GQDA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516799834 (139h27m56.218392285s) [noise=true , address=77.90.31.14:4663 , piece_id=TBLQUWZALDARTQIBY3TYWHK24QQYSRM47NCDFDYI3ROL2P2WFIIA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.517334963 (139h27m56.218927412s) [noise=true , address=145.79.227.27:4690 , piece_id=3ZJPN6QKPUH4RIVRTPOJOV6Y4FD4YKYQHZJGAV3CUNWOUTLUY3ZA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.517377298 (139h27m56.21896976s) [noise=true , address=145.79.227.32:4834 , piece_id=NIF4YDGRWHNM734KXYVCGWWHNQPVPZIWZD5UGJKEWGMKWUYN634A, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.517781951 (139h27m56.2193744s) [noise=true , address=145.79.227.10:4401 , piece_id=E3TWGH2FG7QBTIQMEZ3TVTHRQCRUPVJGF65RSPUPF3M6NKRNZHHA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.518271127 (139h27m56.219863578s) [noise=true , address=77.90.31.18:4805 , piece_id=VUQEVDQ3UPZFATAGOQ4QSMTCSWFPMBEK4MRZLF4VOKPLFPYIMMBA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.518768591 (139h27m56.220361047s) [noise=true , address=145.79.227.24:4582 , piece_id=SM2SI6H445LDLWKN4433SZ2JWNKW7VWCUZD6QI5DC72FFKXIZKGA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:54.140079499 (139h27m56.841671947s) [upload=122653]: Next returned nil segment Nov 5 07:08:54.140092295 (139h27m56.84168473s) [upload=122653]: Waiting for error group managing segments... Nov 5 07:08:54.275546767 (139h27m56.977139218s) [noise=true , address=77.90.31.18:4807 , piece_id=2OGMRA7RUWBJOKEN3MNZLTW6VIXPOLHRR3O5H3CVNT4AUEKWBXBQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.285318656 (139h27m56.98691112s) [noise=true , address=77.90.31.13:4628 , piece_id=L3WCZYDMEM2W2552M7LG2A5PBP4HU64JNQMCJAENRC46BUAOG3EQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.285563903 (139h27m56.987156362s) [noise=true , address=77.90.31.18:4805 , piece_id=VUQEVDQ3UPZFATAGOQ4QSMTCSWFPMBEK4MRZLF4VOKPLFPYIMMBA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.307243468 (139h27m57.008835919s) [noise=true , address=77.90.31.21:4879 , piece_id=DTTPJA34K7VYRBAGQ6LYIPKNR4ZUFWNU3XICCXFSIS5VCOCFRGRA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.311638805 (139h27m57.013231286s) [noise=true , address=77.90.31.14:4663 , piece_id=TBLQUWZALDARTQIBY3TYWHK24QQYSRM47NCDFDYI3ROL2P2WFIIA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.403790160 (139h27m57.105382594s) [noise=true , address=77.90.31.12:4590 , piece_id=DEMANEV6ZWO62EVVT6FXPJOTNYQLPJIC6ABQCTJCF2MPI2PF2MCA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:09:23.097144333 (139h28m25.7987368s) [noise=true , address=145.79.227.21:4475 , piece_id=42BQHCUFBK2SODY3PIVRYD4N3CQRXDZ6CHXZP3OM2BGRPYBDTI7Q, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:128SC59hAFBUVutPFym3iZv9qNYtgiqrk6q9RpfWL2ftyTt5Tzk, address:145.79.227.21:4475): protocol: rpc: dial tcp 145.79.227.21:4475: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097392075 (139h28m25.798984538s) [noise=true , address=145.79.227.27:4690 , piece_id=3ZJPN6QKPUH4RIVRTPOJOV6Y4FD4YKYQHZJGAV3CUNWOUTLUY3ZA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: rpc: dial tcp 145.79.227.27:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097504503 (139h28m25.799096966s) [noise=true , address=145.79.227.24:4582 , piece_id=SM2SI6H445LDLWKN4433SZ2JWNKW7VWCUZD6QI5DC72FFKXIZKGA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1YLhczXWKCwzCQxM97Q77HpiWNMe5E15Nv679FZo5uHw6MRXa, address:145.79.227.24:4582): protocol: rpc: dial tcp 145.79.227.24:4582: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097732944 (139h28m25.799325406s) [noise=true , address=145.79.227.32:4834 , piece_id=NIF4YDGRWHNM734KXYVCGWWHNQPVPZIWZD5UGJKEWGMKWUYN634A, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: rpc: dial tcp 145.79.227.32:4834: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097846391 (139h28m25.799438853s) [noise=true , address=145.79.227.13:4802 , piece_id=PQEQCX24FLF7WNZL6NVRSBH2E4IESGGMEESF77XLCL2DZXTML6DA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:145.79.227.13:4802): protocol: rpc: dial tcp 145.79.227.13:4802: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098108991 (139h28m25.799701452s) [noise=true , address=145.79.227.10:4401 , piece_id=E3TWGH2FG7QBTIQMEZ3TVTHRQCRUPVJGF65RSPUPF3M6NKRNZHHA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: rpc: dial tcp 145.79.227.10:4401: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098205549 (139h28m25.799798009s) [noise=true , address=145.79.227.15:4876 , piece_id=P65E3AEIWXVODXYXRLYI6CCUCL37W4LVPVZFE7RYG3L6KJ7SMJVQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: rpc: dial tcp 145.79.227.15:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098453405 (139h28m25.80004587s) [noise=true , address=145.79.227.18:4005 , piece_id=5AKQXEJY72OR6NAVFSE5S6EX2TQUURE6R4JCYFWOZNNULR5SS6FQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098482863 (139h28m25.800075324s) [noise=true , address=145.79.227.31:4797 , piece_id=4IXOFVR7EBRFFWAQKCLNWGE7J5ZARFXBJQ67A3H7FUEP2IQYKNBA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: rpc: dial tcp 145.79.227.31:4797: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098774913 (139h28m25.800367375s) [noise=true , address=145.79.227.14:4840 , piece_id=JVAE4ZXJ6MX4KFVAXDYKSXDYJ5EMNCGMBXRK3WVIPD6NGWB2GQDA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: rpc: dial tcp 145.79.227.14:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098823983 (139h28m25.800416425s) [seg_pos={0 0}, upload=122653]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Nov 5 07:09:23.210562291 (139h28m25.912154746s) [upload=122653]: Done uploading segments... Nov 5 07:09:23.210617929 (139h28m25.912210379s) [upload=122653]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration:* 2.382612429s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 2.305172492s trace: https://jaeger.dev.storj.io/trace/73a29de9c809b554 FAIL (2025-11-05 07:51:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 53.391572844 (139h27m56.093165284s) [upload=122653]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 5 07:08:53.511496551 (139h27m56.213088992s) [seg_pos={0 0}, upload=122653]: Begin upload segment... Nov 5 07:08:53.511554279 (139h27m56.21314672s) [seg_pos={0 0}, upload=122653]: Done begin upload segment. Nov 5 07:08:53.511581616 (139h27m56.213174066s) [noise=true , address=145.79.227.13:4802 , piece_id=PQEQCX24FLF7WNZL6NVRSBH2E4IESGGMEESF77XLCL2DZXTML6DA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.513588552 (139h27m56.215181007s) [noise=true , address=145.79.227.18:4005 , piece_id=5AKQXEJY72OR6NAVFSE5S6EX2TQUURE6R4JCYFWOZNNULR5SS6FQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.513752850 (139h27m56.215345301s) [noise=true , address=77.90.31.13:4628 , piece_id=L3WCZYDMEM2W2552M7LG2A5PBP4HU64JNQMCJAENRC46BUAOG3EQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.514354806 (139h27m56.215947257s) [noise=true , address=77.90.31.18:4807 , piece_id=2OGMRA7RUWBJOKEN3MNZLTW6VIXPOLHRR3O5H3CVNT4AUEKWBXBQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.515244711 (139h27m56.216837159s) [noise=true , address=77.90.31.12:4590 , piece_id=DEMANEV6ZWO62EVVT6FXPJOTNYQLPJIC6ABQCTJCF2MPI2PF2MCA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.515784410 (139h27m56.217376865s) [noise=true , address=77.90.31.21:4879 , piece_id=DTTPJA34K7VYRBAGQ6LYIPKNR4ZUFWNU3XICCXFSIS5VCOCFRGRA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516021191 (139h27m56.217613644s) [noise=true , address=145.79.227.15:4876 , piece_id=P65E3AEIWXVODXYXRLYI6CCUCL37W4LVPVZFE7RYG3L6KJ7SMJVQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516273977 (139h27m56.217866426s) [noise=true , address=145.79.227.31:4797 , piece_id=4IXOFVR7EBRFFWAQKCLNWGE7J5ZARFXBJQ67A3H7FUEP2IQYKNBA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516440796 (139h27m56.218033244s) [noise=true , address=145.79.227.21:4475 , piece_id=42BQHCUFBK2SODY3PIVRYD4N3CQRXDZ6CHXZP3OM2BGRPYBDTI7Q, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516682821 (139h27m56.218275269s) [noise=true , address=145.79.227.14:4840 , piece_id=JVAE4ZXJ6MX4KFVAXDYKSXDYJ5EMNCGMBXRK3WVIPD6NGWB2GQDA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516799834 (139h27m56.218392285s) [noise=true , address=77.90.31.14:4663 , piece_id=TBLQUWZALDARTQIBY3TYWHK24QQYSRM47NCDFDYI3ROL2P2WFIIA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.517334963 (139h27m56.218927412s) [noise=true , address=145.79.227.27:4690 , piece_id=3ZJPN6QKPUH4RIVRTPOJOV6Y4FD4YKYQHZJGAV3CUNWOUTLUY3ZA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.517377298 (139h27m56.21896976s) [noise=true , address=145.79.227.32:4834 , piece_id=NIF4YDGRWHNM734KXYVCGWWHNQPVPZIWZD5UGJKEWGMKWUYN634A, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.517781951 (139h27m56.2193744s) [noise=true , address=145.79.227.10:4401 , piece_id=E3TWGH2FG7QBTIQMEZ3TVTHRQCRUPVJGF65RSPUPF3M6NKRNZHHA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.518271127 (139h27m56.219863578s) [noise=true , address=77.90.31.18:4805 , piece_id=VUQEVDQ3UPZFATAGOQ4QSMTCSWFPMBEK4MRZLF4VOKPLFPYIMMBA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.518768591 (139h27m56.220361047s) [noise=true , address=145.79.227.24:4582 , piece_id=SM2SI6H445LDLWKN4433SZ2JWNKW7VWCUZD6QI5DC72FFKXIZKGA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:54.140079499 (139h27m56.841671947s) [upload=122653]: Next returned nil segment Nov 5 07:08:54.140092295 (139h27m56.84168473s) [upload=122653]: Waiting for error group managing segments... Nov 5 07:08:54.275546767 (139h27m56.977139218s) [noise=true , address=77.90.31.18:4807 , piece_id=2OGMRA7RUWBJOKEN3MNZLTW6VIXPOLHRR3O5H3CVNT4AUEKWBXBQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.285318656 (139h27m56.98691112s) [noise=true , address=77.90.31.13:4628 , piece_id=L3WCZYDMEM2W2552M7LG2A5PBP4HU64JNQMCJAENRC46BUAOG3EQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.285563903 (139h27m56.987156362s) [noise=true , address=77.90.31.18:4805 , piece_id=VUQEVDQ3UPZFATAGOQ4QSMTCSWFPMBEK4MRZLF4VOKPLFPYIMMBA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.307243468 (139h27m57.008835919s) [noise=true , address=77.90.31.21:4879 , piece_id=DTTPJA34K7VYRBAGQ6LYIPKNR4ZUFWNU3XICCXFSIS5VCOCFRGRA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.311638805 (139h27m57.013231286s) [noise=true , address=77.90.31.14:4663 , piece_id=TBLQUWZALDARTQIBY3TYWHK24QQYSRM47NCDFDYI3ROL2P2WFIIA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.403790160 (139h27m57.105382594s) [noise=true , address=77.90.31.12:4590 , piece_id=DEMANEV6ZWO62EVVT6FXPJOTNYQLPJIC6ABQCTJCF2MPI2PF2MCA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:09:23.097144333 (139h28m25.7987368s) [noise=true , address=145.79.227.21:4475 , piece_id=42BQHCUFBK2SODY3PIVRYD4N3CQRXDZ6CHXZP3OM2BGRPYBDTI7Q, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:128SC59hAFBUVutPFym3iZv9qNYtgiqrk6q9RpfWL2ftyTt5Tzk, address:145.79.227.21:4475): protocol: rpc: dial tcp 145.79.227.21:4475: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097392075 (139h28m25.798984538s) [noise=true , address=145.79.227.27:4690 , piece_id=3ZJPN6QKPUH4RIVRTPOJOV6Y4FD4YKYQHZJGAV3CUNWOUTLUY3ZA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: rpc: dial tcp 145.79.227.27:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097504503 (139h28m25.799096966s) [noise=true , address=145.79.227.24:4582 , piece_id=SM2SI6H445LDLWKN4433SZ2JWNKW7VWCUZD6QI5DC72FFKXIZKGA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1YLhczXWKCwzCQxM97Q77HpiWNMe5E15Nv679FZo5uHw6MRXa, address:145.79.227.24:4582): protocol: rpc: dial tcp 145.79.227.24:4582: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097732944 (139h28m25.799325406s) [noise=true , address=145.79.227.32:4834 , piece_id=NIF4YDGRWHNM734KXYVCGWWHNQPVPZIWZD5UGJKEWGMKWUYN634A, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: rpc: dial tcp 145.79.227.32:4834: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097846391 (139h28m25.799438853s) [noise=true , address=145.79.227.13:4802 , piece_id=PQEQCX24FLF7WNZL6NVRSBH2E4IESGGMEESF77XLCL2DZXTML6DA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:145.79.227.13:4802): protocol: rpc: dial tcp 145.79.227.13:4802: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098108991 (139h28m25.799701452s) [noise=true , address=145.79.227.10:4401 , piece_id=E3TWGH2FG7QBTIQMEZ3TVTHRQCRUPVJGF65RSPUPF3M6NKRNZHHA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: rpc: dial tcp 145.79.227.10:4401: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098205549 (139h28m25.799798009s) [noise=true , address=145.79.227.15:4876 , piece_id=P65E3AEIWXVODXYXRLYI6CCUCL37W4LVPVZFE7RYG3L6KJ7SMJVQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: rpc: dial tcp 145.79.227.15:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098453405 (139h28m25.80004587s) [noise=true , address=145.79.227.18:4005 , piece_id=5AKQXEJY72OR6NAVFSE5S6EX2TQUURE6R4JCYFWOZNNULR5SS6FQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098482863 (139h28m25.800075324s) [noise=true , address=145.79.227.31:4797 , piece_id=4IXOFVR7EBRFFWAQKCLNWGE7J5ZARFXBJQ67A3H7FUEP2IQYKNBA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: rpc: dial tcp 145.79.227.31:4797: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098774913 (139h28m25.800367375s) [noise=true , address=145.79.227.14:4840 , piece_id=JVAE4ZXJ6MX4KFVAXDYKSXDYJ5EMNCGMBXRK3WVIPD6NGWB2GQDA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: rpc: dial tcp 145.79.227.14:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098823983 (139h28m25.800416425s) [seg_pos={0 0}, upload=122653]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Nov 5 07:09:23.210562291 (139h28m25.912154746s) [upload=122653]: Done uploading segments... Nov 5 07:09:23.210617929 (139h28m25.912210379s) [upload=122653]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration:* 2.30520313s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 2.372942997s trace: https://jaeger.dev.storj.io/trace/73a29de9c809b554 FAIL (2025-11-05 07:50:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 53.391572844 (139h27m56.093165284s) [upload=122653]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 5 07:08:53.511496551 (139h27m56.213088992s) [seg_pos={0 0}, upload=122653]: Begin upload segment... Nov 5 07:08:53.511554279 (139h27m56.21314672s) [seg_pos={0 0}, upload=122653]: Done begin upload segment. Nov 5 07:08:53.511581616 (139h27m56.213174066s) [noise=true , address=145.79.227.13:4802 , piece_id=PQEQCX24FLF7WNZL6NVRSBH2E4IESGGMEESF77XLCL2DZXTML6DA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.513588552 (139h27m56.215181007s) [noise=true , address=145.79.227.18:4005 , piece_id=5AKQXEJY72OR6NAVFSE5S6EX2TQUURE6R4JCYFWOZNNULR5SS6FQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.513752850 (139h27m56.215345301s) [noise=true , address=77.90.31.13:4628 , piece_id=L3WCZYDMEM2W2552M7LG2A5PBP4HU64JNQMCJAENRC46BUAOG3EQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.514354806 (139h27m56.215947257s) [noise=true , address=77.90.31.18:4807 , piece_id=2OGMRA7RUWBJOKEN3MNZLTW6VIXPOLHRR3O5H3CVNT4AUEKWBXBQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.515244711 (139h27m56.216837159s) [noise=true , address=77.90.31.12:4590 , piece_id=DEMANEV6ZWO62EVVT6FXPJOTNYQLPJIC6ABQCTJCF2MPI2PF2MCA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.515784410 (139h27m56.217376865s) [noise=true , address=77.90.31.21:4879 , piece_id=DTTPJA34K7VYRBAGQ6LYIPKNR4ZUFWNU3XICCXFSIS5VCOCFRGRA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516021191 (139h27m56.217613644s) [noise=true , address=145.79.227.15:4876 , piece_id=P65E3AEIWXVODXYXRLYI6CCUCL37W4LVPVZFE7RYG3L6KJ7SMJVQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516273977 (139h27m56.217866426s) [noise=true , address=145.79.227.31:4797 , piece_id=4IXOFVR7EBRFFWAQKCLNWGE7J5ZARFXBJQ67A3H7FUEP2IQYKNBA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516440796 (139h27m56.218033244s) [noise=true , address=145.79.227.21:4475 , piece_id=42BQHCUFBK2SODY3PIVRYD4N3CQRXDZ6CHXZP3OM2BGRPYBDTI7Q, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516682821 (139h27m56.218275269s) [noise=true , address=145.79.227.14:4840 , piece_id=JVAE4ZXJ6MX4KFVAXDYKSXDYJ5EMNCGMBXRK3WVIPD6NGWB2GQDA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516799834 (139h27m56.218392285s) [noise=true , address=77.90.31.14:4663 , piece_id=TBLQUWZALDARTQIBY3TYWHK24QQYSRM47NCDFDYI3ROL2P2WFIIA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.517334963 (139h27m56.218927412s) [noise=true , address=145.79.227.27:4690 , piece_id=3ZJPN6QKPUH4RIVRTPOJOV6Y4FD4YKYQHZJGAV3CUNWOUTLUY3ZA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.517377298 (139h27m56.21896976s) [noise=true , address=145.79.227.32:4834 , piece_id=NIF4YDGRWHNM734KXYVCGWWHNQPVPZIWZD5UGJKEWGMKWUYN634A, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.517781951 (139h27m56.2193744s) [noise=true , address=145.79.227.10:4401 , piece_id=E3TWGH2FG7QBTIQMEZ3TVTHRQCRUPVJGF65RSPUPF3M6NKRNZHHA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.518271127 (139h27m56.219863578s) [noise=true , address=77.90.31.18:4805 , piece_id=VUQEVDQ3UPZFATAGOQ4QSMTCSWFPMBEK4MRZLF4VOKPLFPYIMMBA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.518768591 (139h27m56.220361047s) [noise=true , address=145.79.227.24:4582 , piece_id=SM2SI6H445LDLWKN4433SZ2JWNKW7VWCUZD6QI5DC72FFKXIZKGA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:54.140079499 (139h27m56.841671947s) [upload=122653]: Next returned nil segment Nov 5 07:08:54.140092295 (139h27m56.84168473s) [upload=122653]: Waiting for error group managing segments... Nov 5 07:08:54.275546767 (139h27m56.977139218s) [noise=true , address=77.90.31.18:4807 , piece_id=2OGMRA7RUWBJOKEN3MNZLTW6VIXPOLHRR3O5H3CVNT4AUEKWBXBQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.285318656 (139h27m56.98691112s) [noise=true , address=77.90.31.13:4628 , piece_id=L3WCZYDMEM2W2552M7LG2A5PBP4HU64JNQMCJAENRC46BUAOG3EQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.285563903 (139h27m56.987156362s) [noise=true , address=77.90.31.18:4805 , piece_id=VUQEVDQ3UPZFATAGOQ4QSMTCSWFPMBEK4MRZLF4VOKPLFPYIMMBA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.307243468 (139h27m57.008835919s) [noise=true , address=77.90.31.21:4879 , piece_id=DTTPJA34K7VYRBAGQ6LYIPKNR4ZUFWNU3XICCXFSIS5VCOCFRGRA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.311638805 (139h27m57.013231286s) [noise=true , address=77.90.31.14:4663 , piece_id=TBLQUWZALDARTQIBY3TYWHK24QQYSRM47NCDFDYI3ROL2P2WFIIA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.403790160 (139h27m57.105382594s) [noise=true , address=77.90.31.12:4590 , piece_id=DEMANEV6ZWO62EVVT6FXPJOTNYQLPJIC6ABQCTJCF2MPI2PF2MCA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:09:23.097144333 (139h28m25.7987368s) [noise=true , address=145.79.227.21:4475 , piece_id=42BQHCUFBK2SODY3PIVRYD4N3CQRXDZ6CHXZP3OM2BGRPYBDTI7Q, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:128SC59hAFBUVutPFym3iZv9qNYtgiqrk6q9RpfWL2ftyTt5Tzk, address:145.79.227.21:4475): protocol: rpc: dial tcp 145.79.227.21:4475: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097392075 (139h28m25.798984538s) [noise=true , address=145.79.227.27:4690 , piece_id=3ZJPN6QKPUH4RIVRTPOJOV6Y4FD4YKYQHZJGAV3CUNWOUTLUY3ZA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: rpc: dial tcp 145.79.227.27:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097504503 (139h28m25.799096966s) [noise=true , address=145.79.227.24:4582 , piece_id=SM2SI6H445LDLWKN4433SZ2JWNKW7VWCUZD6QI5DC72FFKXIZKGA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1YLhczXWKCwzCQxM97Q77HpiWNMe5E15Nv679FZo5uHw6MRXa, address:145.79.227.24:4582): protocol: rpc: dial tcp 145.79.227.24:4582: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097732944 (139h28m25.799325406s) [noise=true , address=145.79.227.32:4834 , piece_id=NIF4YDGRWHNM734KXYVCGWWHNQPVPZIWZD5UGJKEWGMKWUYN634A, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: rpc: dial tcp 145.79.227.32:4834: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097846391 (139h28m25.799438853s) [noise=true , address=145.79.227.13:4802 , piece_id=PQEQCX24FLF7WNZL6NVRSBH2E4IESGGMEESF77XLCL2DZXTML6DA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:145.79.227.13:4802): protocol: rpc: dial tcp 145.79.227.13:4802: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098108991 (139h28m25.799701452s) [noise=true , address=145.79.227.10:4401 , piece_id=E3TWGH2FG7QBTIQMEZ3TVTHRQCRUPVJGF65RSPUPF3M6NKRNZHHA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: rpc: dial tcp 145.79.227.10:4401: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098205549 (139h28m25.799798009s) [noise=true , address=145.79.227.15:4876 , piece_id=P65E3AEIWXVODXYXRLYI6CCUCL37W4LVPVZFE7RYG3L6KJ7SMJVQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: rpc: dial tcp 145.79.227.15:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098453405 (139h28m25.80004587s) [noise=true , address=145.79.227.18:4005 , piece_id=5AKQXEJY72OR6NAVFSE5S6EX2TQUURE6R4JCYFWOZNNULR5SS6FQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098482863 (139h28m25.800075324s) [noise=true , address=145.79.227.31:4797 , piece_id=4IXOFVR7EBRFFWAQKCLNWGE7J5ZARFXBJQ67A3H7FUEP2IQYKNBA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: rpc: dial tcp 145.79.227.31:4797: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098774913 (139h28m25.800367375s) [noise=true , address=145.79.227.14:4840 , piece_id=JVAE4ZXJ6MX4KFVAXDYKSXDYJ5EMNCGMBXRK3WVIPD6NGWB2GQDA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: rpc: dial tcp 145.79.227.14:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098823983 (139h28m25.800416425s) [seg_pos={0 0}, upload=122653]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Nov 5 07:09:23.210562291 (139h28m25.912154746s) [upload=122653]: Done uploading segments... Nov 5 07:09:23.210617929 (139h28m25.912210379s) [upload=122653]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration:* 2.372972492s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.531396477s trace: https://jaeger.dev.storj.io/trace/49cd51d016f1caf5 FAIL (2025-11-05 07:50:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.531427405s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 2.814507043s trace: https://jaeger.dev.storj.io/trace/49cd51d016f1caf5 FAIL (2025-11-05 07:50:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.814533707s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 2.388779948s trace: https://jaeger.dev.storj.io/trace/73a29de9c809b554 FAIL (2025-11-05 07:49:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 53.391572844 (139h27m56.093165284s) [upload=122653]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 5 07:08:53.511496551 (139h27m56.213088992s) [seg_pos={0 0}, upload=122653]: Begin upload segment... Nov 5 07:08:53.511554279 (139h27m56.21314672s) [seg_pos={0 0}, upload=122653]: Done begin upload segment. Nov 5 07:08:53.511581616 (139h27m56.213174066s) [noise=true , address=145.79.227.13:4802 , piece_id=PQEQCX24FLF7WNZL6NVRSBH2E4IESGGMEESF77XLCL2DZXTML6DA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.513588552 (139h27m56.215181007s) [noise=true , address=145.79.227.18:4005 , piece_id=5AKQXEJY72OR6NAVFSE5S6EX2TQUURE6R4JCYFWOZNNULR5SS6FQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.513752850 (139h27m56.215345301s) [noise=true , address=77.90.31.13:4628 , piece_id=L3WCZYDMEM2W2552M7LG2A5PBP4HU64JNQMCJAENRC46BUAOG3EQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.514354806 (139h27m56.215947257s) [noise=true , address=77.90.31.18:4807 , piece_id=2OGMRA7RUWBJOKEN3MNZLTW6VIXPOLHRR3O5H3CVNT4AUEKWBXBQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.515244711 (139h27m56.216837159s) [noise=true , address=77.90.31.12:4590 , piece_id=DEMANEV6ZWO62EVVT6FXPJOTNYQLPJIC6ABQCTJCF2MPI2PF2MCA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.515784410 (139h27m56.217376865s) [noise=true , address=77.90.31.21:4879 , piece_id=DTTPJA34K7VYRBAGQ6LYIPKNR4ZUFWNU3XICCXFSIS5VCOCFRGRA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516021191 (139h27m56.217613644s) [noise=true , address=145.79.227.15:4876 , piece_id=P65E3AEIWXVODXYXRLYI6CCUCL37W4LVPVZFE7RYG3L6KJ7SMJVQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516273977 (139h27m56.217866426s) [noise=true , address=145.79.227.31:4797 , piece_id=4IXOFVR7EBRFFWAQKCLNWGE7J5ZARFXBJQ67A3H7FUEP2IQYKNBA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516440796 (139h27m56.218033244s) [noise=true , address=145.79.227.21:4475 , piece_id=42BQHCUFBK2SODY3PIVRYD4N3CQRXDZ6CHXZP3OM2BGRPYBDTI7Q, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516682821 (139h27m56.218275269s) [noise=true , address=145.79.227.14:4840 , piece_id=JVAE4ZXJ6MX4KFVAXDYKSXDYJ5EMNCGMBXRK3WVIPD6NGWB2GQDA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516799834 (139h27m56.218392285s) [noise=true , address=77.90.31.14:4663 , piece_id=TBLQUWZALDARTQIBY3TYWHK24QQYSRM47NCDFDYI3ROL2P2WFIIA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.517334963 (139h27m56.218927412s) [noise=true , address=145.79.227.27:4690 , piece_id=3ZJPN6QKPUH4RIVRTPOJOV6Y4FD4YKYQHZJGAV3CUNWOUTLUY3ZA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.517377298 (139h27m56.21896976s) [noise=true , address=145.79.227.32:4834 , piece_id=NIF4YDGRWHNM734KXYVCGWWHNQPVPZIWZD5UGJKEWGMKWUYN634A, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.517781951 (139h27m56.2193744s) [noise=true , address=145.79.227.10:4401 , piece_id=E3TWGH2FG7QBTIQMEZ3TVTHRQCRUPVJGF65RSPUPF3M6NKRNZHHA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.518271127 (139h27m56.219863578s) [noise=true , address=77.90.31.18:4805 , piece_id=VUQEVDQ3UPZFATAGOQ4QSMTCSWFPMBEK4MRZLF4VOKPLFPYIMMBA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.518768591 (139h27m56.220361047s) [noise=true , address=145.79.227.24:4582 , piece_id=SM2SI6H445LDLWKN4433SZ2JWNKW7VWCUZD6QI5DC72FFKXIZKGA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:54.140079499 (139h27m56.841671947s) [upload=122653]: Next returned nil segment Nov 5 07:08:54.140092295 (139h27m56.84168473s) [upload=122653]: Waiting for error group managing segments... Nov 5 07:08:54.275546767 (139h27m56.977139218s) [noise=true , address=77.90.31.18:4807 , piece_id=2OGMRA7RUWBJOKEN3MNZLTW6VIXPOLHRR3O5H3CVNT4AUEKWBXBQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.285318656 (139h27m56.98691112s) [noise=true , address=77.90.31.13:4628 , piece_id=L3WCZYDMEM2W2552M7LG2A5PBP4HU64JNQMCJAENRC46BUAOG3EQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.285563903 (139h27m56.987156362s) [noise=true , address=77.90.31.18:4805 , piece_id=VUQEVDQ3UPZFATAGOQ4QSMTCSWFPMBEK4MRZLF4VOKPLFPYIMMBA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.307243468 (139h27m57.008835919s) [noise=true , address=77.90.31.21:4879 , piece_id=DTTPJA34K7VYRBAGQ6LYIPKNR4ZUFWNU3XICCXFSIS5VCOCFRGRA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.311638805 (139h27m57.013231286s) [noise=true , address=77.90.31.14:4663 , piece_id=TBLQUWZALDARTQIBY3TYWHK24QQYSRM47NCDFDYI3ROL2P2WFIIA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.403790160 (139h27m57.105382594s) [noise=true , address=77.90.31.12:4590 , piece_id=DEMANEV6ZWO62EVVT6FXPJOTNYQLPJIC6ABQCTJCF2MPI2PF2MCA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:09:23.097144333 (139h28m25.7987368s) [noise=true , address=145.79.227.21:4475 , piece_id=42BQHCUFBK2SODY3PIVRYD4N3CQRXDZ6CHXZP3OM2BGRPYBDTI7Q, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:128SC59hAFBUVutPFym3iZv9qNYtgiqrk6q9RpfWL2ftyTt5Tzk, address:145.79.227.21:4475): protocol: rpc: dial tcp 145.79.227.21:4475: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097392075 (139h28m25.798984538s) [noise=true , address=145.79.227.27:4690 , piece_id=3ZJPN6QKPUH4RIVRTPOJOV6Y4FD4YKYQHZJGAV3CUNWOUTLUY3ZA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: rpc: dial tcp 145.79.227.27:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097504503 (139h28m25.799096966s) [noise=true , address=145.79.227.24:4582 , piece_id=SM2SI6H445LDLWKN4433SZ2JWNKW7VWCUZD6QI5DC72FFKXIZKGA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1YLhczXWKCwzCQxM97Q77HpiWNMe5E15Nv679FZo5uHw6MRXa, address:145.79.227.24:4582): protocol: rpc: dial tcp 145.79.227.24:4582: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097732944 (139h28m25.799325406s) [noise=true , address=145.79.227.32:4834 , piece_id=NIF4YDGRWHNM734KXYVCGWWHNQPVPZIWZD5UGJKEWGMKWUYN634A, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: rpc: dial tcp 145.79.227.32:4834: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097846391 (139h28m25.799438853s) [noise=true , address=145.79.227.13:4802 , piece_id=PQEQCX24FLF7WNZL6NVRSBH2E4IESGGMEESF77XLCL2DZXTML6DA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:145.79.227.13:4802): protocol: rpc: dial tcp 145.79.227.13:4802: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098108991 (139h28m25.799701452s) [noise=true , address=145.79.227.10:4401 , piece_id=E3TWGH2FG7QBTIQMEZ3TVTHRQCRUPVJGF65RSPUPF3M6NKRNZHHA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: rpc: dial tcp 145.79.227.10:4401: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098205549 (139h28m25.799798009s) [noise=true , address=145.79.227.15:4876 , piece_id=P65E3AEIWXVODXYXRLYI6CCUCL37W4LVPVZFE7RYG3L6KJ7SMJVQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: rpc: dial tcp 145.79.227.15:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098453405 (139h28m25.80004587s) [noise=true , address=145.79.227.18:4005 , piece_id=5AKQXEJY72OR6NAVFSE5S6EX2TQUURE6R4JCYFWOZNNULR5SS6FQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098482863 (139h28m25.800075324s) [noise=true , address=145.79.227.31:4797 , piece_id=4IXOFVR7EBRFFWAQKCLNWGE7J5ZARFXBJQ67A3H7FUEP2IQYKNBA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: rpc: dial tcp 145.79.227.31:4797: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098774913 (139h28m25.800367375s) [noise=true , address=145.79.227.14:4840 , piece_id=JVAE4ZXJ6MX4KFVAXDYKSXDYJ5EMNCGMBXRK3WVIPD6NGWB2GQDA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: rpc: dial tcp 145.79.227.14:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098823983 (139h28m25.800416425s) [seg_pos={0 0}, upload=122653]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Nov 5 07:09:23.210562291 (139h28m25.912154746s) [upload=122653]: Done uploading segments... Nov 5 07:09:23.210617929 (139h28m25.912210379s) [upload=122653]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration:* 2.388815941s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 2.317489889s trace: https://jaeger.dev.storj.io/trace/73a29de9c809b554 FAIL (2025-11-05 07:49:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 53.391572844 (139h27m56.093165284s) [upload=122653]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 5 07:08:53.511496551 (139h27m56.213088992s) [seg_pos={0 0}, upload=122653]: Begin upload segment... Nov 5 07:08:53.511554279 (139h27m56.21314672s) [seg_pos={0 0}, upload=122653]: Done begin upload segment. Nov 5 07:08:53.511581616 (139h27m56.213174066s) [noise=true , address=145.79.227.13:4802 , piece_id=PQEQCX24FLF7WNZL6NVRSBH2E4IESGGMEESF77XLCL2DZXTML6DA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.513588552 (139h27m56.215181007s) [noise=true , address=145.79.227.18:4005 , piece_id=5AKQXEJY72OR6NAVFSE5S6EX2TQUURE6R4JCYFWOZNNULR5SS6FQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.513752850 (139h27m56.215345301s) [noise=true , address=77.90.31.13:4628 , piece_id=L3WCZYDMEM2W2552M7LG2A5PBP4HU64JNQMCJAENRC46BUAOG3EQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.514354806 (139h27m56.215947257s) [noise=true , address=77.90.31.18:4807 , piece_id=2OGMRA7RUWBJOKEN3MNZLTW6VIXPOLHRR3O5H3CVNT4AUEKWBXBQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.515244711 (139h27m56.216837159s) [noise=true , address=77.90.31.12:4590 , piece_id=DEMANEV6ZWO62EVVT6FXPJOTNYQLPJIC6ABQCTJCF2MPI2PF2MCA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.515784410 (139h27m56.217376865s) [noise=true , address=77.90.31.21:4879 , piece_id=DTTPJA34K7VYRBAGQ6LYIPKNR4ZUFWNU3XICCXFSIS5VCOCFRGRA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516021191 (139h27m56.217613644s) [noise=true , address=145.79.227.15:4876 , piece_id=P65E3AEIWXVODXYXRLYI6CCUCL37W4LVPVZFE7RYG3L6KJ7SMJVQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516273977 (139h27m56.217866426s) [noise=true , address=145.79.227.31:4797 , piece_id=4IXOFVR7EBRFFWAQKCLNWGE7J5ZARFXBJQ67A3H7FUEP2IQYKNBA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516440796 (139h27m56.218033244s) [noise=true , address=145.79.227.21:4475 , piece_id=42BQHCUFBK2SODY3PIVRYD4N3CQRXDZ6CHXZP3OM2BGRPYBDTI7Q, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516682821 (139h27m56.218275269s) [noise=true , address=145.79.227.14:4840 , piece_id=JVAE4ZXJ6MX4KFVAXDYKSXDYJ5EMNCGMBXRK3WVIPD6NGWB2GQDA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516799834 (139h27m56.218392285s) [noise=true , address=77.90.31.14:4663 , piece_id=TBLQUWZALDARTQIBY3TYWHK24QQYSRM47NCDFDYI3ROL2P2WFIIA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.517334963 (139h27m56.218927412s) [noise=true , address=145.79.227.27:4690 , piece_id=3ZJPN6QKPUH4RIVRTPOJOV6Y4FD4YKYQHZJGAV3CUNWOUTLUY3ZA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.517377298 (139h27m56.21896976s) [noise=true , address=145.79.227.32:4834 , piece_id=NIF4YDGRWHNM734KXYVCGWWHNQPVPZIWZD5UGJKEWGMKWUYN634A, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.517781951 (139h27m56.2193744s) [noise=true , address=145.79.227.10:4401 , piece_id=E3TWGH2FG7QBTIQMEZ3TVTHRQCRUPVJGF65RSPUPF3M6NKRNZHHA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.518271127 (139h27m56.219863578s) [noise=true , address=77.90.31.18:4805 , piece_id=VUQEVDQ3UPZFATAGOQ4QSMTCSWFPMBEK4MRZLF4VOKPLFPYIMMBA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.518768591 (139h27m56.220361047s) [noise=true , address=145.79.227.24:4582 , piece_id=SM2SI6H445LDLWKN4433SZ2JWNKW7VWCUZD6QI5DC72FFKXIZKGA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:54.140079499 (139h27m56.841671947s) [upload=122653]: Next returned nil segment Nov 5 07:08:54.140092295 (139h27m56.84168473s) [upload=122653]: Waiting for error group managing segments... Nov 5 07:08:54.275546767 (139h27m56.977139218s) [noise=true , address=77.90.31.18:4807 , piece_id=2OGMRA7RUWBJOKEN3MNZLTW6VIXPOLHRR3O5H3CVNT4AUEKWBXBQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.285318656 (139h27m56.98691112s) [noise=true , address=77.90.31.13:4628 , piece_id=L3WCZYDMEM2W2552M7LG2A5PBP4HU64JNQMCJAENRC46BUAOG3EQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.285563903 (139h27m56.987156362s) [noise=true , address=77.90.31.18:4805 , piece_id=VUQEVDQ3UPZFATAGOQ4QSMTCSWFPMBEK4MRZLF4VOKPLFPYIMMBA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.307243468 (139h27m57.008835919s) [noise=true , address=77.90.31.21:4879 , piece_id=DTTPJA34K7VYRBAGQ6LYIPKNR4ZUFWNU3XICCXFSIS5VCOCFRGRA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.311638805 (139h27m57.013231286s) [noise=true , address=77.90.31.14:4663 , piece_id=TBLQUWZALDARTQIBY3TYWHK24QQYSRM47NCDFDYI3ROL2P2WFIIA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.403790160 (139h27m57.105382594s) [noise=true , address=77.90.31.12:4590 , piece_id=DEMANEV6ZWO62EVVT6FXPJOTNYQLPJIC6ABQCTJCF2MPI2PF2MCA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:09:23.097144333 (139h28m25.7987368s) [noise=true , address=145.79.227.21:4475 , piece_id=42BQHCUFBK2SODY3PIVRYD4N3CQRXDZ6CHXZP3OM2BGRPYBDTI7Q, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:128SC59hAFBUVutPFym3iZv9qNYtgiqrk6q9RpfWL2ftyTt5Tzk, address:145.79.227.21:4475): protocol: rpc: dial tcp 145.79.227.21:4475: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097392075 (139h28m25.798984538s) [noise=true , address=145.79.227.27:4690 , piece_id=3ZJPN6QKPUH4RIVRTPOJOV6Y4FD4YKYQHZJGAV3CUNWOUTLUY3ZA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: rpc: dial tcp 145.79.227.27:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097504503 (139h28m25.799096966s) [noise=true , address=145.79.227.24:4582 , piece_id=SM2SI6H445LDLWKN4433SZ2JWNKW7VWCUZD6QI5DC72FFKXIZKGA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1YLhczXWKCwzCQxM97Q77HpiWNMe5E15Nv679FZo5uHw6MRXa, address:145.79.227.24:4582): protocol: rpc: dial tcp 145.79.227.24:4582: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097732944 (139h28m25.799325406s) [noise=true , address=145.79.227.32:4834 , piece_id=NIF4YDGRWHNM734KXYVCGWWHNQPVPZIWZD5UGJKEWGMKWUYN634A, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: rpc: dial tcp 145.79.227.32:4834: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097846391 (139h28m25.799438853s) [noise=true , address=145.79.227.13:4802 , piece_id=PQEQCX24FLF7WNZL6NVRSBH2E4IESGGMEESF77XLCL2DZXTML6DA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:145.79.227.13:4802): protocol: rpc: dial tcp 145.79.227.13:4802: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098108991 (139h28m25.799701452s) [noise=true , address=145.79.227.10:4401 , piece_id=E3TWGH2FG7QBTIQMEZ3TVTHRQCRUPVJGF65RSPUPF3M6NKRNZHHA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: rpc: dial tcp 145.79.227.10:4401: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098205549 (139h28m25.799798009s) [noise=true , address=145.79.227.15:4876 , piece_id=P65E3AEIWXVODXYXRLYI6CCUCL37W4LVPVZFE7RYG3L6KJ7SMJVQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: rpc: dial tcp 145.79.227.15:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098453405 (139h28m25.80004587s) [noise=true , address=145.79.227.18:4005 , piece_id=5AKQXEJY72OR6NAVFSE5S6EX2TQUURE6R4JCYFWOZNNULR5SS6FQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098482863 (139h28m25.800075324s) [noise=true , address=145.79.227.31:4797 , piece_id=4IXOFVR7EBRFFWAQKCLNWGE7J5ZARFXBJQ67A3H7FUEP2IQYKNBA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: rpc: dial tcp 145.79.227.31:4797: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098774913 (139h28m25.800367375s) [noise=true , address=145.79.227.14:4840 , piece_id=JVAE4ZXJ6MX4KFVAXDYKSXDYJ5EMNCGMBXRK3WVIPD6NGWB2GQDA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: rpc: dial tcp 145.79.227.14:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098823983 (139h28m25.800416425s) [seg_pos={0 0}, upload=122653]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Nov 5 07:09:23.210562291 (139h28m25.912154746s) [upload=122653]: Done uploading segments... Nov 5 07:09:23.210617929 (139h28m25.912210379s) [upload=122653]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration:* 2.317537098s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.XX.........................................................] duration: 2.630244207s trace: https://jaeger.dev.storj.io/trace/49cd51d016f1caf5 FAIL (2025-11-05 07:49:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.630271239s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 2.449285888s trace: https://jaeger.dev.storj.io/trace/73a29de9c809b554 FAIL (2025-11-05 07:48:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 53.391572844 (139h27m56.093165284s) [upload=122653]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 5 07:08:53.511496551 (139h27m56.213088992s) [seg_pos={0 0}, upload=122653]: Begin upload segment... Nov 5 07:08:53.511554279 (139h27m56.21314672s) [seg_pos={0 0}, upload=122653]: Done begin upload segment. Nov 5 07:08:53.511581616 (139h27m56.213174066s) [noise=true , address=145.79.227.13:4802 , piece_id=PQEQCX24FLF7WNZL6NVRSBH2E4IESGGMEESF77XLCL2DZXTML6DA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.513588552 (139h27m56.215181007s) [noise=true , address=145.79.227.18:4005 , piece_id=5AKQXEJY72OR6NAVFSE5S6EX2TQUURE6R4JCYFWOZNNULR5SS6FQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.513752850 (139h27m56.215345301s) [noise=true , address=77.90.31.13:4628 , piece_id=L3WCZYDMEM2W2552M7LG2A5PBP4HU64JNQMCJAENRC46BUAOG3EQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.514354806 (139h27m56.215947257s) [noise=true , address=77.90.31.18:4807 , piece_id=2OGMRA7RUWBJOKEN3MNZLTW6VIXPOLHRR3O5H3CVNT4AUEKWBXBQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.515244711 (139h27m56.216837159s) [noise=true , address=77.90.31.12:4590 , piece_id=DEMANEV6ZWO62EVVT6FXPJOTNYQLPJIC6ABQCTJCF2MPI2PF2MCA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.515784410 (139h27m56.217376865s) [noise=true , address=77.90.31.21:4879 , piece_id=DTTPJA34K7VYRBAGQ6LYIPKNR4ZUFWNU3XICCXFSIS5VCOCFRGRA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516021191 (139h27m56.217613644s) [noise=true , address=145.79.227.15:4876 , piece_id=P65E3AEIWXVODXYXRLYI6CCUCL37W4LVPVZFE7RYG3L6KJ7SMJVQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516273977 (139h27m56.217866426s) [noise=true , address=145.79.227.31:4797 , piece_id=4IXOFVR7EBRFFWAQKCLNWGE7J5ZARFXBJQ67A3H7FUEP2IQYKNBA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516440796 (139h27m56.218033244s) [noise=true , address=145.79.227.21:4475 , piece_id=42BQHCUFBK2SODY3PIVRYD4N3CQRXDZ6CHXZP3OM2BGRPYBDTI7Q, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516682821 (139h27m56.218275269s) [noise=true , address=145.79.227.14:4840 , piece_id=JVAE4ZXJ6MX4KFVAXDYKSXDYJ5EMNCGMBXRK3WVIPD6NGWB2GQDA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516799834 (139h27m56.218392285s) [noise=true , address=77.90.31.14:4663 , piece_id=TBLQUWZALDARTQIBY3TYWHK24QQYSRM47NCDFDYI3ROL2P2WFIIA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.517334963 (139h27m56.218927412s) [noise=true , address=145.79.227.27:4690 , piece_id=3ZJPN6QKPUH4RIVRTPOJOV6Y4FD4YKYQHZJGAV3CUNWOUTLUY3ZA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.517377298 (139h27m56.21896976s) [noise=true , address=145.79.227.32:4834 , piece_id=NIF4YDGRWHNM734KXYVCGWWHNQPVPZIWZD5UGJKEWGMKWUYN634A, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.517781951 (139h27m56.2193744s) [noise=true , address=145.79.227.10:4401 , piece_id=E3TWGH2FG7QBTIQMEZ3TVTHRQCRUPVJGF65RSPUPF3M6NKRNZHHA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.518271127 (139h27m56.219863578s) [noise=true , address=77.90.31.18:4805 , piece_id=VUQEVDQ3UPZFATAGOQ4QSMTCSWFPMBEK4MRZLF4VOKPLFPYIMMBA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.518768591 (139h27m56.220361047s) [noise=true , address=145.79.227.24:4582 , piece_id=SM2SI6H445LDLWKN4433SZ2JWNKW7VWCUZD6QI5DC72FFKXIZKGA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:54.140079499 (139h27m56.841671947s) [upload=122653]: Next returned nil segment Nov 5 07:08:54.140092295 (139h27m56.84168473s) [upload=122653]: Waiting for error group managing segments... Nov 5 07:08:54.275546767 (139h27m56.977139218s) [noise=true , address=77.90.31.18:4807 , piece_id=2OGMRA7RUWBJOKEN3MNZLTW6VIXPOLHRR3O5H3CVNT4AUEKWBXBQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.285318656 (139h27m56.98691112s) [noise=true , address=77.90.31.13:4628 , piece_id=L3WCZYDMEM2W2552M7LG2A5PBP4HU64JNQMCJAENRC46BUAOG3EQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.285563903 (139h27m56.987156362s) [noise=true , address=77.90.31.18:4805 , piece_id=VUQEVDQ3UPZFATAGOQ4QSMTCSWFPMBEK4MRZLF4VOKPLFPYIMMBA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.307243468 (139h27m57.008835919s) [noise=true , address=77.90.31.21:4879 , piece_id=DTTPJA34K7VYRBAGQ6LYIPKNR4ZUFWNU3XICCXFSIS5VCOCFRGRA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.311638805 (139h27m57.013231286s) [noise=true , address=77.90.31.14:4663 , piece_id=TBLQUWZALDARTQIBY3TYWHK24QQYSRM47NCDFDYI3ROL2P2WFIIA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.403790160 (139h27m57.105382594s) [noise=true , address=77.90.31.12:4590 , piece_id=DEMANEV6ZWO62EVVT6FXPJOTNYQLPJIC6ABQCTJCF2MPI2PF2MCA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:09:23.097144333 (139h28m25.7987368s) [noise=true , address=145.79.227.21:4475 , piece_id=42BQHCUFBK2SODY3PIVRYD4N3CQRXDZ6CHXZP3OM2BGRPYBDTI7Q, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:128SC59hAFBUVutPFym3iZv9qNYtgiqrk6q9RpfWL2ftyTt5Tzk, address:145.79.227.21:4475): protocol: rpc: dial tcp 145.79.227.21:4475: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097392075 (139h28m25.798984538s) [noise=true , address=145.79.227.27:4690 , piece_id=3ZJPN6QKPUH4RIVRTPOJOV6Y4FD4YKYQHZJGAV3CUNWOUTLUY3ZA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: rpc: dial tcp 145.79.227.27:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097504503 (139h28m25.799096966s) [noise=true , address=145.79.227.24:4582 , piece_id=SM2SI6H445LDLWKN4433SZ2JWNKW7VWCUZD6QI5DC72FFKXIZKGA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1YLhczXWKCwzCQxM97Q77HpiWNMe5E15Nv679FZo5uHw6MRXa, address:145.79.227.24:4582): protocol: rpc: dial tcp 145.79.227.24:4582: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097732944 (139h28m25.799325406s) [noise=true , address=145.79.227.32:4834 , piece_id=NIF4YDGRWHNM734KXYVCGWWHNQPVPZIWZD5UGJKEWGMKWUYN634A, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: rpc: dial tcp 145.79.227.32:4834: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097846391 (139h28m25.799438853s) [noise=true , address=145.79.227.13:4802 , piece_id=PQEQCX24FLF7WNZL6NVRSBH2E4IESGGMEESF77XLCL2DZXTML6DA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:145.79.227.13:4802): protocol: rpc: dial tcp 145.79.227.13:4802: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098108991 (139h28m25.799701452s) [noise=true , address=145.79.227.10:4401 , piece_id=E3TWGH2FG7QBTIQMEZ3TVTHRQCRUPVJGF65RSPUPF3M6NKRNZHHA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: rpc: dial tcp 145.79.227.10:4401: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098205549 (139h28m25.799798009s) [noise=true , address=145.79.227.15:4876 , piece_id=P65E3AEIWXVODXYXRLYI6CCUCL37W4LVPVZFE7RYG3L6KJ7SMJVQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: rpc: dial tcp 145.79.227.15:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098453405 (139h28m25.80004587s) [noise=true , address=145.79.227.18:4005 , piece_id=5AKQXEJY72OR6NAVFSE5S6EX2TQUURE6R4JCYFWOZNNULR5SS6FQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098482863 (139h28m25.800075324s) [noise=true , address=145.79.227.31:4797 , piece_id=4IXOFVR7EBRFFWAQKCLNWGE7J5ZARFXBJQ67A3H7FUEP2IQYKNBA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: rpc: dial tcp 145.79.227.31:4797: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098774913 (139h28m25.800367375s) [noise=true , address=145.79.227.14:4840 , piece_id=JVAE4ZXJ6MX4KFVAXDYKSXDYJ5EMNCGMBXRK3WVIPD6NGWB2GQDA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: rpc: dial tcp 145.79.227.14:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098823983 (139h28m25.800416425s) [seg_pos={0 0}, upload=122653]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Nov 5 07:09:23.210562291 (139h28m25.912154746s) [upload=122653]: Done uploading segments... Nov 5 07:09:23.210617929 (139h28m25.912210379s) [upload=122653]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration:* 2.449323204s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..XX........................................................] duration: 2.702808661s trace: https://jaeger.dev.storj.io/trace/49cd51d016f1caf5 FAIL (2025-11-05 07:48:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.702916981s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 2.413920866s trace: https://jaeger.dev.storj.io/trace/73a29de9c809b554 FAIL (2025-11-05 07:48:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 53.391572844 (139h27m56.093165284s) [upload=122653]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 5 07:08:53.511496551 (139h27m56.213088992s) [seg_pos={0 0}, upload=122653]: Begin upload segment... Nov 5 07:08:53.511554279 (139h27m56.21314672s) [seg_pos={0 0}, upload=122653]: Done begin upload segment. Nov 5 07:08:53.511581616 (139h27m56.213174066s) [noise=true , address=145.79.227.13:4802 , piece_id=PQEQCX24FLF7WNZL6NVRSBH2E4IESGGMEESF77XLCL2DZXTML6DA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.513588552 (139h27m56.215181007s) [noise=true , address=145.79.227.18:4005 , piece_id=5AKQXEJY72OR6NAVFSE5S6EX2TQUURE6R4JCYFWOZNNULR5SS6FQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.513752850 (139h27m56.215345301s) [noise=true , address=77.90.31.13:4628 , piece_id=L3WCZYDMEM2W2552M7LG2A5PBP4HU64JNQMCJAENRC46BUAOG3EQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.514354806 (139h27m56.215947257s) [noise=true , address=77.90.31.18:4807 , piece_id=2OGMRA7RUWBJOKEN3MNZLTW6VIXPOLHRR3O5H3CVNT4AUEKWBXBQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.515244711 (139h27m56.216837159s) [noise=true , address=77.90.31.12:4590 , piece_id=DEMANEV6ZWO62EVVT6FXPJOTNYQLPJIC6ABQCTJCF2MPI2PF2MCA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.515784410 (139h27m56.217376865s) [noise=true , address=77.90.31.21:4879 , piece_id=DTTPJA34K7VYRBAGQ6LYIPKNR4ZUFWNU3XICCXFSIS5VCOCFRGRA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516021191 (139h27m56.217613644s) [noise=true , address=145.79.227.15:4876 , piece_id=P65E3AEIWXVODXYXRLYI6CCUCL37W4LVPVZFE7RYG3L6KJ7SMJVQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516273977 (139h27m56.217866426s) [noise=true , address=145.79.227.31:4797 , piece_id=4IXOFVR7EBRFFWAQKCLNWGE7J5ZARFXBJQ67A3H7FUEP2IQYKNBA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516440796 (139h27m56.218033244s) [noise=true , address=145.79.227.21:4475 , piece_id=42BQHCUFBK2SODY3PIVRYD4N3CQRXDZ6CHXZP3OM2BGRPYBDTI7Q, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516682821 (139h27m56.218275269s) [noise=true , address=145.79.227.14:4840 , piece_id=JVAE4ZXJ6MX4KFVAXDYKSXDYJ5EMNCGMBXRK3WVIPD6NGWB2GQDA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516799834 (139h27m56.218392285s) [noise=true , address=77.90.31.14:4663 , piece_id=TBLQUWZALDARTQIBY3TYWHK24QQYSRM47NCDFDYI3ROL2P2WFIIA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.517334963 (139h27m56.218927412s) [noise=true , address=145.79.227.27:4690 , piece_id=3ZJPN6QKPUH4RIVRTPOJOV6Y4FD4YKYQHZJGAV3CUNWOUTLUY3ZA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.517377298 (139h27m56.21896976s) [noise=true , address=145.79.227.32:4834 , piece_id=NIF4YDGRWHNM734KXYVCGWWHNQPVPZIWZD5UGJKEWGMKWUYN634A, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.517781951 (139h27m56.2193744s) [noise=true , address=145.79.227.10:4401 , piece_id=E3TWGH2FG7QBTIQMEZ3TVTHRQCRUPVJGF65RSPUPF3M6NKRNZHHA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.518271127 (139h27m56.219863578s) [noise=true , address=77.90.31.18:4805 , piece_id=VUQEVDQ3UPZFATAGOQ4QSMTCSWFPMBEK4MRZLF4VOKPLFPYIMMBA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.518768591 (139h27m56.220361047s) [noise=true , address=145.79.227.24:4582 , piece_id=SM2SI6H445LDLWKN4433SZ2JWNKW7VWCUZD6QI5DC72FFKXIZKGA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:54.140079499 (139h27m56.841671947s) [upload=122653]: Next returned nil segment Nov 5 07:08:54.140092295 (139h27m56.84168473s) [upload=122653]: Waiting for error group managing segments... Nov 5 07:08:54.275546767 (139h27m56.977139218s) [noise=true , address=77.90.31.18:4807 , piece_id=2OGMRA7RUWBJOKEN3MNZLTW6VIXPOLHRR3O5H3CVNT4AUEKWBXBQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.285318656 (139h27m56.98691112s) [noise=true , address=77.90.31.13:4628 , piece_id=L3WCZYDMEM2W2552M7LG2A5PBP4HU64JNQMCJAENRC46BUAOG3EQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.285563903 (139h27m56.987156362s) [noise=true , address=77.90.31.18:4805 , piece_id=VUQEVDQ3UPZFATAGOQ4QSMTCSWFPMBEK4MRZLF4VOKPLFPYIMMBA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.307243468 (139h27m57.008835919s) [noise=true , address=77.90.31.21:4879 , piece_id=DTTPJA34K7VYRBAGQ6LYIPKNR4ZUFWNU3XICCXFSIS5VCOCFRGRA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.311638805 (139h27m57.013231286s) [noise=true , address=77.90.31.14:4663 , piece_id=TBLQUWZALDARTQIBY3TYWHK24QQYSRM47NCDFDYI3ROL2P2WFIIA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.403790160 (139h27m57.105382594s) [noise=true , address=77.90.31.12:4590 , piece_id=DEMANEV6ZWO62EVVT6FXPJOTNYQLPJIC6ABQCTJCF2MPI2PF2MCA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:09:23.097144333 (139h28m25.7987368s) [noise=true , address=145.79.227.21:4475 , piece_id=42BQHCUFBK2SODY3PIVRYD4N3CQRXDZ6CHXZP3OM2BGRPYBDTI7Q, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:128SC59hAFBUVutPFym3iZv9qNYtgiqrk6q9RpfWL2ftyTt5Tzk, address:145.79.227.21:4475): protocol: rpc: dial tcp 145.79.227.21:4475: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097392075 (139h28m25.798984538s) [noise=true , address=145.79.227.27:4690 , piece_id=3ZJPN6QKPUH4RIVRTPOJOV6Y4FD4YKYQHZJGAV3CUNWOUTLUY3ZA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: rpc: dial tcp 145.79.227.27:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097504503 (139h28m25.799096966s) [noise=true , address=145.79.227.24:4582 , piece_id=SM2SI6H445LDLWKN4433SZ2JWNKW7VWCUZD6QI5DC72FFKXIZKGA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1YLhczXWKCwzCQxM97Q77HpiWNMe5E15Nv679FZo5uHw6MRXa, address:145.79.227.24:4582): protocol: rpc: dial tcp 145.79.227.24:4582: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097732944 (139h28m25.799325406s) [noise=true , address=145.79.227.32:4834 , piece_id=NIF4YDGRWHNM734KXYVCGWWHNQPVPZIWZD5UGJKEWGMKWUYN634A, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: rpc: dial tcp 145.79.227.32:4834: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097846391 (139h28m25.799438853s) [noise=true , address=145.79.227.13:4802 , piece_id=PQEQCX24FLF7WNZL6NVRSBH2E4IESGGMEESF77XLCL2DZXTML6DA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:145.79.227.13:4802): protocol: rpc: dial tcp 145.79.227.13:4802: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098108991 (139h28m25.799701452s) [noise=true , address=145.79.227.10:4401 , piece_id=E3TWGH2FG7QBTIQMEZ3TVTHRQCRUPVJGF65RSPUPF3M6NKRNZHHA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: rpc: dial tcp 145.79.227.10:4401: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098205549 (139h28m25.799798009s) [noise=true , address=145.79.227.15:4876 , piece_id=P65E3AEIWXVODXYXRLYI6CCUCL37W4LVPVZFE7RYG3L6KJ7SMJVQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: rpc: dial tcp 145.79.227.15:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098453405 (139h28m25.80004587s) [noise=true , address=145.79.227.18:4005 , piece_id=5AKQXEJY72OR6NAVFSE5S6EX2TQUURE6R4JCYFWOZNNULR5SS6FQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098482863 (139h28m25.800075324s) [noise=true , address=145.79.227.31:4797 , piece_id=4IXOFVR7EBRFFWAQKCLNWGE7J5ZARFXBJQ67A3H7FUEP2IQYKNBA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: rpc: dial tcp 145.79.227.31:4797: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098774913 (139h28m25.800367375s) [noise=true , address=145.79.227.14:4840 , piece_id=JVAE4ZXJ6MX4KFVAXDYKSXDYJ5EMNCGMBXRK3WVIPD6NGWB2GQDA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: rpc: dial tcp 145.79.227.14:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098823983 (139h28m25.800416425s) [seg_pos={0 0}, upload=122653]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Nov 5 07:09:23.210562291 (139h28m25.912154746s) [upload=122653]: Done uploading segments... Nov 5 07:09:23.210617929 (139h28m25.912210379s) [upload=122653]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration:* 2.413946277s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...XX.......................................................] duration: 2.506566988s trace: https://jaeger.dev.storj.io/trace/49cd51d016f1caf5 FAIL (2025-11-05 07:47:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.506612961s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 2.267103116s trace: https://jaeger.dev.storj.io/trace/73a29de9c809b554 FAIL (2025-11-05 07:47:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 53.391572844 (139h27m56.093165284s) [upload=122653]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 5 07:08:53.511496551 (139h27m56.213088992s) [seg_pos={0 0}, upload=122653]: Begin upload segment... Nov 5 07:08:53.511554279 (139h27m56.21314672s) [seg_pos={0 0}, upload=122653]: Done begin upload segment. Nov 5 07:08:53.511581616 (139h27m56.213174066s) [noise=true , address=145.79.227.13:4802 , piece_id=PQEQCX24FLF7WNZL6NVRSBH2E4IESGGMEESF77XLCL2DZXTML6DA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.513588552 (139h27m56.215181007s) [noise=true , address=145.79.227.18:4005 , piece_id=5AKQXEJY72OR6NAVFSE5S6EX2TQUURE6R4JCYFWOZNNULR5SS6FQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.513752850 (139h27m56.215345301s) [noise=true , address=77.90.31.13:4628 , piece_id=L3WCZYDMEM2W2552M7LG2A5PBP4HU64JNQMCJAENRC46BUAOG3EQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.514354806 (139h27m56.215947257s) [noise=true , address=77.90.31.18:4807 , piece_id=2OGMRA7RUWBJOKEN3MNZLTW6VIXPOLHRR3O5H3CVNT4AUEKWBXBQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.515244711 (139h27m56.216837159s) [noise=true , address=77.90.31.12:4590 , piece_id=DEMANEV6ZWO62EVVT6FXPJOTNYQLPJIC6ABQCTJCF2MPI2PF2MCA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.515784410 (139h27m56.217376865s) [noise=true , address=77.90.31.21:4879 , piece_id=DTTPJA34K7VYRBAGQ6LYIPKNR4ZUFWNU3XICCXFSIS5VCOCFRGRA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516021191 (139h27m56.217613644s) [noise=true , address=145.79.227.15:4876 , piece_id=P65E3AEIWXVODXYXRLYI6CCUCL37W4LVPVZFE7RYG3L6KJ7SMJVQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516273977 (139h27m56.217866426s) [noise=true , address=145.79.227.31:4797 , piece_id=4IXOFVR7EBRFFWAQKCLNWGE7J5ZARFXBJQ67A3H7FUEP2IQYKNBA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516440796 (139h27m56.218033244s) [noise=true , address=145.79.227.21:4475 , piece_id=42BQHCUFBK2SODY3PIVRYD4N3CQRXDZ6CHXZP3OM2BGRPYBDTI7Q, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516682821 (139h27m56.218275269s) [noise=true , address=145.79.227.14:4840 , piece_id=JVAE4ZXJ6MX4KFVAXDYKSXDYJ5EMNCGMBXRK3WVIPD6NGWB2GQDA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516799834 (139h27m56.218392285s) [noise=true , address=77.90.31.14:4663 , piece_id=TBLQUWZALDARTQIBY3TYWHK24QQYSRM47NCDFDYI3ROL2P2WFIIA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.517334963 (139h27m56.218927412s) [noise=true , address=145.79.227.27:4690 , piece_id=3ZJPN6QKPUH4RIVRTPOJOV6Y4FD4YKYQHZJGAV3CUNWOUTLUY3ZA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.517377298 (139h27m56.21896976s) [noise=true , address=145.79.227.32:4834 , piece_id=NIF4YDGRWHNM734KXYVCGWWHNQPVPZIWZD5UGJKEWGMKWUYN634A, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.517781951 (139h27m56.2193744s) [noise=true , address=145.79.227.10:4401 , piece_id=E3TWGH2FG7QBTIQMEZ3TVTHRQCRUPVJGF65RSPUPF3M6NKRNZHHA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.518271127 (139h27m56.219863578s) [noise=true , address=77.90.31.18:4805 , piece_id=VUQEVDQ3UPZFATAGOQ4QSMTCSWFPMBEK4MRZLF4VOKPLFPYIMMBA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.518768591 (139h27m56.220361047s) [noise=true , address=145.79.227.24:4582 , piece_id=SM2SI6H445LDLWKN4433SZ2JWNKW7VWCUZD6QI5DC72FFKXIZKGA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:54.140079499 (139h27m56.841671947s) [upload=122653]: Next returned nil segment Nov 5 07:08:54.140092295 (139h27m56.84168473s) [upload=122653]: Waiting for error group managing segments... Nov 5 07:08:54.275546767 (139h27m56.977139218s) [noise=true , address=77.90.31.18:4807 , piece_id=2OGMRA7RUWBJOKEN3MNZLTW6VIXPOLHRR3O5H3CVNT4AUEKWBXBQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.285318656 (139h27m56.98691112s) [noise=true , address=77.90.31.13:4628 , piece_id=L3WCZYDMEM2W2552M7LG2A5PBP4HU64JNQMCJAENRC46BUAOG3EQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.285563903 (139h27m56.987156362s) [noise=true , address=77.90.31.18:4805 , piece_id=VUQEVDQ3UPZFATAGOQ4QSMTCSWFPMBEK4MRZLF4VOKPLFPYIMMBA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.307243468 (139h27m57.008835919s) [noise=true , address=77.90.31.21:4879 , piece_id=DTTPJA34K7VYRBAGQ6LYIPKNR4ZUFWNU3XICCXFSIS5VCOCFRGRA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.311638805 (139h27m57.013231286s) [noise=true , address=77.90.31.14:4663 , piece_id=TBLQUWZALDARTQIBY3TYWHK24QQYSRM47NCDFDYI3ROL2P2WFIIA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.403790160 (139h27m57.105382594s) [noise=true , address=77.90.31.12:4590 , piece_id=DEMANEV6ZWO62EVVT6FXPJOTNYQLPJIC6ABQCTJCF2MPI2PF2MCA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:09:23.097144333 (139h28m25.7987368s) [noise=true , address=145.79.227.21:4475 , piece_id=42BQHCUFBK2SODY3PIVRYD4N3CQRXDZ6CHXZP3OM2BGRPYBDTI7Q, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:128SC59hAFBUVutPFym3iZv9qNYtgiqrk6q9RpfWL2ftyTt5Tzk, address:145.79.227.21:4475): protocol: rpc: dial tcp 145.79.227.21:4475: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097392075 (139h28m25.798984538s) [noise=true , address=145.79.227.27:4690 , piece_id=3ZJPN6QKPUH4RIVRTPOJOV6Y4FD4YKYQHZJGAV3CUNWOUTLUY3ZA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: rpc: dial tcp 145.79.227.27:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097504503 (139h28m25.799096966s) [noise=true , address=145.79.227.24:4582 , piece_id=SM2SI6H445LDLWKN4433SZ2JWNKW7VWCUZD6QI5DC72FFKXIZKGA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1YLhczXWKCwzCQxM97Q77HpiWNMe5E15Nv679FZo5uHw6MRXa, address:145.79.227.24:4582): protocol: rpc: dial tcp 145.79.227.24:4582: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097732944 (139h28m25.799325406s) [noise=true , address=145.79.227.32:4834 , piece_id=NIF4YDGRWHNM734KXYVCGWWHNQPVPZIWZD5UGJKEWGMKWUYN634A, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: rpc: dial tcp 145.79.227.32:4834: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097846391 (139h28m25.799438853s) [noise=true , address=145.79.227.13:4802 , piece_id=PQEQCX24FLF7WNZL6NVRSBH2E4IESGGMEESF77XLCL2DZXTML6DA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:145.79.227.13:4802): protocol: rpc: dial tcp 145.79.227.13:4802: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098108991 (139h28m25.799701452s) [noise=true , address=145.79.227.10:4401 , piece_id=E3TWGH2FG7QBTIQMEZ3TVTHRQCRUPVJGF65RSPUPF3M6NKRNZHHA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: rpc: dial tcp 145.79.227.10:4401: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098205549 (139h28m25.799798009s) [noise=true , address=145.79.227.15:4876 , piece_id=P65E3AEIWXVODXYXRLYI6CCUCL37W4LVPVZFE7RYG3L6KJ7SMJVQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: rpc: dial tcp 145.79.227.15:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098453405 (139h28m25.80004587s) [noise=true , address=145.79.227.18:4005 , piece_id=5AKQXEJY72OR6NAVFSE5S6EX2TQUURE6R4JCYFWOZNNULR5SS6FQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098482863 (139h28m25.800075324s) [noise=true , address=145.79.227.31:4797 , piece_id=4IXOFVR7EBRFFWAQKCLNWGE7J5ZARFXBJQ67A3H7FUEP2IQYKNBA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: rpc: dial tcp 145.79.227.31:4797: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098774913 (139h28m25.800367375s) [noise=true , address=145.79.227.14:4840 , piece_id=JVAE4ZXJ6MX4KFVAXDYKSXDYJ5EMNCGMBXRK3WVIPD6NGWB2GQDA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: rpc: dial tcp 145.79.227.14:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098823983 (139h28m25.800416425s) [seg_pos={0 0}, upload=122653]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Nov 5 07:09:23.210562291 (139h28m25.912154746s) [upload=122653]: Done uploading segments... Nov 5 07:09:23.210617929 (139h28m25.912210379s) [upload=122653]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration:* 2.267182876s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....XX......................................................] duration: 2.699644706s trace: https://jaeger.dev.storj.io/trace/49cd51d016f1caf5 FAIL (2025-11-05 07:46:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.699679064s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....XX.....................................................] duration: 2.758184644s trace: https://jaeger.dev.storj.io/trace/49cd51d016f1caf5 FAIL (2025-11-05 07:46:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.758208754s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 2.404565721s trace: https://jaeger.dev.storj.io/trace/73a29de9c809b554 FAIL (2025-11-05 07:46:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 53.391572844 (139h27m56.093165284s) [upload=122653]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 5 07:08:53.511496551 (139h27m56.213088992s) [seg_pos={0 0}, upload=122653]: Begin upload segment... Nov 5 07:08:53.511554279 (139h27m56.21314672s) [seg_pos={0 0}, upload=122653]: Done begin upload segment. Nov 5 07:08:53.511581616 (139h27m56.213174066s) [noise=true , address=145.79.227.13:4802 , piece_id=PQEQCX24FLF7WNZL6NVRSBH2E4IESGGMEESF77XLCL2DZXTML6DA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.513588552 (139h27m56.215181007s) [noise=true , address=145.79.227.18:4005 , piece_id=5AKQXEJY72OR6NAVFSE5S6EX2TQUURE6R4JCYFWOZNNULR5SS6FQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.513752850 (139h27m56.215345301s) [noise=true , address=77.90.31.13:4628 , piece_id=L3WCZYDMEM2W2552M7LG2A5PBP4HU64JNQMCJAENRC46BUAOG3EQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.514354806 (139h27m56.215947257s) [noise=true , address=77.90.31.18:4807 , piece_id=2OGMRA7RUWBJOKEN3MNZLTW6VIXPOLHRR3O5H3CVNT4AUEKWBXBQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.515244711 (139h27m56.216837159s) [noise=true , address=77.90.31.12:4590 , piece_id=DEMANEV6ZWO62EVVT6FXPJOTNYQLPJIC6ABQCTJCF2MPI2PF2MCA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.515784410 (139h27m56.217376865s) [noise=true , address=77.90.31.21:4879 , piece_id=DTTPJA34K7VYRBAGQ6LYIPKNR4ZUFWNU3XICCXFSIS5VCOCFRGRA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516021191 (139h27m56.217613644s) [noise=true , address=145.79.227.15:4876 , piece_id=P65E3AEIWXVODXYXRLYI6CCUCL37W4LVPVZFE7RYG3L6KJ7SMJVQ, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516273977 (139h27m56.217866426s) [noise=true , address=145.79.227.31:4797 , piece_id=4IXOFVR7EBRFFWAQKCLNWGE7J5ZARFXBJQ67A3H7FUEP2IQYKNBA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516440796 (139h27m56.218033244s) [noise=true , address=145.79.227.21:4475 , piece_id=42BQHCUFBK2SODY3PIVRYD4N3CQRXDZ6CHXZP3OM2BGRPYBDTI7Q, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516682821 (139h27m56.218275269s) [noise=true , address=145.79.227.14:4840 , piece_id=JVAE4ZXJ6MX4KFVAXDYKSXDYJ5EMNCGMBXRK3WVIPD6NGWB2GQDA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.516799834 (139h27m56.218392285s) [noise=true , address=77.90.31.14:4663 , piece_id=TBLQUWZALDARTQIBY3TYWHK24QQYSRM47NCDFDYI3ROL2P2WFIIA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.517334963 (139h27m56.218927412s) [noise=true , address=145.79.227.27:4690 , piece_id=3ZJPN6QKPUH4RIVRTPOJOV6Y4FD4YKYQHZJGAV3CUNWOUTLUY3ZA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.517377298 (139h27m56.21896976s) [noise=true , address=145.79.227.32:4834 , piece_id=NIF4YDGRWHNM734KXYVCGWWHNQPVPZIWZD5UGJKEWGMKWUYN634A, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.517781951 (139h27m56.2193744s) [noise=true , address=145.79.227.10:4401 , piece_id=E3TWGH2FG7QBTIQMEZ3TVTHRQCRUPVJGF65RSPUPF3M6NKRNZHHA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.518271127 (139h27m56.219863578s) [noise=true , address=77.90.31.18:4805 , piece_id=VUQEVDQ3UPZFATAGOQ4QSMTCSWFPMBEK4MRZLF4VOKPLFPYIMMBA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:53.518768591 (139h27m56.220361047s) [noise=true , address=145.79.227.24:4582 , piece_id=SM2SI6H445LDLWKN4433SZ2JWNKW7VWCUZD6QI5DC72FFKXIZKGA, seg_pos={0 0}, upload=122653]: Uploading piece... Nov 5 07:08:54.140079499 (139h27m56.841671947s) [upload=122653]: Next returned nil segment Nov 5 07:08:54.140092295 (139h27m56.84168473s) [upload=122653]: Waiting for error group managing segments... Nov 5 07:08:54.275546767 (139h27m56.977139218s) [noise=true , address=77.90.31.18:4807 , piece_id=2OGMRA7RUWBJOKEN3MNZLTW6VIXPOLHRR3O5H3CVNT4AUEKWBXBQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.285318656 (139h27m56.98691112s) [noise=true , address=77.90.31.13:4628 , piece_id=L3WCZYDMEM2W2552M7LG2A5PBP4HU64JNQMCJAENRC46BUAOG3EQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.285563903 (139h27m56.987156362s) [noise=true , address=77.90.31.18:4805 , piece_id=VUQEVDQ3UPZFATAGOQ4QSMTCSWFPMBEK4MRZLF4VOKPLFPYIMMBA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.307243468 (139h27m57.008835919s) [noise=true , address=77.90.31.21:4879 , piece_id=DTTPJA34K7VYRBAGQ6LYIPKNR4ZUFWNU3XICCXFSIS5VCOCFRGRA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.311638805 (139h27m57.013231286s) [noise=true , address=77.90.31.14:4663 , piece_id=TBLQUWZALDARTQIBY3TYWHK24QQYSRM47NCDFDYI3ROL2P2WFIIA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:08:54.403790160 (139h27m57.105382594s) [noise=true , address=77.90.31.12:4590 , piece_id=DEMANEV6ZWO62EVVT6FXPJOTNYQLPJIC6ABQCTJCF2MPI2PF2MCA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: <nil> Nov 5 07:09:23.097144333 (139h28m25.7987368s) [noise=true , address=145.79.227.21:4475 , piece_id=42BQHCUFBK2SODY3PIVRYD4N3CQRXDZ6CHXZP3OM2BGRPYBDTI7Q, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:128SC59hAFBUVutPFym3iZv9qNYtgiqrk6q9RpfWL2ftyTt5Tzk, address:145.79.227.21:4475): protocol: rpc: dial tcp 145.79.227.21:4475: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097392075 (139h28m25.798984538s) [noise=true , address=145.79.227.27:4690 , piece_id=3ZJPN6QKPUH4RIVRTPOJOV6Y4FD4YKYQHZJGAV3CUNWOUTLUY3ZA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H, address:145.79.227.27:4690): protocol: rpc: dial tcp 145.79.227.27:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097504503 (139h28m25.799096966s) [noise=true , address=145.79.227.24:4582 , piece_id=SM2SI6H445LDLWKN4433SZ2JWNKW7VWCUZD6QI5DC72FFKXIZKGA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1YLhczXWKCwzCQxM97Q77HpiWNMe5E15Nv679FZo5uHw6MRXa, address:145.79.227.24:4582): protocol: rpc: dial tcp 145.79.227.24:4582: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097732944 (139h28m25.799325406s) [noise=true , address=145.79.227.32:4834 , piece_id=NIF4YDGRWHNM734KXYVCGWWHNQPVPZIWZD5UGJKEWGMKWUYN634A, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: rpc: dial tcp 145.79.227.32:4834: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.097846391 (139h28m25.799438853s) [noise=true , address=145.79.227.13:4802 , piece_id=PQEQCX24FLF7WNZL6NVRSBH2E4IESGGMEESF77XLCL2DZXTML6DA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:145.79.227.13:4802): protocol: rpc: dial tcp 145.79.227.13:4802: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098108991 (139h28m25.799701452s) [noise=true , address=145.79.227.10:4401 , piece_id=E3TWGH2FG7QBTIQMEZ3TVTHRQCRUPVJGF65RSPUPF3M6NKRNZHHA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1, address:145.79.227.10:4401): protocol: rpc: dial tcp 145.79.227.10:4401: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098205549 (139h28m25.799798009s) [noise=true , address=145.79.227.15:4876 , piece_id=P65E3AEIWXVODXYXRLYI6CCUCL37W4LVPVZFE7RYG3L6KJ7SMJVQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: rpc: dial tcp 145.79.227.15:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098453405 (139h28m25.80004587s) [noise=true , address=145.79.227.18:4005 , piece_id=5AKQXEJY72OR6NAVFSE5S6EX2TQUURE6R4JCYFWOZNNULR5SS6FQ, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098482863 (139h28m25.800075324s) [noise=true , address=145.79.227.31:4797 , piece_id=4IXOFVR7EBRFFWAQKCLNWGE7J5ZARFXBJQ67A3H7FUEP2IQYKNBA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: rpc: dial tcp 145.79.227.31:4797: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098774913 (139h28m25.800367375s) [noise=true , address=145.79.227.14:4840 , piece_id=JVAE4ZXJ6MX4KFVAXDYKSXDYJ5EMNCGMBXRK3WVIPD6NGWB2GQDA, seg_pos={0 0}, upload=122653]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: rpc: dial tcp 145.79.227.14:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 5 07:09:23.098823983 (139h28m25.800416425s) [seg_pos={0 0}, upload=122653]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Nov 5 07:09:23.210562291 (139h28m25.912154746s) [upload=122653]: Done uploading segments... Nov 5 07:09:23.210617929 (139h28m25.912210379s) [upload=122653]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration:* 2.404592912s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)