Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [............................X.....................X.X.......] duration: 1.581307295s trace: https://jaeger.dev.storj.io/trace/722fac3a50dfa0f2 FAIL (2025-11-22 14:19:27 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.581353777s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [.............................X.....................X.X......] duration: 1.673961603s trace: https://jaeger.dev.storj.io/trace/722fac3a50dfa0f2 FAIL (2025-11-22 14:18:53 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.674044639s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [..............................X.....................X.X.....] duration: 1.637610317s trace: https://jaeger.dev.storj.io/trace/722fac3a50dfa0f2 FAIL (2025-11-22 14:17:55 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.637636957s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [...............................X.....................X.X....] duration: 1.617228371s trace: https://jaeger.dev.storj.io/trace/722fac3a50dfa0f2 FAIL (2025-11-22 14:16:44 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.617270205s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [................................X.....................X.X...] duration: 1.647398748s trace: https://jaeger.dev.storj.io/trace/722fac3a50dfa0f2 FAIL (2025-11-22 14:16:19 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.647480075s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [.................................X.....................X.X..] duration: 1.630380686s trace: https://jaeger.dev.storj.io/trace/722fac3a50dfa0f2 FAIL (2025-11-22 14:16:00 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.630411905s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [..................................X.....................X.X.] duration: 1.554147192s trace: https://jaeger.dev.storj.io/trace/722fac3a50dfa0f2 FAIL (2025-11-22 14:15:18 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.554177317s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [...................................X.....................X.X] duration: 5.001768372s trace: https://jaeger.dev.storj.io/trace/722fac3a50dfa0f2 FAIL (2025-11-22 14:14:51 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001969487s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [....................................X.....................X.] duration: 1.684060722s trace: https://jaeger.dev.storj.io/trace/719bc544fa18903e FAIL (2025-11-22 14:14:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.68408988s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.....................................X.....................X] duration: 5.000837713s trace: https://jaeger.dev.storj.io/trace/719bc544fa18903e FAIL (2025-11-22 14:13:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.000859503s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [......................................X.....................] duration: 3.555620182s trace: https://jaeger.dev.storj.io/trace/14a2f1be1b3db1b3 FAIL (2025-11-22 14:12:56 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.55564918s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.......................................X....................] duration: 3.635735141s trace: https://jaeger.dev.storj.io/trace/14a2f1be1b3db1b3 FAIL (2025-11-22 14:12:40 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.635759443s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [........................................X...................] duration: 3.442620494s trace: https://jaeger.dev.storj.io/trace/14a2f1be1b3db1b3 FAIL (2025-11-22 14:11:41 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.443593882s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.........................................X..................] duration: 3.281508042s trace: https://jaeger.dev.storj.io/trace/14a2f1be1b3db1b3 FAIL (2025-11-22 14:11:17 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.281532629s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..........................................X.................] duration: 4.477724072s trace: https://jaeger.dev.storj.io/trace/14a2f1be1b3db1b3 FAIL (2025-11-22 14:10:35 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.477746632s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...........................................X................] duration: 1.688760584s trace: https://jaeger.dev.storj.io/trace/14a2f1be1b3db1b3 FAIL (2025-11-22 14:10:15 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.688784212s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [............................................X...............] duration: 1.590561038s trace: https://jaeger.dev.storj.io/trace/14a2f1be1b3db1b3 FAIL (2025-11-22 14:09:28 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.590594106s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.............................................X..............] duration: 1.698816569s trace: https://jaeger.dev.storj.io/trace/14a2f1be1b3db1b3 FAIL (2025-11-22 14:08:36 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.698840239s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..............................................X.............] duration: 1.574237949s trace: https://jaeger.dev.storj.io/trace/14a2f1be1b3db1b3 FAIL (2025-11-22 14:07:27 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.574336105s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...............................................X............] duration: 4.404572971s trace: https://jaeger.dev.storj.io/trace/14a2f1be1b3db1b3 FAIL (2025-11-22 14:06:30 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.404609696s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [................................................X...........] duration: 3.560090121s trace: https://jaeger.dev.storj.io/trace/14a2f1be1b3db1b3 FAIL (2025-11-22 14:05:45 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.560182031s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.................................................X..........] duration: 2.602760858s trace: https://jaeger.dev.storj.io/trace/14a2f1be1b3db1b3 FAIL (2025-11-22 14:04:44 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.602786993s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..................................................X.........] duration: 2.740637969s trace: https://jaeger.dev.storj.io/trace/14a2f1be1b3db1b3 FAIL (2025-11-22 14:04:22 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.74066388s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...................................................X........] duration: 4.35809242s trace: https://jaeger.dev.storj.io/trace/14a2f1be1b3db1b3 FAIL (2025-11-22 14:03:14 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.358116033s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [....................................................X.......] duration: 2.237268378s trace: https://jaeger.dev.storj.io/trace/14a2f1be1b3db1b3 FAIL (2025-11-22 14:02:32 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.237300455s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.....................................................X......] duration: 3.788805292s trace: https://jaeger.dev.storj.io/trace/14a2f1be1b3db1b3 FAIL (2025-11-22 14:01:56 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.788828918s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [......................................................X.....] duration: 3.153419572s trace: https://jaeger.dev.storj.io/trace/14a2f1be1b3db1b3 FAIL (2025-11-22 14:01:09 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.153457012s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.......................................................X....] duration: 4.757344214s trace: https://jaeger.dev.storj.io/trace/14a2f1be1b3db1b3 FAIL (2025-11-22 14:00:35 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.757366808s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [........................................................X...] duration: 4.04256084s trace: https://jaeger.dev.storj.io/trace/14a2f1be1b3db1b3 FAIL (2025-11-22 14:00:19 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.042588s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.........................................................X..] duration: 1.7616469s trace: https://jaeger.dev.storj.io/trace/14a2f1be1b3db1b3 FAIL (2025-11-22 13:59:12 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.761668668s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..........................................................X.] duration: 4.825955247s trace: https://jaeger.dev.storj.io/trace/14a2f1be1b3db1b3 FAIL (2025-11-22 13:58:52 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.825979877s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...........................................................X] duration: 5.00165653s trace: https://jaeger.dev.storj.io/trace/14a2f1be1b3db1b3 FAIL (2025-11-22 13:58:08 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001672896s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.284569335s trace: https://jaeger.dev.storj.io/trace/5656a0c7e0b53cda FAIL (2025-11-22 13:17:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Nov 22 12:30:52.003787021 (266h46m37.362445675s) [upload=235241]: Starting upload Nov 22 12:30:52.003885048 (266h46m37.362543729s) [upload=235241]: Done starting upload Nov 22 12:30:52.003977160 (266h46m37.362635814s) [upload=235241]: Uploading segments... Nov 22 12:30:52.004140730 (266h46m37.362799384s) [upload=235241]: Got next segment. Inline: false Nov 22 12:30:52.004176073 (266h46m37.362834729s) [upload=235241]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 22 12:30:52.004181703 (266h46m37.362840356s) [upload=235241]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 22 12:30:52.113539293 (266h46m37.472197949s) [seg_pos={0 0}, upload=235241]: Begin upload segment... Nov 22 12:30:52.113592796 (266h46m37.472251458s) [seg_pos={0 0}, upload=235241]: Done begin upload segment. Nov 22 12:30:52.113622689 (266h46m37.472281343s) [noise=true , address=77.90.31.21:4877 , piece_id=AAKMO3BX7T6ISQFVZKNSL374X6PMRPBGWCMXW2NIAUILGWN2Y6WQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.113670993 (266h46m37.472329655s) [noise=true , address=145.79.227.25:4619 , piece_id=4KKZOITVXGP7TQ5Q3WGKU5IOSY7CBDX4YHNKUD2TAFRFCNDDFWLA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.114132816 (266h46m37.472791473s) [noise=true , address=145.79.227.11:4730 , piece_id=A6MKVCMMOEDQCNVLZVLQE7727WRCTWSQESD5OTLYMEXXPGZXULYA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.114351061 (266h46m37.473009726s) [noise=true , address=145.79.227.22:4509 , piece_id=WZ556AW2TRVM3GC2DQEFDSBXOGTYGDBH424ZLBFOBB5NKAT3JUKA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.114514284 (266h46m37.47317294s) [noise=true , address=145.79.227.27:4691 , piece_id=HJV42J2TB2Y5SXVDTKJS265UQ34MAPMNAH2ZWH3NUXBWLXHWOOYQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.114801466 (266h46m37.473460129s) [noise=true , address=145.79.227.31:4799 , piece_id=6J5APHQR2STIABAKSUZQEBPSR6XIVVK2LXNDU6JDMIBCJRGT54GA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.114835097 (266h46m37.473493753s) [noise=true , address=145.79.227.17:4970 , piece_id=VGCXEGCG2ZJSGHCPUW6TEJC33G3PT4OGK2WG6PCW6FBYUOH5Q6LA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.115174456 (266h46m37.47383311s) [noise=true , address=145.79.227.22:4510 , piece_id=GEQO4OVLPB3CWCYVOCAWMP7WUYVRIWNNICOUSTHHLKWXNQL46HLQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.115233219 (266h46m37.473891879s) [noise=true , address=145.79.227.28:4726 , piece_id=EHG25AF4CZ3UMVOVG6TSX3TFAXMTFZND4INOLLX5IZUM3744FQQA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.115559987 (266h46m37.474218641s) [noise=true , address=45.91.116.6:4975 , piece_id=CVSCYXLALCDU5O4NPWK4T73Q5DZFFM4YA6EZMNGSCYRRYLOGAGSA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.115737038 (266h46m37.474395697s) [noise=true , address=145.79.227.10:4402 , piece_id=2QSJODIK6I6QYQXATBBLAL5SDGB7O3H5OO4GLRQAGUCAXHI337VA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.115831971 (266h46m37.474490611s) [noise=true , address=77.90.31.18:4806 , piece_id=RSJWLFHBYMTJO47QF7ZK3ZE2GV7Z2B2T3NQDMH644VRFUQAJCYVA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.116101068 (266h46m37.474759721s) [noise=true , address=145.79.227.24:4581 , piece_id=7Y32DQ7BHOPIZWX7STPG2YHOWBY5Q3XW74S7D2U25POIKVKFYIUQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.116130009 (266h46m37.474788662s) [noise=true , address=45.91.116.11:4117 , piece_id=VJ2G5TGE46SVXX574ZGMRQUER6CJUYDHB47BNVWNPBIFDFFNKZBQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.116505131 (266h46m37.475163791s) [noise=true , address=145.79.227.15:4875 , piece_id=RN4TLLBVGZQ2LZZES36O4XGTPSZLU4EYMA5VZ7DZBP46H342C64A, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.116936492 (266h46m37.47559515s) [noise=true , address=45.91.116.4:4939 , piece_id=CY72GWLHSUZJ6JL2O57SYBNYYTKC4CMT3EKX4NWFE3JWWXB7YVLQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.720203965 (266h46m38.078862618s) [upload=235241]: Next returned nil segment Nov 22 12:30:52.720216190 (266h46m38.078874829s) [upload=235241]: Waiting for error group managing segments... Nov 22 12:30:52.859097543 (266h46m38.2177562s) [noise=true , address=145.79.227.28:4726 , piece_id=EHG25AF4CZ3UMVOVG6TSX3TFAXMTFZND4INOLLX5IZUM3744FQQA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.859631290 (266h46m38.218289947s) [noise=true , address=145.79.227.27:4691 , piece_id=HJV42J2TB2Y5SXVDTKJS265UQ34MAPMNAH2ZWH3NUXBWLXHWOOYQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.859726981 (266h46m38.21838564s) [noise=true , address=145.79.227.31:4799 , piece_id=6J5APHQR2STIABAKSUZQEBPSR6XIVVK2LXNDU6JDMIBCJRGT54GA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.861545214 (266h46m38.22020387s) [noise=true , address=145.79.227.10:4402 , piece_id=2QSJODIK6I6QYQXATBBLAL5SDGB7O3H5OO4GLRQAGUCAXHI337VA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.864774401 (266h46m38.223433064s) [noise=true , address=145.79.227.25:4619 , piece_id=4KKZOITVXGP7TQ5Q3WGKU5IOSY7CBDX4YHNKUD2TAFRFCNDDFWLA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.868389753 (266h46m38.22704841s) [noise=true , address=145.79.227.17:4970 , piece_id=VGCXEGCG2ZJSGHCPUW6TEJC33G3PT4OGK2WG6PCW6FBYUOH5Q6LA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.871585428 (266h46m38.230244094s) [noise=true , address=145.79.227.11:4730 , piece_id=A6MKVCMMOEDQCNVLZVLQE7727WRCTWSQESD5OTLYMEXXPGZXULYA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.873727410 (266h46m38.232386067s) [noise=true , address=145.79.227.22:4509 , piece_id=WZ556AW2TRVM3GC2DQEFDSBXOGTYGDBH424ZLBFOBB5NKAT3JUKA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.875573701 (266h46m38.234232366s) [noise=true , address=145.79.227.22:4510 , piece_id=GEQO4OVLPB3CWCYVOCAWMP7WUYVRIWNNICOUSTHHLKWXNQL46HLQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.877964136 (266h46m38.236622776s) [noise=true , address=145.79.227.15:4875 , piece_id=RN4TLLBVGZQ2LZZES36O4XGTPSZLU4EYMA5VZ7DZBP46H342C64A, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.884853383 (266h46m38.243512048s) [noise=true , address=145.79.227.24:4581 , piece_id=7Y32DQ7BHOPIZWX7STPG2YHOWBY5Q3XW74S7D2U25POIKVKFYIUQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:31:21.718054035 (266h47m7.076712695s) [noise=true , address=77.90.31.21:4877 , piece_id=AAKMO3BX7T6ISQFVZKNSL374X6PMRPBGWCMXW2NIAUILGWN2Y6WQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: write tcp 139.59.144.22:44008->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 22 12:31:21.718401995 (266h47m7.077060726s) [noise=true , address=77.90.31.18:4806 , piece_id=RSJWLFHBYMTJO47QF7ZK3ZE2GV7Z2B2T3NQDMH644VRFUQAJCYVA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 22 12:31:21.718466899 (266h47m7.077125546s) [noise=true , address=45.91.116.4:4939 , piece_id=CY72GWLHSUZJ6JL2O57SYBNYYTKC4CMT3EKX4NWFE3JWWXB7YVLQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:49860->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 22 12:31:21.718639920 (266h47m7.07729858s) [noise=true , address=45.91.116.6:4975 , piece_id=CVSCYXLALCDU5O4NPWK4T73Q5DZFFM4YA6EZMNGSCYRRYLOGAGSA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: context deadline exceeded; context deadline exceeded Nov 22 12:31:21.718670710 (266h47m7.077329363s) [noise=true , address=45.91.116.11:4117 , piece_id=VJ2G5TGE46SVXX574ZGMRQUER6CJUYDHB47BNVWNPBIFDFFNKZBQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:40600->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 22 12:31:21.718696534 (266h47m7.077355177s) [seg_pos={0 0}, upload=235241]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Nov 22 12:31:21.821464823 (266h47m7.180123476s) [upload=235241]: Done uploading segments... Nov 22 12:31:21.821497872 (266h47m7.180156526s) [upload=235241]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.284619336s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [X...........................................................] duration: 1.568644598s trace: https://jaeger.dev.storj.io/trace/5bcba4939824f50a FAIL (2025-11-22 13:17:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.568736772s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [XX..........................................................] duration: 1.63904616s trace: https://jaeger.dev.storj.io/trace/5bcba4939824f50a FAIL (2025-11-22 13:16:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.639068874s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 2.311148926s trace: https://jaeger.dev.storj.io/trace/5656a0c7e0b53cda FAIL (2025-11-22 13:16:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Nov 22 12:30:52.003787021 (266h46m37.362445675s) [upload=235241]: Starting upload Nov 22 12:30:52.003885048 (266h46m37.362543729s) [upload=235241]: Done starting upload Nov 22 12:30:52.003977160 (266h46m37.362635814s) [upload=235241]: Uploading segments... Nov 22 12:30:52.004140730 (266h46m37.362799384s) [upload=235241]: Got next segment. Inline: false Nov 22 12:30:52.004176073 (266h46m37.362834729s) [upload=235241]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 22 12:30:52.004181703 (266h46m37.362840356s) [upload=235241]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 22 12:30:52.113539293 (266h46m37.472197949s) [seg_pos={0 0}, upload=235241]: Begin upload segment... Nov 22 12:30:52.113592796 (266h46m37.472251458s) [seg_pos={0 0}, upload=235241]: Done begin upload segment. Nov 22 12:30:52.113622689 (266h46m37.472281343s) [noise=true , address=77.90.31.21:4877 , piece_id=AAKMO3BX7T6ISQFVZKNSL374X6PMRPBGWCMXW2NIAUILGWN2Y6WQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.113670993 (266h46m37.472329655s) [noise=true , address=145.79.227.25:4619 , piece_id=4KKZOITVXGP7TQ5Q3WGKU5IOSY7CBDX4YHNKUD2TAFRFCNDDFWLA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.114132816 (266h46m37.472791473s) [noise=true , address=145.79.227.11:4730 , piece_id=A6MKVCMMOEDQCNVLZVLQE7727WRCTWSQESD5OTLYMEXXPGZXULYA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.114351061 (266h46m37.473009726s) [noise=true , address=145.79.227.22:4509 , piece_id=WZ556AW2TRVM3GC2DQEFDSBXOGTYGDBH424ZLBFOBB5NKAT3JUKA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.114514284 (266h46m37.47317294s) [noise=true , address=145.79.227.27:4691 , piece_id=HJV42J2TB2Y5SXVDTKJS265UQ34MAPMNAH2ZWH3NUXBWLXHWOOYQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.114801466 (266h46m37.473460129s) [noise=true , address=145.79.227.31:4799 , piece_id=6J5APHQR2STIABAKSUZQEBPSR6XIVVK2LXNDU6JDMIBCJRGT54GA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.114835097 (266h46m37.473493753s) [noise=true , address=145.79.227.17:4970 , piece_id=VGCXEGCG2ZJSGHCPUW6TEJC33G3PT4OGK2WG6PCW6FBYUOH5Q6LA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.115174456 (266h46m37.47383311s) [noise=true , address=145.79.227.22:4510 , piece_id=GEQO4OVLPB3CWCYVOCAWMP7WUYVRIWNNICOUSTHHLKWXNQL46HLQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.115233219 (266h46m37.473891879s) [noise=true , address=145.79.227.28:4726 , piece_id=EHG25AF4CZ3UMVOVG6TSX3TFAXMTFZND4INOLLX5IZUM3744FQQA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.115559987 (266h46m37.474218641s) [noise=true , address=45.91.116.6:4975 , piece_id=CVSCYXLALCDU5O4NPWK4T73Q5DZFFM4YA6EZMNGSCYRRYLOGAGSA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.115737038 (266h46m37.474395697s) [noise=true , address=145.79.227.10:4402 , piece_id=2QSJODIK6I6QYQXATBBLAL5SDGB7O3H5OO4GLRQAGUCAXHI337VA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.115831971 (266h46m37.474490611s) [noise=true , address=77.90.31.18:4806 , piece_id=RSJWLFHBYMTJO47QF7ZK3ZE2GV7Z2B2T3NQDMH644VRFUQAJCYVA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.116101068 (266h46m37.474759721s) [noise=true , address=145.79.227.24:4581 , piece_id=7Y32DQ7BHOPIZWX7STPG2YHOWBY5Q3XW74S7D2U25POIKVKFYIUQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.116130009 (266h46m37.474788662s) [noise=true , address=45.91.116.11:4117 , piece_id=VJ2G5TGE46SVXX574ZGMRQUER6CJUYDHB47BNVWNPBIFDFFNKZBQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.116505131 (266h46m37.475163791s) [noise=true , address=145.79.227.15:4875 , piece_id=RN4TLLBVGZQ2LZZES36O4XGTPSZLU4EYMA5VZ7DZBP46H342C64A, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.116936492 (266h46m37.47559515s) [noise=true , address=45.91.116.4:4939 , piece_id=CY72GWLHSUZJ6JL2O57SYBNYYTKC4CMT3EKX4NWFE3JWWXB7YVLQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.720203965 (266h46m38.078862618s) [upload=235241]: Next returned nil segment Nov 22 12:30:52.720216190 (266h46m38.078874829s) [upload=235241]: Waiting for error group managing segments... Nov 22 12:30:52.859097543 (266h46m38.2177562s) [noise=true , address=145.79.227.28:4726 , piece_id=EHG25AF4CZ3UMVOVG6TSX3TFAXMTFZND4INOLLX5IZUM3744FQQA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.859631290 (266h46m38.218289947s) [noise=true , address=145.79.227.27:4691 , piece_id=HJV42J2TB2Y5SXVDTKJS265UQ34MAPMNAH2ZWH3NUXBWLXHWOOYQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.859726981 (266h46m38.21838564s) [noise=true , address=145.79.227.31:4799 , piece_id=6J5APHQR2STIABAKSUZQEBPSR6XIVVK2LXNDU6JDMIBCJRGT54GA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.861545214 (266h46m38.22020387s) [noise=true , address=145.79.227.10:4402 , piece_id=2QSJODIK6I6QYQXATBBLAL5SDGB7O3H5OO4GLRQAGUCAXHI337VA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.864774401 (266h46m38.223433064s) [noise=true , address=145.79.227.25:4619 , piece_id=4KKZOITVXGP7TQ5Q3WGKU5IOSY7CBDX4YHNKUD2TAFRFCNDDFWLA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.868389753 (266h46m38.22704841s) [noise=true , address=145.79.227.17:4970 , piece_id=VGCXEGCG2ZJSGHCPUW6TEJC33G3PT4OGK2WG6PCW6FBYUOH5Q6LA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.871585428 (266h46m38.230244094s) [noise=true , address=145.79.227.11:4730 , piece_id=A6MKVCMMOEDQCNVLZVLQE7727WRCTWSQESD5OTLYMEXXPGZXULYA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.873727410 (266h46m38.232386067s) [noise=true , address=145.79.227.22:4509 , piece_id=WZ556AW2TRVM3GC2DQEFDSBXOGTYGDBH424ZLBFOBB5NKAT3JUKA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.875573701 (266h46m38.234232366s) [noise=true , address=145.79.227.22:4510 , piece_id=GEQO4OVLPB3CWCYVOCAWMP7WUYVRIWNNICOUSTHHLKWXNQL46HLQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.877964136 (266h46m38.236622776s) [noise=true , address=145.79.227.15:4875 , piece_id=RN4TLLBVGZQ2LZZES36O4XGTPSZLU4EYMA5VZ7DZBP46H342C64A, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.884853383 (266h46m38.243512048s) [noise=true , address=145.79.227.24:4581 , piece_id=7Y32DQ7BHOPIZWX7STPG2YHOWBY5Q3XW74S7D2U25POIKVKFYIUQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:31:21.718054035 (266h47m7.076712695s) [noise=true , address=77.90.31.21:4877 , piece_id=AAKMO3BX7T6ISQFVZKNSL374X6PMRPBGWCMXW2NIAUILGWN2Y6WQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: write tcp 139.59.144.22:44008->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 22 12:31:21.718401995 (266h47m7.077060726s) [noise=true , address=77.90.31.18:4806 , piece_id=RSJWLFHBYMTJO47QF7ZK3ZE2GV7Z2B2T3NQDMH644VRFUQAJCYVA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 22 12:31:21.718466899 (266h47m7.077125546s) [noise=true , address=45.91.116.4:4939 , piece_id=CY72GWLHSUZJ6JL2O57SYBNYYTKC4CMT3EKX4NWFE3JWWXB7YVLQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:49860->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 22 12:31:21.718639920 (266h47m7.07729858s) [noise=true , address=45.91.116.6:4975 , piece_id=CVSCYXLALCDU5O4NPWK4T73Q5DZFFM4YA6EZMNGSCYRRYLOGAGSA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: context deadline exceeded; context deadline exceeded Nov 22 12:31:21.718670710 (266h47m7.077329363s) [noise=true , address=45.91.116.11:4117 , piece_id=VJ2G5TGE46SVXX574ZGMRQUER6CJUYDHB47BNVWNPBIFDFFNKZBQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:40600->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 22 12:31:21.718696534 (266h47m7.077355177s) [seg_pos={0 0}, upload=235241]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Nov 22 12:31:21.821464823 (266h47m7.180123476s) [upload=235241]: Done uploading segments... Nov 22 12:31:21.821497872 (266h47m7.180156526s) [upload=235241]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.311181779s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 2.262676408s trace: https://jaeger.dev.storj.io/trace/5656a0c7e0b53cda FAIL (2025-11-22 13:16:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Nov 22 12:30:52.003787021 (266h46m37.362445675s) [upload=235241]: Starting upload Nov 22 12:30:52.003885048 (266h46m37.362543729s) [upload=235241]: Done starting upload Nov 22 12:30:52.003977160 (266h46m37.362635814s) [upload=235241]: Uploading segments... Nov 22 12:30:52.004140730 (266h46m37.362799384s) [upload=235241]: Got next segment. Inline: false Nov 22 12:30:52.004176073 (266h46m37.362834729s) [upload=235241]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 22 12:30:52.004181703 (266h46m37.362840356s) [upload=235241]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 22 12:30:52.113539293 (266h46m37.472197949s) [seg_pos={0 0}, upload=235241]: Begin upload segment... Nov 22 12:30:52.113592796 (266h46m37.472251458s) [seg_pos={0 0}, upload=235241]: Done begin upload segment. Nov 22 12:30:52.113622689 (266h46m37.472281343s) [noise=true , address=77.90.31.21:4877 , piece_id=AAKMO3BX7T6ISQFVZKNSL374X6PMRPBGWCMXW2NIAUILGWN2Y6WQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.113670993 (266h46m37.472329655s) [noise=true , address=145.79.227.25:4619 , piece_id=4KKZOITVXGP7TQ5Q3WGKU5IOSY7CBDX4YHNKUD2TAFRFCNDDFWLA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.114132816 (266h46m37.472791473s) [noise=true , address=145.79.227.11:4730 , piece_id=A6MKVCMMOEDQCNVLZVLQE7727WRCTWSQESD5OTLYMEXXPGZXULYA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.114351061 (266h46m37.473009726s) [noise=true , address=145.79.227.22:4509 , piece_id=WZ556AW2TRVM3GC2DQEFDSBXOGTYGDBH424ZLBFOBB5NKAT3JUKA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.114514284 (266h46m37.47317294s) [noise=true , address=145.79.227.27:4691 , piece_id=HJV42J2TB2Y5SXVDTKJS265UQ34MAPMNAH2ZWH3NUXBWLXHWOOYQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.114801466 (266h46m37.473460129s) [noise=true , address=145.79.227.31:4799 , piece_id=6J5APHQR2STIABAKSUZQEBPSR6XIVVK2LXNDU6JDMIBCJRGT54GA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.114835097 (266h46m37.473493753s) [noise=true , address=145.79.227.17:4970 , piece_id=VGCXEGCG2ZJSGHCPUW6TEJC33G3PT4OGK2WG6PCW6FBYUOH5Q6LA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.115174456 (266h46m37.47383311s) [noise=true , address=145.79.227.22:4510 , piece_id=GEQO4OVLPB3CWCYVOCAWMP7WUYVRIWNNICOUSTHHLKWXNQL46HLQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.115233219 (266h46m37.473891879s) [noise=true , address=145.79.227.28:4726 , piece_id=EHG25AF4CZ3UMVOVG6TSX3TFAXMTFZND4INOLLX5IZUM3744FQQA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.115559987 (266h46m37.474218641s) [noise=true , address=45.91.116.6:4975 , piece_id=CVSCYXLALCDU5O4NPWK4T73Q5DZFFM4YA6EZMNGSCYRRYLOGAGSA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.115737038 (266h46m37.474395697s) [noise=true , address=145.79.227.10:4402 , piece_id=2QSJODIK6I6QYQXATBBLAL5SDGB7O3H5OO4GLRQAGUCAXHI337VA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.115831971 (266h46m37.474490611s) [noise=true , address=77.90.31.18:4806 , piece_id=RSJWLFHBYMTJO47QF7ZK3ZE2GV7Z2B2T3NQDMH644VRFUQAJCYVA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.116101068 (266h46m37.474759721s) [noise=true , address=145.79.227.24:4581 , piece_id=7Y32DQ7BHOPIZWX7STPG2YHOWBY5Q3XW74S7D2U25POIKVKFYIUQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.116130009 (266h46m37.474788662s) [noise=true , address=45.91.116.11:4117 , piece_id=VJ2G5TGE46SVXX574ZGMRQUER6CJUYDHB47BNVWNPBIFDFFNKZBQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.116505131 (266h46m37.475163791s) [noise=true , address=145.79.227.15:4875 , piece_id=RN4TLLBVGZQ2LZZES36O4XGTPSZLU4EYMA5VZ7DZBP46H342C64A, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.116936492 (266h46m37.47559515s) [noise=true , address=45.91.116.4:4939 , piece_id=CY72GWLHSUZJ6JL2O57SYBNYYTKC4CMT3EKX4NWFE3JWWXB7YVLQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.720203965 (266h46m38.078862618s) [upload=235241]: Next returned nil segment Nov 22 12:30:52.720216190 (266h46m38.078874829s) [upload=235241]: Waiting for error group managing segments... Nov 22 12:30:52.859097543 (266h46m38.2177562s) [noise=true , address=145.79.227.28:4726 , piece_id=EHG25AF4CZ3UMVOVG6TSX3TFAXMTFZND4INOLLX5IZUM3744FQQA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.859631290 (266h46m38.218289947s) [noise=true , address=145.79.227.27:4691 , piece_id=HJV42J2TB2Y5SXVDTKJS265UQ34MAPMNAH2ZWH3NUXBWLXHWOOYQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.859726981 (266h46m38.21838564s) [noise=true , address=145.79.227.31:4799 , piece_id=6J5APHQR2STIABAKSUZQEBPSR6XIVVK2LXNDU6JDMIBCJRGT54GA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.861545214 (266h46m38.22020387s) [noise=true , address=145.79.227.10:4402 , piece_id=2QSJODIK6I6QYQXATBBLAL5SDGB7O3H5OO4GLRQAGUCAXHI337VA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.864774401 (266h46m38.223433064s) [noise=true , address=145.79.227.25:4619 , piece_id=4KKZOITVXGP7TQ5Q3WGKU5IOSY7CBDX4YHNKUD2TAFRFCNDDFWLA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.868389753 (266h46m38.22704841s) [noise=true , address=145.79.227.17:4970 , piece_id=VGCXEGCG2ZJSGHCPUW6TEJC33G3PT4OGK2WG6PCW6FBYUOH5Q6LA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.871585428 (266h46m38.230244094s) [noise=true , address=145.79.227.11:4730 , piece_id=A6MKVCMMOEDQCNVLZVLQE7727WRCTWSQESD5OTLYMEXXPGZXULYA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.873727410 (266h46m38.232386067s) [noise=true , address=145.79.227.22:4509 , piece_id=WZ556AW2TRVM3GC2DQEFDSBXOGTYGDBH424ZLBFOBB5NKAT3JUKA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.875573701 (266h46m38.234232366s) [noise=true , address=145.79.227.22:4510 , piece_id=GEQO4OVLPB3CWCYVOCAWMP7WUYVRIWNNICOUSTHHLKWXNQL46HLQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.877964136 (266h46m38.236622776s) [noise=true , address=145.79.227.15:4875 , piece_id=RN4TLLBVGZQ2LZZES36O4XGTPSZLU4EYMA5VZ7DZBP46H342C64A, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.884853383 (266h46m38.243512048s) [noise=true , address=145.79.227.24:4581 , piece_id=7Y32DQ7BHOPIZWX7STPG2YHOWBY5Q3XW74S7D2U25POIKVKFYIUQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:31:21.718054035 (266h47m7.076712695s) [noise=true , address=77.90.31.21:4877 , piece_id=AAKMO3BX7T6ISQFVZKNSL374X6PMRPBGWCMXW2NIAUILGWN2Y6WQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: write tcp 139.59.144.22:44008->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 22 12:31:21.718401995 (266h47m7.077060726s) [noise=true , address=77.90.31.18:4806 , piece_id=RSJWLFHBYMTJO47QF7ZK3ZE2GV7Z2B2T3NQDMH644VRFUQAJCYVA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 22 12:31:21.718466899 (266h47m7.077125546s) [noise=true , address=45.91.116.4:4939 , piece_id=CY72GWLHSUZJ6JL2O57SYBNYYTKC4CMT3EKX4NWFE3JWWXB7YVLQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:49860->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 22 12:31:21.718639920 (266h47m7.07729858s) [noise=true , address=45.91.116.6:4975 , piece_id=CVSCYXLALCDU5O4NPWK4T73Q5DZFFM4YA6EZMNGSCYRRYLOGAGSA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: context deadline exceeded; context deadline exceeded Nov 22 12:31:21.718670710 (266h47m7.077329363s) [noise=true , address=45.91.116.11:4117 , piece_id=VJ2G5TGE46SVXX574ZGMRQUER6CJUYDHB47BNVWNPBIFDFFNKZBQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:40600->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 22 12:31:21.718696534 (266h47m7.077355177s) [seg_pos={0 0}, upload=235241]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Nov 22 12:31:21.821464823 (266h47m7.180123476s) [upload=235241]: Done uploading segments... Nov 22 12:31:21.821497872 (266h47m7.180156526s) [upload=235241]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.262771565s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.485558313s trace: https://jaeger.dev.storj.io/trace/220a09ca74d9d1c8 FAIL (2025-11-22 13:15:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.485645936s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.XX.........................................................] duration: 1.586385532s trace: https://jaeger.dev.storj.io/trace/5bcba4939824f50a FAIL (2025-11-22 13:15:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.586408467s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [X..X........................................................] duration: 2.222735605s trace: https://jaeger.dev.storj.io/trace/5656a0c7e0b53cda FAIL (2025-11-22 13:15:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Nov 22 12:30:52.003787021 (266h46m37.362445675s) [upload=235241]: Starting upload Nov 22 12:30:52.003885048 (266h46m37.362543729s) [upload=235241]: Done starting upload Nov 22 12:30:52.003977160 (266h46m37.362635814s) [upload=235241]: Uploading segments... Nov 22 12:30:52.004140730 (266h46m37.362799384s) [upload=235241]: Got next segment. Inline: false Nov 22 12:30:52.004176073 (266h46m37.362834729s) [upload=235241]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 22 12:30:52.004181703 (266h46m37.362840356s) [upload=235241]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 22 12:30:52.113539293 (266h46m37.472197949s) [seg_pos={0 0}, upload=235241]: Begin upload segment... Nov 22 12:30:52.113592796 (266h46m37.472251458s) [seg_pos={0 0}, upload=235241]: Done begin upload segment. Nov 22 12:30:52.113622689 (266h46m37.472281343s) [noise=true , address=77.90.31.21:4877 , piece_id=AAKMO3BX7T6ISQFVZKNSL374X6PMRPBGWCMXW2NIAUILGWN2Y6WQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.113670993 (266h46m37.472329655s) [noise=true , address=145.79.227.25:4619 , piece_id=4KKZOITVXGP7TQ5Q3WGKU5IOSY7CBDX4YHNKUD2TAFRFCNDDFWLA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.114132816 (266h46m37.472791473s) [noise=true , address=145.79.227.11:4730 , piece_id=A6MKVCMMOEDQCNVLZVLQE7727WRCTWSQESD5OTLYMEXXPGZXULYA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.114351061 (266h46m37.473009726s) [noise=true , address=145.79.227.22:4509 , piece_id=WZ556AW2TRVM3GC2DQEFDSBXOGTYGDBH424ZLBFOBB5NKAT3JUKA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.114514284 (266h46m37.47317294s) [noise=true , address=145.79.227.27:4691 , piece_id=HJV42J2TB2Y5SXVDTKJS265UQ34MAPMNAH2ZWH3NUXBWLXHWOOYQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.114801466 (266h46m37.473460129s) [noise=true , address=145.79.227.31:4799 , piece_id=6J5APHQR2STIABAKSUZQEBPSR6XIVVK2LXNDU6JDMIBCJRGT54GA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.114835097 (266h46m37.473493753s) [noise=true , address=145.79.227.17:4970 , piece_id=VGCXEGCG2ZJSGHCPUW6TEJC33G3PT4OGK2WG6PCW6FBYUOH5Q6LA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.115174456 (266h46m37.47383311s) [noise=true , address=145.79.227.22:4510 , piece_id=GEQO4OVLPB3CWCYVOCAWMP7WUYVRIWNNICOUSTHHLKWXNQL46HLQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.115233219 (266h46m37.473891879s) [noise=true , address=145.79.227.28:4726 , piece_id=EHG25AF4CZ3UMVOVG6TSX3TFAXMTFZND4INOLLX5IZUM3744FQQA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.115559987 (266h46m37.474218641s) [noise=true , address=45.91.116.6:4975 , piece_id=CVSCYXLALCDU5O4NPWK4T73Q5DZFFM4YA6EZMNGSCYRRYLOGAGSA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.115737038 (266h46m37.474395697s) [noise=true , address=145.79.227.10:4402 , piece_id=2QSJODIK6I6QYQXATBBLAL5SDGB7O3H5OO4GLRQAGUCAXHI337VA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.115831971 (266h46m37.474490611s) [noise=true , address=77.90.31.18:4806 , piece_id=RSJWLFHBYMTJO47QF7ZK3ZE2GV7Z2B2T3NQDMH644VRFUQAJCYVA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.116101068 (266h46m37.474759721s) [noise=true , address=145.79.227.24:4581 , piece_id=7Y32DQ7BHOPIZWX7STPG2YHOWBY5Q3XW74S7D2U25POIKVKFYIUQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.116130009 (266h46m37.474788662s) [noise=true , address=45.91.116.11:4117 , piece_id=VJ2G5TGE46SVXX574ZGMRQUER6CJUYDHB47BNVWNPBIFDFFNKZBQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.116505131 (266h46m37.475163791s) [noise=true , address=145.79.227.15:4875 , piece_id=RN4TLLBVGZQ2LZZES36O4XGTPSZLU4EYMA5VZ7DZBP46H342C64A, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.116936492 (266h46m37.47559515s) [noise=true , address=45.91.116.4:4939 , piece_id=CY72GWLHSUZJ6JL2O57SYBNYYTKC4CMT3EKX4NWFE3JWWXB7YVLQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.720203965 (266h46m38.078862618s) [upload=235241]: Next returned nil segment Nov 22 12:30:52.720216190 (266h46m38.078874829s) [upload=235241]: Waiting for error group managing segments... Nov 22 12:30:52.859097543 (266h46m38.2177562s) [noise=true , address=145.79.227.28:4726 , piece_id=EHG25AF4CZ3UMVOVG6TSX3TFAXMTFZND4INOLLX5IZUM3744FQQA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.859631290 (266h46m38.218289947s) [noise=true , address=145.79.227.27:4691 , piece_id=HJV42J2TB2Y5SXVDTKJS265UQ34MAPMNAH2ZWH3NUXBWLXHWOOYQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.859726981 (266h46m38.21838564s) [noise=true , address=145.79.227.31:4799 , piece_id=6J5APHQR2STIABAKSUZQEBPSR6XIVVK2LXNDU6JDMIBCJRGT54GA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.861545214 (266h46m38.22020387s) [noise=true , address=145.79.227.10:4402 , piece_id=2QSJODIK6I6QYQXATBBLAL5SDGB7O3H5OO4GLRQAGUCAXHI337VA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.864774401 (266h46m38.223433064s) [noise=true , address=145.79.227.25:4619 , piece_id=4KKZOITVXGP7TQ5Q3WGKU5IOSY7CBDX4YHNKUD2TAFRFCNDDFWLA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.868389753 (266h46m38.22704841s) [noise=true , address=145.79.227.17:4970 , piece_id=VGCXEGCG2ZJSGHCPUW6TEJC33G3PT4OGK2WG6PCW6FBYUOH5Q6LA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.871585428 (266h46m38.230244094s) [noise=true , address=145.79.227.11:4730 , piece_id=A6MKVCMMOEDQCNVLZVLQE7727WRCTWSQESD5OTLYMEXXPGZXULYA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.873727410 (266h46m38.232386067s) [noise=true , address=145.79.227.22:4509 , piece_id=WZ556AW2TRVM3GC2DQEFDSBXOGTYGDBH424ZLBFOBB5NKAT3JUKA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.875573701 (266h46m38.234232366s) [noise=true , address=145.79.227.22:4510 , piece_id=GEQO4OVLPB3CWCYVOCAWMP7WUYVRIWNNICOUSTHHLKWXNQL46HLQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.877964136 (266h46m38.236622776s) [noise=true , address=145.79.227.15:4875 , piece_id=RN4TLLBVGZQ2LZZES36O4XGTPSZLU4EYMA5VZ7DZBP46H342C64A, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.884853383 (266h46m38.243512048s) [noise=true , address=145.79.227.24:4581 , piece_id=7Y32DQ7BHOPIZWX7STPG2YHOWBY5Q3XW74S7D2U25POIKVKFYIUQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:31:21.718054035 (266h47m7.076712695s) [noise=true , address=77.90.31.21:4877 , piece_id=AAKMO3BX7T6ISQFVZKNSL374X6PMRPBGWCMXW2NIAUILGWN2Y6WQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: write tcp 139.59.144.22:44008->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 22 12:31:21.718401995 (266h47m7.077060726s) [noise=true , address=77.90.31.18:4806 , piece_id=RSJWLFHBYMTJO47QF7ZK3ZE2GV7Z2B2T3NQDMH644VRFUQAJCYVA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 22 12:31:21.718466899 (266h47m7.077125546s) [noise=true , address=45.91.116.4:4939 , piece_id=CY72GWLHSUZJ6JL2O57SYBNYYTKC4CMT3EKX4NWFE3JWWXB7YVLQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:49860->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 22 12:31:21.718639920 (266h47m7.07729858s) [noise=true , address=45.91.116.6:4975 , piece_id=CVSCYXLALCDU5O4NPWK4T73Q5DZFFM4YA6EZMNGSCYRRYLOGAGSA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: context deadline exceeded; context deadline exceeded Nov 22 12:31:21.718670710 (266h47m7.077329363s) [noise=true , address=45.91.116.11:4117 , piece_id=VJ2G5TGE46SVXX574ZGMRQUER6CJUYDHB47BNVWNPBIFDFFNKZBQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:40600->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 22 12:31:21.718696534 (266h47m7.077355177s) [seg_pos={0 0}, upload=235241]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Nov 22 12:31:21.821464823 (266h47m7.180123476s) [upload=235241]: Done uploading segments... Nov 22 12:31:21.821497872 (266h47m7.180156526s) [upload=235241]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.222760865s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 2.753218568s trace: https://jaeger.dev.storj.io/trace/220a09ca74d9d1c8 FAIL (2025-11-22 13:15:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.753250498s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..XX........................................................] duration: 1.523333867s trace: https://jaeger.dev.storj.io/trace/5bcba4939824f50a FAIL (2025-11-22 13:15:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.523357424s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 2.514722872s trace: https://jaeger.dev.storj.io/trace/220a09ca74d9d1c8 FAIL (2025-11-22 13:14:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.514746778s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [X..XX.......................................................] duration: 1.609205997s trace: https://jaeger.dev.storj.io/trace/5bcba4939824f50a FAIL (2025-11-22 13:14:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.609231605s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.X..X.......................................................] duration: 2.332428651s trace: https://jaeger.dev.storj.io/trace/5656a0c7e0b53cda FAIL (2025-11-22 13:14:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Nov 22 12:30:52.003787021 (266h46m37.362445675s) [upload=235241]: Starting upload Nov 22 12:30:52.003885048 (266h46m37.362543729s) [upload=235241]: Done starting upload Nov 22 12:30:52.003977160 (266h46m37.362635814s) [upload=235241]: Uploading segments... Nov 22 12:30:52.004140730 (266h46m37.362799384s) [upload=235241]: Got next segment. Inline: false Nov 22 12:30:52.004176073 (266h46m37.362834729s) [upload=235241]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 22 12:30:52.004181703 (266h46m37.362840356s) [upload=235241]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 22 12:30:52.113539293 (266h46m37.472197949s) [seg_pos={0 0}, upload=235241]: Begin upload segment... Nov 22 12:30:52.113592796 (266h46m37.472251458s) [seg_pos={0 0}, upload=235241]: Done begin upload segment. Nov 22 12:30:52.113622689 (266h46m37.472281343s) [noise=true , address=77.90.31.21:4877 , piece_id=AAKMO3BX7T6ISQFVZKNSL374X6PMRPBGWCMXW2NIAUILGWN2Y6WQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.113670993 (266h46m37.472329655s) [noise=true , address=145.79.227.25:4619 , piece_id=4KKZOITVXGP7TQ5Q3WGKU5IOSY7CBDX4YHNKUD2TAFRFCNDDFWLA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.114132816 (266h46m37.472791473s) [noise=true , address=145.79.227.11:4730 , piece_id=A6MKVCMMOEDQCNVLZVLQE7727WRCTWSQESD5OTLYMEXXPGZXULYA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.114351061 (266h46m37.473009726s) [noise=true , address=145.79.227.22:4509 , piece_id=WZ556AW2TRVM3GC2DQEFDSBXOGTYGDBH424ZLBFOBB5NKAT3JUKA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.114514284 (266h46m37.47317294s) [noise=true , address=145.79.227.27:4691 , piece_id=HJV42J2TB2Y5SXVDTKJS265UQ34MAPMNAH2ZWH3NUXBWLXHWOOYQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.114801466 (266h46m37.473460129s) [noise=true , address=145.79.227.31:4799 , piece_id=6J5APHQR2STIABAKSUZQEBPSR6XIVVK2LXNDU6JDMIBCJRGT54GA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.114835097 (266h46m37.473493753s) [noise=true , address=145.79.227.17:4970 , piece_id=VGCXEGCG2ZJSGHCPUW6TEJC33G3PT4OGK2WG6PCW6FBYUOH5Q6LA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.115174456 (266h46m37.47383311s) [noise=true , address=145.79.227.22:4510 , piece_id=GEQO4OVLPB3CWCYVOCAWMP7WUYVRIWNNICOUSTHHLKWXNQL46HLQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.115233219 (266h46m37.473891879s) [noise=true , address=145.79.227.28:4726 , piece_id=EHG25AF4CZ3UMVOVG6TSX3TFAXMTFZND4INOLLX5IZUM3744FQQA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.115559987 (266h46m37.474218641s) [noise=true , address=45.91.116.6:4975 , piece_id=CVSCYXLALCDU5O4NPWK4T73Q5DZFFM4YA6EZMNGSCYRRYLOGAGSA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.115737038 (266h46m37.474395697s) [noise=true , address=145.79.227.10:4402 , piece_id=2QSJODIK6I6QYQXATBBLAL5SDGB7O3H5OO4GLRQAGUCAXHI337VA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.115831971 (266h46m37.474490611s) [noise=true , address=77.90.31.18:4806 , piece_id=RSJWLFHBYMTJO47QF7ZK3ZE2GV7Z2B2T3NQDMH644VRFUQAJCYVA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.116101068 (266h46m37.474759721s) [noise=true , address=145.79.227.24:4581 , piece_id=7Y32DQ7BHOPIZWX7STPG2YHOWBY5Q3XW74S7D2U25POIKVKFYIUQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.116130009 (266h46m37.474788662s) [noise=true , address=45.91.116.11:4117 , piece_id=VJ2G5TGE46SVXX574ZGMRQUER6CJUYDHB47BNVWNPBIFDFFNKZBQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.116505131 (266h46m37.475163791s) [noise=true , address=145.79.227.15:4875 , piece_id=RN4TLLBVGZQ2LZZES36O4XGTPSZLU4EYMA5VZ7DZBP46H342C64A, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.116936492 (266h46m37.47559515s) [noise=true , address=45.91.116.4:4939 , piece_id=CY72GWLHSUZJ6JL2O57SYBNYYTKC4CMT3EKX4NWFE3JWWXB7YVLQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.720203965 (266h46m38.078862618s) [upload=235241]: Next returned nil segment Nov 22 12:30:52.720216190 (266h46m38.078874829s) [upload=235241]: Waiting for error group managing segments... Nov 22 12:30:52.859097543 (266h46m38.2177562s) [noise=true , address=145.79.227.28:4726 , piece_id=EHG25AF4CZ3UMVOVG6TSX3TFAXMTFZND4INOLLX5IZUM3744FQQA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.859631290 (266h46m38.218289947s) [noise=true , address=145.79.227.27:4691 , piece_id=HJV42J2TB2Y5SXVDTKJS265UQ34MAPMNAH2ZWH3NUXBWLXHWOOYQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.859726981 (266h46m38.21838564s) [noise=true , address=145.79.227.31:4799 , piece_id=6J5APHQR2STIABAKSUZQEBPSR6XIVVK2LXNDU6JDMIBCJRGT54GA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.861545214 (266h46m38.22020387s) [noise=true , address=145.79.227.10:4402 , piece_id=2QSJODIK6I6QYQXATBBLAL5SDGB7O3H5OO4GLRQAGUCAXHI337VA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.864774401 (266h46m38.223433064s) [noise=true , address=145.79.227.25:4619 , piece_id=4KKZOITVXGP7TQ5Q3WGKU5IOSY7CBDX4YHNKUD2TAFRFCNDDFWLA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.868389753 (266h46m38.22704841s) [noise=true , address=145.79.227.17:4970 , piece_id=VGCXEGCG2ZJSGHCPUW6TEJC33G3PT4OGK2WG6PCW6FBYUOH5Q6LA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.871585428 (266h46m38.230244094s) [noise=true , address=145.79.227.11:4730 , piece_id=A6MKVCMMOEDQCNVLZVLQE7727WRCTWSQESD5OTLYMEXXPGZXULYA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.873727410 (266h46m38.232386067s) [noise=true , address=145.79.227.22:4509 , piece_id=WZ556AW2TRVM3GC2DQEFDSBXOGTYGDBH424ZLBFOBB5NKAT3JUKA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.875573701 (266h46m38.234232366s) [noise=true , address=145.79.227.22:4510 , piece_id=GEQO4OVLPB3CWCYVOCAWMP7WUYVRIWNNICOUSTHHLKWXNQL46HLQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.877964136 (266h46m38.236622776s) [noise=true , address=145.79.227.15:4875 , piece_id=RN4TLLBVGZQ2LZZES36O4XGTPSZLU4EYMA5VZ7DZBP46H342C64A, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.884853383 (266h46m38.243512048s) [noise=true , address=145.79.227.24:4581 , piece_id=7Y32DQ7BHOPIZWX7STPG2YHOWBY5Q3XW74S7D2U25POIKVKFYIUQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:31:21.718054035 (266h47m7.076712695s) [noise=true , address=77.90.31.21:4877 , piece_id=AAKMO3BX7T6ISQFVZKNSL374X6PMRPBGWCMXW2NIAUILGWN2Y6WQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: write tcp 139.59.144.22:44008->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 22 12:31:21.718401995 (266h47m7.077060726s) [noise=true , address=77.90.31.18:4806 , piece_id=RSJWLFHBYMTJO47QF7ZK3ZE2GV7Z2B2T3NQDMH644VRFUQAJCYVA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 22 12:31:21.718466899 (266h47m7.077125546s) [noise=true , address=45.91.116.4:4939 , piece_id=CY72GWLHSUZJ6JL2O57SYBNYYTKC4CMT3EKX4NWFE3JWWXB7YVLQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:49860->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 22 12:31:21.718639920 (266h47m7.07729858s) [noise=true , address=45.91.116.6:4975 , piece_id=CVSCYXLALCDU5O4NPWK4T73Q5DZFFM4YA6EZMNGSCYRRYLOGAGSA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: context deadline exceeded; context deadline exceeded Nov 22 12:31:21.718670710 (266h47m7.077329363s) [noise=true , address=45.91.116.11:4117 , piece_id=VJ2G5TGE46SVXX574ZGMRQUER6CJUYDHB47BNVWNPBIFDFFNKZBQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:40600->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 22 12:31:21.718696534 (266h47m7.077355177s) [seg_pos={0 0}, upload=235241]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Nov 22 12:31:21.821464823 (266h47m7.180123476s) [upload=235241]: Done uploading segments... Nov 22 12:31:21.821497872 (266h47m7.180156526s) [upload=235241]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.332458237s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 2.738504214s trace: https://jaeger.dev.storj.io/trace/220a09ca74d9d1c8 FAIL (2025-11-22 13:14:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.738527163s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 4 out of 60 is failed (threshold=6) [XX..XX......................................................] duration: 1.715800095s trace: https://jaeger.dev.storj.io/trace/5bcba4939824f50a FAIL (2025-11-22 13:14:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.715826243s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..X..X......................................................] duration: 2.38821433s trace: https://jaeger.dev.storj.io/trace/5656a0c7e0b53cda FAIL (2025-11-22 13:14:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Nov 22 12:30:52.003787021 (266h46m37.362445675s) [upload=235241]: Starting upload Nov 22 12:30:52.003885048 (266h46m37.362543729s) [upload=235241]: Done starting upload Nov 22 12:30:52.003977160 (266h46m37.362635814s) [upload=235241]: Uploading segments... Nov 22 12:30:52.004140730 (266h46m37.362799384s) [upload=235241]: Got next segment. Inline: false Nov 22 12:30:52.004176073 (266h46m37.362834729s) [upload=235241]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 22 12:30:52.004181703 (266h46m37.362840356s) [upload=235241]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 22 12:30:52.113539293 (266h46m37.472197949s) [seg_pos={0 0}, upload=235241]: Begin upload segment... Nov 22 12:30:52.113592796 (266h46m37.472251458s) [seg_pos={0 0}, upload=235241]: Done begin upload segment. Nov 22 12:30:52.113622689 (266h46m37.472281343s) [noise=true , address=77.90.31.21:4877 , piece_id=AAKMO3BX7T6ISQFVZKNSL374X6PMRPBGWCMXW2NIAUILGWN2Y6WQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.113670993 (266h46m37.472329655s) [noise=true , address=145.79.227.25:4619 , piece_id=4KKZOITVXGP7TQ5Q3WGKU5IOSY7CBDX4YHNKUD2TAFRFCNDDFWLA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.114132816 (266h46m37.472791473s) [noise=true , address=145.79.227.11:4730 , piece_id=A6MKVCMMOEDQCNVLZVLQE7727WRCTWSQESD5OTLYMEXXPGZXULYA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.114351061 (266h46m37.473009726s) [noise=true , address=145.79.227.22:4509 , piece_id=WZ556AW2TRVM3GC2DQEFDSBXOGTYGDBH424ZLBFOBB5NKAT3JUKA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.114514284 (266h46m37.47317294s) [noise=true , address=145.79.227.27:4691 , piece_id=HJV42J2TB2Y5SXVDTKJS265UQ34MAPMNAH2ZWH3NUXBWLXHWOOYQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.114801466 (266h46m37.473460129s) [noise=true , address=145.79.227.31:4799 , piece_id=6J5APHQR2STIABAKSUZQEBPSR6XIVVK2LXNDU6JDMIBCJRGT54GA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.114835097 (266h46m37.473493753s) [noise=true , address=145.79.227.17:4970 , piece_id=VGCXEGCG2ZJSGHCPUW6TEJC33G3PT4OGK2WG6PCW6FBYUOH5Q6LA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.115174456 (266h46m37.47383311s) [noise=true , address=145.79.227.22:4510 , piece_id=GEQO4OVLPB3CWCYVOCAWMP7WUYVRIWNNICOUSTHHLKWXNQL46HLQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.115233219 (266h46m37.473891879s) [noise=true , address=145.79.227.28:4726 , piece_id=EHG25AF4CZ3UMVOVG6TSX3TFAXMTFZND4INOLLX5IZUM3744FQQA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.115559987 (266h46m37.474218641s) [noise=true , address=45.91.116.6:4975 , piece_id=CVSCYXLALCDU5O4NPWK4T73Q5DZFFM4YA6EZMNGSCYRRYLOGAGSA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.115737038 (266h46m37.474395697s) [noise=true , address=145.79.227.10:4402 , piece_id=2QSJODIK6I6QYQXATBBLAL5SDGB7O3H5OO4GLRQAGUCAXHI337VA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.115831971 (266h46m37.474490611s) [noise=true , address=77.90.31.18:4806 , piece_id=RSJWLFHBYMTJO47QF7ZK3ZE2GV7Z2B2T3NQDMH644VRFUQAJCYVA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.116101068 (266h46m37.474759721s) [noise=true , address=145.79.227.24:4581 , piece_id=7Y32DQ7BHOPIZWX7STPG2YHOWBY5Q3XW74S7D2U25POIKVKFYIUQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.116130009 (266h46m37.474788662s) [noise=true , address=45.91.116.11:4117 , piece_id=VJ2G5TGE46SVXX574ZGMRQUER6CJUYDHB47BNVWNPBIFDFFNKZBQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.116505131 (266h46m37.475163791s) [noise=true , address=145.79.227.15:4875 , piece_id=RN4TLLBVGZQ2LZZES36O4XGTPSZLU4EYMA5VZ7DZBP46H342C64A, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.116936492 (266h46m37.47559515s) [noise=true , address=45.91.116.4:4939 , piece_id=CY72GWLHSUZJ6JL2O57SYBNYYTKC4CMT3EKX4NWFE3JWWXB7YVLQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.720203965 (266h46m38.078862618s) [upload=235241]: Next returned nil segment Nov 22 12:30:52.720216190 (266h46m38.078874829s) [upload=235241]: Waiting for error group managing segments... Nov 22 12:30:52.859097543 (266h46m38.2177562s) [noise=true , address=145.79.227.28:4726 , piece_id=EHG25AF4CZ3UMVOVG6TSX3TFAXMTFZND4INOLLX5IZUM3744FQQA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.859631290 (266h46m38.218289947s) [noise=true , address=145.79.227.27:4691 , piece_id=HJV42J2TB2Y5SXVDTKJS265UQ34MAPMNAH2ZWH3NUXBWLXHWOOYQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.859726981 (266h46m38.21838564s) [noise=true , address=145.79.227.31:4799 , piece_id=6J5APHQR2STIABAKSUZQEBPSR6XIVVK2LXNDU6JDMIBCJRGT54GA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.861545214 (266h46m38.22020387s) [noise=true , address=145.79.227.10:4402 , piece_id=2QSJODIK6I6QYQXATBBLAL5SDGB7O3H5OO4GLRQAGUCAXHI337VA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.864774401 (266h46m38.223433064s) [noise=true , address=145.79.227.25:4619 , piece_id=4KKZOITVXGP7TQ5Q3WGKU5IOSY7CBDX4YHNKUD2TAFRFCNDDFWLA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.868389753 (266h46m38.22704841s) [noise=true , address=145.79.227.17:4970 , piece_id=VGCXEGCG2ZJSGHCPUW6TEJC33G3PT4OGK2WG6PCW6FBYUOH5Q6LA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.871585428 (266h46m38.230244094s) [noise=true , address=145.79.227.11:4730 , piece_id=A6MKVCMMOEDQCNVLZVLQE7727WRCTWSQESD5OTLYMEXXPGZXULYA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.873727410 (266h46m38.232386067s) [noise=true , address=145.79.227.22:4509 , piece_id=WZ556AW2TRVM3GC2DQEFDSBXOGTYGDBH424ZLBFOBB5NKAT3JUKA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.875573701 (266h46m38.234232366s) [noise=true , address=145.79.227.22:4510 , piece_id=GEQO4OVLPB3CWCYVOCAWMP7WUYVRIWNNICOUSTHHLKWXNQL46HLQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.877964136 (266h46m38.236622776s) [noise=true , address=145.79.227.15:4875 , piece_id=RN4TLLBVGZQ2LZZES36O4XGTPSZLU4EYMA5VZ7DZBP46H342C64A, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.884853383 (266h46m38.243512048s) [noise=true , address=145.79.227.24:4581 , piece_id=7Y32DQ7BHOPIZWX7STPG2YHOWBY5Q3XW74S7D2U25POIKVKFYIUQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:31:21.718054035 (266h47m7.076712695s) [noise=true , address=77.90.31.21:4877 , piece_id=AAKMO3BX7T6ISQFVZKNSL374X6PMRPBGWCMXW2NIAUILGWN2Y6WQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: write tcp 139.59.144.22:44008->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 22 12:31:21.718401995 (266h47m7.077060726s) [noise=true , address=77.90.31.18:4806 , piece_id=RSJWLFHBYMTJO47QF7ZK3ZE2GV7Z2B2T3NQDMH644VRFUQAJCYVA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 22 12:31:21.718466899 (266h47m7.077125546s) [noise=true , address=45.91.116.4:4939 , piece_id=CY72GWLHSUZJ6JL2O57SYBNYYTKC4CMT3EKX4NWFE3JWWXB7YVLQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:49860->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 22 12:31:21.718639920 (266h47m7.07729858s) [noise=true , address=45.91.116.6:4975 , piece_id=CVSCYXLALCDU5O4NPWK4T73Q5DZFFM4YA6EZMNGSCYRRYLOGAGSA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: context deadline exceeded; context deadline exceeded Nov 22 12:31:21.718670710 (266h47m7.077329363s) [noise=true , address=45.91.116.11:4117 , piece_id=VJ2G5TGE46SVXX574ZGMRQUER6CJUYDHB47BNVWNPBIFDFFNKZBQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:40600->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 22 12:31:21.718696534 (266h47m7.077355177s) [seg_pos={0 0}, upload=235241]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Nov 22 12:31:21.821464823 (266h47m7.180123476s) [upload=235241]: Done uploading segments... Nov 22 12:31:21.821497872 (266h47m7.180156526s) [upload=235241]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.3882468s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...X..X.....................................................] duration: 2.314481271s trace: https://jaeger.dev.storj.io/trace/5656a0c7e0b53cda FAIL (2025-11-22 13:13:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Nov 22 12:30:52.003787021 (266h46m37.362445675s) [upload=235241]: Starting upload Nov 22 12:30:52.003885048 (266h46m37.362543729s) [upload=235241]: Done starting upload Nov 22 12:30:52.003977160 (266h46m37.362635814s) [upload=235241]: Uploading segments... Nov 22 12:30:52.004140730 (266h46m37.362799384s) [upload=235241]: Got next segment. Inline: false Nov 22 12:30:52.004176073 (266h46m37.362834729s) [upload=235241]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 22 12:30:52.004181703 (266h46m37.362840356s) [upload=235241]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 22 12:30:52.113539293 (266h46m37.472197949s) [seg_pos={0 0}, upload=235241]: Begin upload segment... Nov 22 12:30:52.113592796 (266h46m37.472251458s) [seg_pos={0 0}, upload=235241]: Done begin upload segment. Nov 22 12:30:52.113622689 (266h46m37.472281343s) [noise=true , address=77.90.31.21:4877 , piece_id=AAKMO3BX7T6ISQFVZKNSL374X6PMRPBGWCMXW2NIAUILGWN2Y6WQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.113670993 (266h46m37.472329655s) [noise=true , address=145.79.227.25:4619 , piece_id=4KKZOITVXGP7TQ5Q3WGKU5IOSY7CBDX4YHNKUD2TAFRFCNDDFWLA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.114132816 (266h46m37.472791473s) [noise=true , address=145.79.227.11:4730 , piece_id=A6MKVCMMOEDQCNVLZVLQE7727WRCTWSQESD5OTLYMEXXPGZXULYA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.114351061 (266h46m37.473009726s) [noise=true , address=145.79.227.22:4509 , piece_id=WZ556AW2TRVM3GC2DQEFDSBXOGTYGDBH424ZLBFOBB5NKAT3JUKA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.114514284 (266h46m37.47317294s) [noise=true , address=145.79.227.27:4691 , piece_id=HJV42J2TB2Y5SXVDTKJS265UQ34MAPMNAH2ZWH3NUXBWLXHWOOYQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.114801466 (266h46m37.473460129s) [noise=true , address=145.79.227.31:4799 , piece_id=6J5APHQR2STIABAKSUZQEBPSR6XIVVK2LXNDU6JDMIBCJRGT54GA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.114835097 (266h46m37.473493753s) [noise=true , address=145.79.227.17:4970 , piece_id=VGCXEGCG2ZJSGHCPUW6TEJC33G3PT4OGK2WG6PCW6FBYUOH5Q6LA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.115174456 (266h46m37.47383311s) [noise=true , address=145.79.227.22:4510 , piece_id=GEQO4OVLPB3CWCYVOCAWMP7WUYVRIWNNICOUSTHHLKWXNQL46HLQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.115233219 (266h46m37.473891879s) [noise=true , address=145.79.227.28:4726 , piece_id=EHG25AF4CZ3UMVOVG6TSX3TFAXMTFZND4INOLLX5IZUM3744FQQA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.115559987 (266h46m37.474218641s) [noise=true , address=45.91.116.6:4975 , piece_id=CVSCYXLALCDU5O4NPWK4T73Q5DZFFM4YA6EZMNGSCYRRYLOGAGSA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.115737038 (266h46m37.474395697s) [noise=true , address=145.79.227.10:4402 , piece_id=2QSJODIK6I6QYQXATBBLAL5SDGB7O3H5OO4GLRQAGUCAXHI337VA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.115831971 (266h46m37.474490611s) [noise=true , address=77.90.31.18:4806 , piece_id=RSJWLFHBYMTJO47QF7ZK3ZE2GV7Z2B2T3NQDMH644VRFUQAJCYVA, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.116101068 (266h46m37.474759721s) [noise=true , address=145.79.227.24:4581 , piece_id=7Y32DQ7BHOPIZWX7STPG2YHOWBY5Q3XW74S7D2U25POIKVKFYIUQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.116130009 (266h46m37.474788662s) [noise=true , address=45.91.116.11:4117 , piece_id=VJ2G5TGE46SVXX574ZGMRQUER6CJUYDHB47BNVWNPBIFDFFNKZBQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.116505131 (266h46m37.475163791s) [noise=true , address=145.79.227.15:4875 , piece_id=RN4TLLBVGZQ2LZZES36O4XGTPSZLU4EYMA5VZ7DZBP46H342C64A, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.116936492 (266h46m37.47559515s) [noise=true , address=45.91.116.4:4939 , piece_id=CY72GWLHSUZJ6JL2O57SYBNYYTKC4CMT3EKX4NWFE3JWWXB7YVLQ, seg_pos={0 0}, upload=235241]: Uploading piece... Nov 22 12:30:52.720203965 (266h46m38.078862618s) [upload=235241]: Next returned nil segment Nov 22 12:30:52.720216190 (266h46m38.078874829s) [upload=235241]: Waiting for error group managing segments... Nov 22 12:30:52.859097543 (266h46m38.2177562s) [noise=true , address=145.79.227.28:4726 , piece_id=EHG25AF4CZ3UMVOVG6TSX3TFAXMTFZND4INOLLX5IZUM3744FQQA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.859631290 (266h46m38.218289947s) [noise=true , address=145.79.227.27:4691 , piece_id=HJV42J2TB2Y5SXVDTKJS265UQ34MAPMNAH2ZWH3NUXBWLXHWOOYQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.859726981 (266h46m38.21838564s) [noise=true , address=145.79.227.31:4799 , piece_id=6J5APHQR2STIABAKSUZQEBPSR6XIVVK2LXNDU6JDMIBCJRGT54GA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.861545214 (266h46m38.22020387s) [noise=true , address=145.79.227.10:4402 , piece_id=2QSJODIK6I6QYQXATBBLAL5SDGB7O3H5OO4GLRQAGUCAXHI337VA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.864774401 (266h46m38.223433064s) [noise=true , address=145.79.227.25:4619 , piece_id=4KKZOITVXGP7TQ5Q3WGKU5IOSY7CBDX4YHNKUD2TAFRFCNDDFWLA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.868389753 (266h46m38.22704841s) [noise=true , address=145.79.227.17:4970 , piece_id=VGCXEGCG2ZJSGHCPUW6TEJC33G3PT4OGK2WG6PCW6FBYUOH5Q6LA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.871585428 (266h46m38.230244094s) [noise=true , address=145.79.227.11:4730 , piece_id=A6MKVCMMOEDQCNVLZVLQE7727WRCTWSQESD5OTLYMEXXPGZXULYA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.873727410 (266h46m38.232386067s) [noise=true , address=145.79.227.22:4509 , piece_id=WZ556AW2TRVM3GC2DQEFDSBXOGTYGDBH424ZLBFOBB5NKAT3JUKA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.875573701 (266h46m38.234232366s) [noise=true , address=145.79.227.22:4510 , piece_id=GEQO4OVLPB3CWCYVOCAWMP7WUYVRIWNNICOUSTHHLKWXNQL46HLQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.877964136 (266h46m38.236622776s) [noise=true , address=145.79.227.15:4875 , piece_id=RN4TLLBVGZQ2LZZES36O4XGTPSZLU4EYMA5VZ7DZBP46H342C64A, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:30:52.884853383 (266h46m38.243512048s) [noise=true , address=145.79.227.24:4581 , piece_id=7Y32DQ7BHOPIZWX7STPG2YHOWBY5Q3XW74S7D2U25POIKVKFYIUQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: <nil> Nov 22 12:31:21.718054035 (266h47m7.076712695s) [noise=true , address=77.90.31.21:4877 , piece_id=AAKMO3BX7T6ISQFVZKNSL374X6PMRPBGWCMXW2NIAUILGWN2Y6WQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: write tcp 139.59.144.22:44008->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 22 12:31:21.718401995 (266h47m7.077060726s) [noise=true , address=77.90.31.18:4806 , piece_id=RSJWLFHBYMTJO47QF7ZK3ZE2GV7Z2B2T3NQDMH644VRFUQAJCYVA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 22 12:31:21.718466899 (266h47m7.077125546s) [noise=true , address=45.91.116.4:4939 , piece_id=CY72GWLHSUZJ6JL2O57SYBNYYTKC4CMT3EKX4NWFE3JWWXB7YVLQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 139.59.144.22:49860->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 22 12:31:21.718639920 (266h47m7.07729858s) [noise=true , address=45.91.116.6:4975 , piece_id=CVSCYXLALCDU5O4NPWK4T73Q5DZFFM4YA6EZMNGSCYRRYLOGAGSA, seg_pos={0 0}, upload=235241]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: context deadline exceeded; context deadline exceeded Nov 22 12:31:21.718670710 (266h47m7.077329363s) [noise=true , address=45.91.116.11:4117 , piece_id=VJ2G5TGE46SVXX574ZGMRQUER6CJUYDHB47BNVWNPBIFDFFNKZBQ, seg_pos={0 0}, upload=235241]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 139.59.144.22:40600->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 22 12:31:21.718696534 (266h47m7.077355177s) [seg_pos={0 0}, upload=235241]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Nov 22 12:31:21.821464823 (266h47m7.180123476s) [upload=235241]: Done uploading segments... Nov 22 12:31:21.821497872 (266h47m7.180156526s) [upload=235241]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.314515683s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 4 out of 60 is failed (threshold=6) [.XX..XX.....................................................] duration: 2.121032511s trace: https://jaeger.dev.storj.io/trace/5bcba4939824f50a FAIL (2025-11-22 13:13:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.121064194s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)