Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 2.814633546s FAIL (2025-10-01 03:09:36 UTC) orbiter warning: context deadline exceeded Duration: 2.814680489s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 2.638003313s FAIL (2025-10-01 03:08:54 UTC) orbiter warning: context deadline exceeded Duration: 2.638032085s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 2.515664318s FAIL (2025-10-01 03:08:17 UTC) orbiter warning: context deadline exceeded Duration: 2.515694112s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 2.761374246s FAIL (2025-10-01 03:07:24 UTC) orbiter warning: context deadline exceeded Duration: 2.761420033s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 2.565191988s FAIL (2025-10-01 03:06:59 UTC) orbiter warning: context deadline exceeded Duration: 2.565222838s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 2.632174511s FAIL (2025-10-01 03:05:51 UTC) orbiter warning: context deadline exceeded Duration: 2.632199088s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 2.79175921s FAIL (2025-10-01 03:04:37 UTC) orbiter warning: context deadline exceeded Duration: 2.791777822s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 2.78735471s FAIL (2025-10-01 03:04:17 UTC) orbiter warning: context deadline exceeded Duration: 2.787379909s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 2.574509553s FAIL (2025-10-01 03:03:28 UTC) orbiter warning: context deadline exceeded Duration: 2.574531017s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 2.757388584s FAIL (2025-10-01 03:03:12 UTC) orbiter warning: context deadline exceeded Duration: 2.757409235s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 2.722140268s FAIL (2025-10-01 03:01:57 UTC) orbiter warning: context deadline exceeded Duration: 2.722167812s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 2.722516242s FAIL (2025-10-01 03:01:13 UTC) orbiter warning: context deadline exceeded Duration: 2.72259261s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 2.622982256s FAIL (2025-10-01 03:00:11 UTC) orbiter warning: context deadline exceeded Duration: 2.623010323s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 2.708978749s FAIL (2025-10-01 02:59:22 UTC) orbiter warning: context deadline exceeded Duration: 2.709014862s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 2.738833944s FAIL (2025-10-01 02:58:09 UTC) orbiter warning: context deadline exceeded Duration: 2.738867412s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 2.582542794s FAIL (2025-10-01 02:57:52 UTC) orbiter warning: context deadline exceeded Duration: 2.582566275s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 2.708549193s FAIL (2025-10-01 02:56:45 UTC) orbiter warning: context deadline exceeded Duration: 2.708596381s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 2.545798099s FAIL (2025-10-01 02:56:17 UTC) orbiter warning: context deadline exceeded Duration: 2.545816645s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 2.594494548s FAIL (2025-10-01 02:55:07 UTC) orbiter warning: context deadline exceeded Duration: 2.59452023s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 2.724741609s FAIL (2025-10-01 02:53:58 UTC) orbiter warning: context deadline exceeded Duration: 2.724769068s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 2.792261709s FAIL (2025-10-01 02:53:03 UTC) orbiter warning: context deadline exceeded Duration: 2.792357106s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 2.650557878s FAIL (2025-10-01 02:52:22 UTC) orbiter warning: context deadline exceeded Duration: 2.650601338s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 2.856501779s FAIL (2025-10-01 02:51:16 UTC) orbiter warning: context deadline exceeded Duration: 2.856528766s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 2.833881577s FAIL (2025-10-01 02:50:27 UTC) orbiter warning: context deadline exceeded Duration: 2.833905335s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 2.62569147s FAIL (2025-10-01 02:49:34 UTC) orbiter warning: context deadline exceeded Duration: 2.625740952s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 2.589247862s FAIL (2025-10-01 02:49:04 UTC) orbiter warning: context deadline exceeded Duration: 2.589353664s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 2.782197284s FAIL (2025-10-01 02:48:02 UTC) orbiter warning: context deadline exceeded Duration: 2.782221021s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 2.856154472s FAIL (2025-10-01 02:47:22 UTC) orbiter warning: context deadline exceeded Duration: 2.856169538s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 2.822482011s FAIL (2025-10-01 02:46:13 UTC) orbiter warning: context deadline exceeded Duration: 2.823663245s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 2.679351121s FAIL (2025-10-01 02:45:44 UTC) orbiter warning: context deadline exceeded Duration: 2.679382456s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 2.589074174s FAIL (2025-10-01 02:44:54 UTC) orbiter warning: context deadline exceeded Duration: 2.589166713s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 2.542282742s FAIL (2025-10-01 02:44:12 UTC) orbiter warning: context deadline exceeded Duration: 2.542314937s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 2.980029596s FAIL (2025-10-01 02:43:07 UTC) orbiter warning: context deadline exceeded Duration: 2.980053964s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.001249582s FAIL (2025-10-01 02:42:19 UTC) orbiter warning: context deadline exceeded Duration: 30.001278256s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.466061383s FAIL (2025-09-30 22:13:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 30 21:29:22.797897319 (175h59m10.601361902s) [upload=154939]: Starting upload Sep 30 21:29:22.798006215 (175h59m10.601470808s) [upload=154939]: Done starting upload Sep 30 21:29:22.798081644 (175h59m10.601546229s) [upload=154939]: Uploading segments... Sep 30 21:29:22.798290989 (175h59m10.60175557s) [upload=154939]: Got next segment. Inline: false Sep 30 21:29:22.798331474 (175h59m10.601796067s) [upload=154939]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 30 21:29:22.798339902 (175h59m10.601804489s) [upload=154939]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 21:29:22.915769149 (175h59m10.719233752s) [seg_pos={0 0}, upload=154939]: Begin upload segment... Sep 30 21:29:22.915871743 (175h59m10.719336336s) [seg_pos={0 0}, upload=154939]: Done begin upload segment. Sep 30 21:29:22.915919849 (175h59m10.719384451s) [noise=true , address=45.91.116.4:4940 , piece_id=7DW6Y2G3IUR74PJ2ENFV6XLGTBZTISG4BQFI53KSUJ7HUMYIOSTA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.917360095 (175h59m10.720824697s) [noise=true , address=145.79.227.3:4151 , piece_id=LOVPJ4STVGHTGVOBIS5IMXQDUT7QNS55EFEDPZKD65WLBKTEO3OA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.917523118 (175h59m10.720987717s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918065655 (175h59m10.721530259s) [noise=true , address=145.79.227.13:4801 , piece_id=2W5CLWLLIUYIFCFUW7BBO5MGURJ6YBUNAQDVJCJ2F2LSVX3SLI5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918188240 (175h59m10.721652839s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918605397 (175h59m10.722069997s) [noise=true , address=45.91.116.5:4298 , piece_id=DUBHMFQQHF23TXRYMRODBVKGVCLAE6TG7IPEKTBHUE72WEX5O2SA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918717812 (175h59m10.722182405s) [noise=true , address=145.79.227.16:4910 , piece_id=TBS644O7MTLBU7635SRB2N46WHQK6X5KYRYVITTZZAUFPEKC7GDQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919113523 (175h59m10.722578125s) [noise=true , address=145.79.227.7:4295 , piece_id=VLXVHJOSXZ627O6VCWPEL3H3QPLLSK7OZWGRCSRRT3MIPJDB5YBQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919331578 (175h59m10.722796181s) [noise=true , address=145.79.227.18:4005 , piece_id=IYY76ITHCMY4Z22VRD7WRX3Y6BDVLFKOQ2PCQRU7WFBE6VDQKV5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919626618 (175h59m10.72309122s) [noise=true , address=45.91.116.10:4082 , piece_id=UWRIMN2P47HUKBYHCATAARGFTJXFGQZ7KRV5MUMIQ4GQRFQO7D6Q, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920025044 (175h59m10.723489653s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920253240 (175h59m10.723717844s) [noise=true , address=145.79.227.13:4802 , piece_id=Y5SXFQYQF32UFPHKIMIMI3V7WYDHGSZCPONZTZS5EVVKYSPSQONA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920729685 (175h59m10.724194286s) [noise=true , address=145.79.227.11:4731 , piece_id=EDPUPZOFFO7TQWD7MI5MYMZSDONWB22VGCMZBZPMZA5MB3LOWXAA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.921315566 (175h59m10.724780168s) [noise=true , address=145.79.227.6:4257 , piece_id=5YAEQS2PX6L4EXAKWT33WBMWXFRFDKTFKV7CBCPLO4BYCZKKA53A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.923000023 (175h59m10.726464628s) [noise=true , address=145.79.227.2:4114 , piece_id=V2U5JQ6Y5XJOQSXE6EKKQOA7F3ENKEKWTFOQ65LZV6BBRSK6YD5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.923798468 (175h59m10.727263073s) [noise=true , address=45.91.116.10:4081 , piece_id=WJKNHHM37DDABHFA6LPCF35FUW4SEMNMINZTPXDMJULW7TAL22OQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:23.539151225 (175h59m11.342615808s) [upload=154939]: Next returned nil segment Sep 30 21:29:23.539169161 (175h59m11.34263376s) [upload=154939]: Waiting for error group managing segments... Sep 30 21:29:23.685441552 (175h59m11.488906147s) [noise=true , address=45.91.116.4:4940 , piece_id=7DW6Y2G3IUR74PJ2ENFV6XLGTBZTISG4BQFI53KSUJ7HUMYIOSTA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.688251132 (175h59m11.491715718s) [noise=true , address=145.79.227.7:4295 , piece_id=VLXVHJOSXZ627O6VCWPEL3H3QPLLSK7OZWGRCSRRT3MIPJDB5YBQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.691906915 (175h59m11.495371495s) [noise=true , address=145.79.227.16:4910 , piece_id=TBS644O7MTLBU7635SRB2N46WHQK6X5KYRYVITTZZAUFPEKC7GDQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.698112841 (175h59m11.501577422s) [noise=true , address=145.79.227.6:4257 , piece_id=5YAEQS2PX6L4EXAKWT33WBMWXFRFDKTFKV7CBCPLO4BYCZKKA53A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.698258894 (175h59m11.501723478s) [noise=true , address=145.79.227.2:4114 , piece_id=V2U5JQ6Y5XJOQSXE6EKKQOA7F3ENKEKWTFOQ65LZV6BBRSK6YD5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.702276005 (175h59m11.505740586s) [noise=true , address=45.91.116.10:4081 , piece_id=WJKNHHM37DDABHFA6LPCF35FUW4SEMNMINZTPXDMJULW7TAL22OQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.707342122 (175h59m11.51080671s) [noise=true , address=145.79.227.13:4802 , piece_id=Y5SXFQYQF32UFPHKIMIMI3V7WYDHGSZCPONZTZS5EVVKYSPSQONA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.711442145 (175h59m11.51490673s) [noise=true , address=145.79.227.11:4731 , piece_id=EDPUPZOFFO7TQWD7MI5MYMZSDONWB22VGCMZBZPMZA5MB3LOWXAA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.719319539 (175h59m11.522784123s) [noise=true , address=145.79.227.3:4151 , piece_id=LOVPJ4STVGHTGVOBIS5IMXQDUT7QNS55EFEDPZKD65WLBKTEO3OA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.988493881 (175h59m11.791958458s) [noise=true , address=45.91.116.5:4298 , piece_id=DUBHMFQQHF23TXRYMRODBVKGVCLAE6TG7IPEKTBHUE72WEX5O2SA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:24.038956565 (175h59m11.842421143s) [noise=true , address=45.91.116.10:4082 , piece_id=UWRIMN2P47HUKBYHCATAARGFTJXFGQZ7KRV5MUMIQ4GQRFQO7D6Q, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:24.589378044 (175h59m12.392842628s) [noise=true , address=145.79.227.18:4005 , piece_id=IYY76ITHCMY4Z22VRD7WRX3Y6BDVLFKOQ2PCQRU7WFBE6VDQKV5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:25.194244661 (175h59m12.99770926s) [noise=true , address=145.79.227.13:4801 , piece_id=2W5CLWLLIUYIFCFUW7BBO5MGURJ6YBUNAQDVJCJ2F2LSVX3SLI5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:25.194299398 (175h59m12.997763991s) [seg_pos={0 0}, upload=154939]: Segment reached optimal threshold of 13 pieces. Sep 30 21:29:25.194653994 (175h59m12.99811859s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk): protocol: read tcp 139.59.144.22:34818->145.79.227.19:4077: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 30 21:29:25.194670346 (175h59m12.998134947s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194691442 (175h59m12.99815604s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: read tcp 139.59.144.22:59584->77.90.31.17:4770: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 30 21:29:25.194706378 (175h59m12.998170983s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194911052 (175h59m12.998375653s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: context canceled; context canceled Sep 30 21:29:25.194921667 (175h59m12.998386269s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194946997 (175h59m12.998411575s) [seg_pos={0 0}, upload=154939]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 30 21:29:25.195081879 (175h59m12.998546481s) [upload=154939]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 30 21:29:25.195089364 (175h59m12.998553965s) [upload=154939]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 30 21:29:25.348637798 (175h59m13.1521024s) [upload=154939]: Done uploading segments... Sep 30 21:29:25.348680254 (175h59m13.152144854s) [upload=154939]: Upload finished. err: <nil> Duration: 2.466090031s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 2.344026012s FAIL (2025-09-30 22:13:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 30 21:29:22.797897319 (175h59m10.601361902s) [upload=154939]: Starting upload Sep 30 21:29:22.798006215 (175h59m10.601470808s) [upload=154939]: Done starting upload Sep 30 21:29:22.798081644 (175h59m10.601546229s) [upload=154939]: Uploading segments... Sep 30 21:29:22.798290989 (175h59m10.60175557s) [upload=154939]: Got next segment. Inline: false Sep 30 21:29:22.798331474 (175h59m10.601796067s) [upload=154939]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 30 21:29:22.798339902 (175h59m10.601804489s) [upload=154939]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 21:29:22.915769149 (175h59m10.719233752s) [seg_pos={0 0}, upload=154939]: Begin upload segment... Sep 30 21:29:22.915871743 (175h59m10.719336336s) [seg_pos={0 0}, upload=154939]: Done begin upload segment. Sep 30 21:29:22.915919849 (175h59m10.719384451s) [noise=true , address=45.91.116.4:4940 , piece_id=7DW6Y2G3IUR74PJ2ENFV6XLGTBZTISG4BQFI53KSUJ7HUMYIOSTA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.917360095 (175h59m10.720824697s) [noise=true , address=145.79.227.3:4151 , piece_id=LOVPJ4STVGHTGVOBIS5IMXQDUT7QNS55EFEDPZKD65WLBKTEO3OA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.917523118 (175h59m10.720987717s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918065655 (175h59m10.721530259s) [noise=true , address=145.79.227.13:4801 , piece_id=2W5CLWLLIUYIFCFUW7BBO5MGURJ6YBUNAQDVJCJ2F2LSVX3SLI5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918188240 (175h59m10.721652839s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918605397 (175h59m10.722069997s) [noise=true , address=45.91.116.5:4298 , piece_id=DUBHMFQQHF23TXRYMRODBVKGVCLAE6TG7IPEKTBHUE72WEX5O2SA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918717812 (175h59m10.722182405s) [noise=true , address=145.79.227.16:4910 , piece_id=TBS644O7MTLBU7635SRB2N46WHQK6X5KYRYVITTZZAUFPEKC7GDQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919113523 (175h59m10.722578125s) [noise=true , address=145.79.227.7:4295 , piece_id=VLXVHJOSXZ627O6VCWPEL3H3QPLLSK7OZWGRCSRRT3MIPJDB5YBQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919331578 (175h59m10.722796181s) [noise=true , address=145.79.227.18:4005 , piece_id=IYY76ITHCMY4Z22VRD7WRX3Y6BDVLFKOQ2PCQRU7WFBE6VDQKV5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919626618 (175h59m10.72309122s) [noise=true , address=45.91.116.10:4082 , piece_id=UWRIMN2P47HUKBYHCATAARGFTJXFGQZ7KRV5MUMIQ4GQRFQO7D6Q, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920025044 (175h59m10.723489653s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920253240 (175h59m10.723717844s) [noise=true , address=145.79.227.13:4802 , piece_id=Y5SXFQYQF32UFPHKIMIMI3V7WYDHGSZCPONZTZS5EVVKYSPSQONA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920729685 (175h59m10.724194286s) [noise=true , address=145.79.227.11:4731 , piece_id=EDPUPZOFFO7TQWD7MI5MYMZSDONWB22VGCMZBZPMZA5MB3LOWXAA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.921315566 (175h59m10.724780168s) [noise=true , address=145.79.227.6:4257 , piece_id=5YAEQS2PX6L4EXAKWT33WBMWXFRFDKTFKV7CBCPLO4BYCZKKA53A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.923000023 (175h59m10.726464628s) [noise=true , address=145.79.227.2:4114 , piece_id=V2U5JQ6Y5XJOQSXE6EKKQOA7F3ENKEKWTFOQ65LZV6BBRSK6YD5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.923798468 (175h59m10.727263073s) [noise=true , address=45.91.116.10:4081 , piece_id=WJKNHHM37DDABHFA6LPCF35FUW4SEMNMINZTPXDMJULW7TAL22OQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:23.539151225 (175h59m11.342615808s) [upload=154939]: Next returned nil segment Sep 30 21:29:23.539169161 (175h59m11.34263376s) [upload=154939]: Waiting for error group managing segments... Sep 30 21:29:23.685441552 (175h59m11.488906147s) [noise=true , address=45.91.116.4:4940 , piece_id=7DW6Y2G3IUR74PJ2ENFV6XLGTBZTISG4BQFI53KSUJ7HUMYIOSTA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.688251132 (175h59m11.491715718s) [noise=true , address=145.79.227.7:4295 , piece_id=VLXVHJOSXZ627O6VCWPEL3H3QPLLSK7OZWGRCSRRT3MIPJDB5YBQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.691906915 (175h59m11.495371495s) [noise=true , address=145.79.227.16:4910 , piece_id=TBS644O7MTLBU7635SRB2N46WHQK6X5KYRYVITTZZAUFPEKC7GDQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.698112841 (175h59m11.501577422s) [noise=true , address=145.79.227.6:4257 , piece_id=5YAEQS2PX6L4EXAKWT33WBMWXFRFDKTFKV7CBCPLO4BYCZKKA53A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.698258894 (175h59m11.501723478s) [noise=true , address=145.79.227.2:4114 , piece_id=V2U5JQ6Y5XJOQSXE6EKKQOA7F3ENKEKWTFOQ65LZV6BBRSK6YD5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.702276005 (175h59m11.505740586s) [noise=true , address=45.91.116.10:4081 , piece_id=WJKNHHM37DDABHFA6LPCF35FUW4SEMNMINZTPXDMJULW7TAL22OQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.707342122 (175h59m11.51080671s) [noise=true , address=145.79.227.13:4802 , piece_id=Y5SXFQYQF32UFPHKIMIMI3V7WYDHGSZCPONZTZS5EVVKYSPSQONA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.711442145 (175h59m11.51490673s) [noise=true , address=145.79.227.11:4731 , piece_id=EDPUPZOFFO7TQWD7MI5MYMZSDONWB22VGCMZBZPMZA5MB3LOWXAA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.719319539 (175h59m11.522784123s) [noise=true , address=145.79.227.3:4151 , piece_id=LOVPJ4STVGHTGVOBIS5IMXQDUT7QNS55EFEDPZKD65WLBKTEO3OA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.988493881 (175h59m11.791958458s) [noise=true , address=45.91.116.5:4298 , piece_id=DUBHMFQQHF23TXRYMRODBVKGVCLAE6TG7IPEKTBHUE72WEX5O2SA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:24.038956565 (175h59m11.842421143s) [noise=true , address=45.91.116.10:4082 , piece_id=UWRIMN2P47HUKBYHCATAARGFTJXFGQZ7KRV5MUMIQ4GQRFQO7D6Q, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:24.589378044 (175h59m12.392842628s) [noise=true , address=145.79.227.18:4005 , piece_id=IYY76ITHCMY4Z22VRD7WRX3Y6BDVLFKOQ2PCQRU7WFBE6VDQKV5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:25.194244661 (175h59m12.99770926s) [noise=true , address=145.79.227.13:4801 , piece_id=2W5CLWLLIUYIFCFUW7BBO5MGURJ6YBUNAQDVJCJ2F2LSVX3SLI5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:25.194299398 (175h59m12.997763991s) [seg_pos={0 0}, upload=154939]: Segment reached optimal threshold of 13 pieces. Sep 30 21:29:25.194653994 (175h59m12.99811859s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk): protocol: read tcp 139.59.144.22:34818->145.79.227.19:4077: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 30 21:29:25.194670346 (175h59m12.998134947s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194691442 (175h59m12.99815604s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: read tcp 139.59.144.22:59584->77.90.31.17:4770: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 30 21:29:25.194706378 (175h59m12.998170983s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194911052 (175h59m12.998375653s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: context canceled; context canceled Sep 30 21:29:25.194921667 (175h59m12.998386269s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194946997 (175h59m12.998411575s) [seg_pos={0 0}, upload=154939]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 30 21:29:25.195081879 (175h59m12.998546481s) [upload=154939]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 30 21:29:25.195089364 (175h59m12.998553965s) [upload=154939]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 30 21:29:25.348637798 (175h59m13.1521024s) [upload=154939]: Done uploading segments... Sep 30 21:29:25.348680254 (175h59m13.152144854s) [upload=154939]: Upload finished. err: <nil> Duration: 2.344075734s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 2.64982692s FAIL (2025-09-30 22:12:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 30 21:29:22.797897319 (175h59m10.601361902s) [upload=154939]: Starting upload Sep 30 21:29:22.798006215 (175h59m10.601470808s) [upload=154939]: Done starting upload Sep 30 21:29:22.798081644 (175h59m10.601546229s) [upload=154939]: Uploading segments... Sep 30 21:29:22.798290989 (175h59m10.60175557s) [upload=154939]: Got next segment. Inline: false Sep 30 21:29:22.798331474 (175h59m10.601796067s) [upload=154939]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 30 21:29:22.798339902 (175h59m10.601804489s) [upload=154939]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 21:29:22.915769149 (175h59m10.719233752s) [seg_pos={0 0}, upload=154939]: Begin upload segment... Sep 30 21:29:22.915871743 (175h59m10.719336336s) [seg_pos={0 0}, upload=154939]: Done begin upload segment. Sep 30 21:29:22.915919849 (175h59m10.719384451s) [noise=true , address=45.91.116.4:4940 , piece_id=7DW6Y2G3IUR74PJ2ENFV6XLGTBZTISG4BQFI53KSUJ7HUMYIOSTA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.917360095 (175h59m10.720824697s) [noise=true , address=145.79.227.3:4151 , piece_id=LOVPJ4STVGHTGVOBIS5IMXQDUT7QNS55EFEDPZKD65WLBKTEO3OA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.917523118 (175h59m10.720987717s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918065655 (175h59m10.721530259s) [noise=true , address=145.79.227.13:4801 , piece_id=2W5CLWLLIUYIFCFUW7BBO5MGURJ6YBUNAQDVJCJ2F2LSVX3SLI5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918188240 (175h59m10.721652839s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918605397 (175h59m10.722069997s) [noise=true , address=45.91.116.5:4298 , piece_id=DUBHMFQQHF23TXRYMRODBVKGVCLAE6TG7IPEKTBHUE72WEX5O2SA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918717812 (175h59m10.722182405s) [noise=true , address=145.79.227.16:4910 , piece_id=TBS644O7MTLBU7635SRB2N46WHQK6X5KYRYVITTZZAUFPEKC7GDQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919113523 (175h59m10.722578125s) [noise=true , address=145.79.227.7:4295 , piece_id=VLXVHJOSXZ627O6VCWPEL3H3QPLLSK7OZWGRCSRRT3MIPJDB5YBQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919331578 (175h59m10.722796181s) [noise=true , address=145.79.227.18:4005 , piece_id=IYY76ITHCMY4Z22VRD7WRX3Y6BDVLFKOQ2PCQRU7WFBE6VDQKV5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919626618 (175h59m10.72309122s) [noise=true , address=45.91.116.10:4082 , piece_id=UWRIMN2P47HUKBYHCATAARGFTJXFGQZ7KRV5MUMIQ4GQRFQO7D6Q, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920025044 (175h59m10.723489653s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920253240 (175h59m10.723717844s) [noise=true , address=145.79.227.13:4802 , piece_id=Y5SXFQYQF32UFPHKIMIMI3V7WYDHGSZCPONZTZS5EVVKYSPSQONA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920729685 (175h59m10.724194286s) [noise=true , address=145.79.227.11:4731 , piece_id=EDPUPZOFFO7TQWD7MI5MYMZSDONWB22VGCMZBZPMZA5MB3LOWXAA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.921315566 (175h59m10.724780168s) [noise=true , address=145.79.227.6:4257 , piece_id=5YAEQS2PX6L4EXAKWT33WBMWXFRFDKTFKV7CBCPLO4BYCZKKA53A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.923000023 (175h59m10.726464628s) [noise=true , address=145.79.227.2:4114 , piece_id=V2U5JQ6Y5XJOQSXE6EKKQOA7F3ENKEKWTFOQ65LZV6BBRSK6YD5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.923798468 (175h59m10.727263073s) [noise=true , address=45.91.116.10:4081 , piece_id=WJKNHHM37DDABHFA6LPCF35FUW4SEMNMINZTPXDMJULW7TAL22OQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:23.539151225 (175h59m11.342615808s) [upload=154939]: Next returned nil segment Sep 30 21:29:23.539169161 (175h59m11.34263376s) [upload=154939]: Waiting for error group managing segments... Sep 30 21:29:23.685441552 (175h59m11.488906147s) [noise=true , address=45.91.116.4:4940 , piece_id=7DW6Y2G3IUR74PJ2ENFV6XLGTBZTISG4BQFI53KSUJ7HUMYIOSTA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.688251132 (175h59m11.491715718s) [noise=true , address=145.79.227.7:4295 , piece_id=VLXVHJOSXZ627O6VCWPEL3H3QPLLSK7OZWGRCSRRT3MIPJDB5YBQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.691906915 (175h59m11.495371495s) [noise=true , address=145.79.227.16:4910 , piece_id=TBS644O7MTLBU7635SRB2N46WHQK6X5KYRYVITTZZAUFPEKC7GDQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.698112841 (175h59m11.501577422s) [noise=true , address=145.79.227.6:4257 , piece_id=5YAEQS2PX6L4EXAKWT33WBMWXFRFDKTFKV7CBCPLO4BYCZKKA53A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.698258894 (175h59m11.501723478s) [noise=true , address=145.79.227.2:4114 , piece_id=V2U5JQ6Y5XJOQSXE6EKKQOA7F3ENKEKWTFOQ65LZV6BBRSK6YD5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.702276005 (175h59m11.505740586s) [noise=true , address=45.91.116.10:4081 , piece_id=WJKNHHM37DDABHFA6LPCF35FUW4SEMNMINZTPXDMJULW7TAL22OQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.707342122 (175h59m11.51080671s) [noise=true , address=145.79.227.13:4802 , piece_id=Y5SXFQYQF32UFPHKIMIMI3V7WYDHGSZCPONZTZS5EVVKYSPSQONA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.711442145 (175h59m11.51490673s) [noise=true , address=145.79.227.11:4731 , piece_id=EDPUPZOFFO7TQWD7MI5MYMZSDONWB22VGCMZBZPMZA5MB3LOWXAA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.719319539 (175h59m11.522784123s) [noise=true , address=145.79.227.3:4151 , piece_id=LOVPJ4STVGHTGVOBIS5IMXQDUT7QNS55EFEDPZKD65WLBKTEO3OA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.988493881 (175h59m11.791958458s) [noise=true , address=45.91.116.5:4298 , piece_id=DUBHMFQQHF23TXRYMRODBVKGVCLAE6TG7IPEKTBHUE72WEX5O2SA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:24.038956565 (175h59m11.842421143s) [noise=true , address=45.91.116.10:4082 , piece_id=UWRIMN2P47HUKBYHCATAARGFTJXFGQZ7KRV5MUMIQ4GQRFQO7D6Q, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:24.589378044 (175h59m12.392842628s) [noise=true , address=145.79.227.18:4005 , piece_id=IYY76ITHCMY4Z22VRD7WRX3Y6BDVLFKOQ2PCQRU7WFBE6VDQKV5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:25.194244661 (175h59m12.99770926s) [noise=true , address=145.79.227.13:4801 , piece_id=2W5CLWLLIUYIFCFUW7BBO5MGURJ6YBUNAQDVJCJ2F2LSVX3SLI5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:25.194299398 (175h59m12.997763991s) [seg_pos={0 0}, upload=154939]: Segment reached optimal threshold of 13 pieces. Sep 30 21:29:25.194653994 (175h59m12.99811859s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk): protocol: read tcp 139.59.144.22:34818->145.79.227.19:4077: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 30 21:29:25.194670346 (175h59m12.998134947s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194691442 (175h59m12.99815604s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: read tcp 139.59.144.22:59584->77.90.31.17:4770: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 30 21:29:25.194706378 (175h59m12.998170983s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194911052 (175h59m12.998375653s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: context canceled; context canceled Sep 30 21:29:25.194921667 (175h59m12.998386269s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194946997 (175h59m12.998411575s) [seg_pos={0 0}, upload=154939]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 30 21:29:25.195081879 (175h59m12.998546481s) [upload=154939]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 30 21:29:25.195089364 (175h59m12.998553965s) [upload=154939]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 30 21:29:25.348637798 (175h59m13.1521024s) [upload=154939]: Done uploading segments... Sep 30 21:29:25.348680254 (175h59m13.152144854s) [upload=154939]: Upload finished. err: <nil> Duration: 2.649893369s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 2.451212211s FAIL (2025-09-30 22:11:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 30 21:29:22.797897319 (175h59m10.601361902s) [upload=154939]: Starting upload Sep 30 21:29:22.798006215 (175h59m10.601470808s) [upload=154939]: Done starting upload Sep 30 21:29:22.798081644 (175h59m10.601546229s) [upload=154939]: Uploading segments... Sep 30 21:29:22.798290989 (175h59m10.60175557s) [upload=154939]: Got next segment. Inline: false Sep 30 21:29:22.798331474 (175h59m10.601796067s) [upload=154939]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 30 21:29:22.798339902 (175h59m10.601804489s) [upload=154939]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 21:29:22.915769149 (175h59m10.719233752s) [seg_pos={0 0}, upload=154939]: Begin upload segment... Sep 30 21:29:22.915871743 (175h59m10.719336336s) [seg_pos={0 0}, upload=154939]: Done begin upload segment. Sep 30 21:29:22.915919849 (175h59m10.719384451s) [noise=true , address=45.91.116.4:4940 , piece_id=7DW6Y2G3IUR74PJ2ENFV6XLGTBZTISG4BQFI53KSUJ7HUMYIOSTA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.917360095 (175h59m10.720824697s) [noise=true , address=145.79.227.3:4151 , piece_id=LOVPJ4STVGHTGVOBIS5IMXQDUT7QNS55EFEDPZKD65WLBKTEO3OA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.917523118 (175h59m10.720987717s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918065655 (175h59m10.721530259s) [noise=true , address=145.79.227.13:4801 , piece_id=2W5CLWLLIUYIFCFUW7BBO5MGURJ6YBUNAQDVJCJ2F2LSVX3SLI5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918188240 (175h59m10.721652839s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918605397 (175h59m10.722069997s) [noise=true , address=45.91.116.5:4298 , piece_id=DUBHMFQQHF23TXRYMRODBVKGVCLAE6TG7IPEKTBHUE72WEX5O2SA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918717812 (175h59m10.722182405s) [noise=true , address=145.79.227.16:4910 , piece_id=TBS644O7MTLBU7635SRB2N46WHQK6X5KYRYVITTZZAUFPEKC7GDQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919113523 (175h59m10.722578125s) [noise=true , address=145.79.227.7:4295 , piece_id=VLXVHJOSXZ627O6VCWPEL3H3QPLLSK7OZWGRCSRRT3MIPJDB5YBQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919331578 (175h59m10.722796181s) [noise=true , address=145.79.227.18:4005 , piece_id=IYY76ITHCMY4Z22VRD7WRX3Y6BDVLFKOQ2PCQRU7WFBE6VDQKV5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919626618 (175h59m10.72309122s) [noise=true , address=45.91.116.10:4082 , piece_id=UWRIMN2P47HUKBYHCATAARGFTJXFGQZ7KRV5MUMIQ4GQRFQO7D6Q, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920025044 (175h59m10.723489653s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920253240 (175h59m10.723717844s) [noise=true , address=145.79.227.13:4802 , piece_id=Y5SXFQYQF32UFPHKIMIMI3V7WYDHGSZCPONZTZS5EVVKYSPSQONA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920729685 (175h59m10.724194286s) [noise=true , address=145.79.227.11:4731 , piece_id=EDPUPZOFFO7TQWD7MI5MYMZSDONWB22VGCMZBZPMZA5MB3LOWXAA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.921315566 (175h59m10.724780168s) [noise=true , address=145.79.227.6:4257 , piece_id=5YAEQS2PX6L4EXAKWT33WBMWXFRFDKTFKV7CBCPLO4BYCZKKA53A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.923000023 (175h59m10.726464628s) [noise=true , address=145.79.227.2:4114 , piece_id=V2U5JQ6Y5XJOQSXE6EKKQOA7F3ENKEKWTFOQ65LZV6BBRSK6YD5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.923798468 (175h59m10.727263073s) [noise=true , address=45.91.116.10:4081 , piece_id=WJKNHHM37DDABHFA6LPCF35FUW4SEMNMINZTPXDMJULW7TAL22OQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:23.539151225 (175h59m11.342615808s) [upload=154939]: Next returned nil segment Sep 30 21:29:23.539169161 (175h59m11.34263376s) [upload=154939]: Waiting for error group managing segments... Sep 30 21:29:23.685441552 (175h59m11.488906147s) [noise=true , address=45.91.116.4:4940 , piece_id=7DW6Y2G3IUR74PJ2ENFV6XLGTBZTISG4BQFI53KSUJ7HUMYIOSTA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.688251132 (175h59m11.491715718s) [noise=true , address=145.79.227.7:4295 , piece_id=VLXVHJOSXZ627O6VCWPEL3H3QPLLSK7OZWGRCSRRT3MIPJDB5YBQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.691906915 (175h59m11.495371495s) [noise=true , address=145.79.227.16:4910 , piece_id=TBS644O7MTLBU7635SRB2N46WHQK6X5KYRYVITTZZAUFPEKC7GDQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.698112841 (175h59m11.501577422s) [noise=true , address=145.79.227.6:4257 , piece_id=5YAEQS2PX6L4EXAKWT33WBMWXFRFDKTFKV7CBCPLO4BYCZKKA53A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.698258894 (175h59m11.501723478s) [noise=true , address=145.79.227.2:4114 , piece_id=V2U5JQ6Y5XJOQSXE6EKKQOA7F3ENKEKWTFOQ65LZV6BBRSK6YD5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.702276005 (175h59m11.505740586s) [noise=true , address=45.91.116.10:4081 , piece_id=WJKNHHM37DDABHFA6LPCF35FUW4SEMNMINZTPXDMJULW7TAL22OQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.707342122 (175h59m11.51080671s) [noise=true , address=145.79.227.13:4802 , piece_id=Y5SXFQYQF32UFPHKIMIMI3V7WYDHGSZCPONZTZS5EVVKYSPSQONA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.711442145 (175h59m11.51490673s) [noise=true , address=145.79.227.11:4731 , piece_id=EDPUPZOFFO7TQWD7MI5MYMZSDONWB22VGCMZBZPMZA5MB3LOWXAA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.719319539 (175h59m11.522784123s) [noise=true , address=145.79.227.3:4151 , piece_id=LOVPJ4STVGHTGVOBIS5IMXQDUT7QNS55EFEDPZKD65WLBKTEO3OA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.988493881 (175h59m11.791958458s) [noise=true , address=45.91.116.5:4298 , piece_id=DUBHMFQQHF23TXRYMRODBVKGVCLAE6TG7IPEKTBHUE72WEX5O2SA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:24.038956565 (175h59m11.842421143s) [noise=true , address=45.91.116.10:4082 , piece_id=UWRIMN2P47HUKBYHCATAARGFTJXFGQZ7KRV5MUMIQ4GQRFQO7D6Q, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:24.589378044 (175h59m12.392842628s) [noise=true , address=145.79.227.18:4005 , piece_id=IYY76ITHCMY4Z22VRD7WRX3Y6BDVLFKOQ2PCQRU7WFBE6VDQKV5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:25.194244661 (175h59m12.99770926s) [noise=true , address=145.79.227.13:4801 , piece_id=2W5CLWLLIUYIFCFUW7BBO5MGURJ6YBUNAQDVJCJ2F2LSVX3SLI5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:25.194299398 (175h59m12.997763991s) [seg_pos={0 0}, upload=154939]: Segment reached optimal threshold of 13 pieces. Sep 30 21:29:25.194653994 (175h59m12.99811859s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk): protocol: read tcp 139.59.144.22:34818->145.79.227.19:4077: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 30 21:29:25.194670346 (175h59m12.998134947s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194691442 (175h59m12.99815604s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: read tcp 139.59.144.22:59584->77.90.31.17:4770: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 30 21:29:25.194706378 (175h59m12.998170983s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194911052 (175h59m12.998375653s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: context canceled; context canceled Sep 30 21:29:25.194921667 (175h59m12.998386269s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194946997 (175h59m12.998411575s) [seg_pos={0 0}, upload=154939]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 30 21:29:25.195081879 (175h59m12.998546481s) [upload=154939]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 30 21:29:25.195089364 (175h59m12.998553965s) [upload=154939]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 30 21:29:25.348637798 (175h59m13.1521024s) [upload=154939]: Done uploading segments... Sep 30 21:29:25.348680254 (175h59m13.152144854s) [upload=154939]: Upload finished. err: <nil> Duration: 2.451242678s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 2.546550807s FAIL (2025-09-30 22:10:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 30 21:29:22.797897319 (175h59m10.601361902s) [upload=154939]: Starting upload Sep 30 21:29:22.798006215 (175h59m10.601470808s) [upload=154939]: Done starting upload Sep 30 21:29:22.798081644 (175h59m10.601546229s) [upload=154939]: Uploading segments... Sep 30 21:29:22.798290989 (175h59m10.60175557s) [upload=154939]: Got next segment. Inline: false Sep 30 21:29:22.798331474 (175h59m10.601796067s) [upload=154939]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 30 21:29:22.798339902 (175h59m10.601804489s) [upload=154939]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 21:29:22.915769149 (175h59m10.719233752s) [seg_pos={0 0}, upload=154939]: Begin upload segment... Sep 30 21:29:22.915871743 (175h59m10.719336336s) [seg_pos={0 0}, upload=154939]: Done begin upload segment. Sep 30 21:29:22.915919849 (175h59m10.719384451s) [noise=true , address=45.91.116.4:4940 , piece_id=7DW6Y2G3IUR74PJ2ENFV6XLGTBZTISG4BQFI53KSUJ7HUMYIOSTA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.917360095 (175h59m10.720824697s) [noise=true , address=145.79.227.3:4151 , piece_id=LOVPJ4STVGHTGVOBIS5IMXQDUT7QNS55EFEDPZKD65WLBKTEO3OA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.917523118 (175h59m10.720987717s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918065655 (175h59m10.721530259s) [noise=true , address=145.79.227.13:4801 , piece_id=2W5CLWLLIUYIFCFUW7BBO5MGURJ6YBUNAQDVJCJ2F2LSVX3SLI5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918188240 (175h59m10.721652839s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918605397 (175h59m10.722069997s) [noise=true , address=45.91.116.5:4298 , piece_id=DUBHMFQQHF23TXRYMRODBVKGVCLAE6TG7IPEKTBHUE72WEX5O2SA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918717812 (175h59m10.722182405s) [noise=true , address=145.79.227.16:4910 , piece_id=TBS644O7MTLBU7635SRB2N46WHQK6X5KYRYVITTZZAUFPEKC7GDQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919113523 (175h59m10.722578125s) [noise=true , address=145.79.227.7:4295 , piece_id=VLXVHJOSXZ627O6VCWPEL3H3QPLLSK7OZWGRCSRRT3MIPJDB5YBQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919331578 (175h59m10.722796181s) [noise=true , address=145.79.227.18:4005 , piece_id=IYY76ITHCMY4Z22VRD7WRX3Y6BDVLFKOQ2PCQRU7WFBE6VDQKV5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919626618 (175h59m10.72309122s) [noise=true , address=45.91.116.10:4082 , piece_id=UWRIMN2P47HUKBYHCATAARGFTJXFGQZ7KRV5MUMIQ4GQRFQO7D6Q, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920025044 (175h59m10.723489653s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920253240 (175h59m10.723717844s) [noise=true , address=145.79.227.13:4802 , piece_id=Y5SXFQYQF32UFPHKIMIMI3V7WYDHGSZCPONZTZS5EVVKYSPSQONA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920729685 (175h59m10.724194286s) [noise=true , address=145.79.227.11:4731 , piece_id=EDPUPZOFFO7TQWD7MI5MYMZSDONWB22VGCMZBZPMZA5MB3LOWXAA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.921315566 (175h59m10.724780168s) [noise=true , address=145.79.227.6:4257 , piece_id=5YAEQS2PX6L4EXAKWT33WBMWXFRFDKTFKV7CBCPLO4BYCZKKA53A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.923000023 (175h59m10.726464628s) [noise=true , address=145.79.227.2:4114 , piece_id=V2U5JQ6Y5XJOQSXE6EKKQOA7F3ENKEKWTFOQ65LZV6BBRSK6YD5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.923798468 (175h59m10.727263073s) [noise=true , address=45.91.116.10:4081 , piece_id=WJKNHHM37DDABHFA6LPCF35FUW4SEMNMINZTPXDMJULW7TAL22OQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:23.539151225 (175h59m11.342615808s) [upload=154939]: Next returned nil segment Sep 30 21:29:23.539169161 (175h59m11.34263376s) [upload=154939]: Waiting for error group managing segments... Sep 30 21:29:23.685441552 (175h59m11.488906147s) [noise=true , address=45.91.116.4:4940 , piece_id=7DW6Y2G3IUR74PJ2ENFV6XLGTBZTISG4BQFI53KSUJ7HUMYIOSTA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.688251132 (175h59m11.491715718s) [noise=true , address=145.79.227.7:4295 , piece_id=VLXVHJOSXZ627O6VCWPEL3H3QPLLSK7OZWGRCSRRT3MIPJDB5YBQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.691906915 (175h59m11.495371495s) [noise=true , address=145.79.227.16:4910 , piece_id=TBS644O7MTLBU7635SRB2N46WHQK6X5KYRYVITTZZAUFPEKC7GDQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.698112841 (175h59m11.501577422s) [noise=true , address=145.79.227.6:4257 , piece_id=5YAEQS2PX6L4EXAKWT33WBMWXFRFDKTFKV7CBCPLO4BYCZKKA53A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.698258894 (175h59m11.501723478s) [noise=true , address=145.79.227.2:4114 , piece_id=V2U5JQ6Y5XJOQSXE6EKKQOA7F3ENKEKWTFOQ65LZV6BBRSK6YD5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.702276005 (175h59m11.505740586s) [noise=true , address=45.91.116.10:4081 , piece_id=WJKNHHM37DDABHFA6LPCF35FUW4SEMNMINZTPXDMJULW7TAL22OQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.707342122 (175h59m11.51080671s) [noise=true , address=145.79.227.13:4802 , piece_id=Y5SXFQYQF32UFPHKIMIMI3V7WYDHGSZCPONZTZS5EVVKYSPSQONA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.711442145 (175h59m11.51490673s) [noise=true , address=145.79.227.11:4731 , piece_id=EDPUPZOFFO7TQWD7MI5MYMZSDONWB22VGCMZBZPMZA5MB3LOWXAA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.719319539 (175h59m11.522784123s) [noise=true , address=145.79.227.3:4151 , piece_id=LOVPJ4STVGHTGVOBIS5IMXQDUT7QNS55EFEDPZKD65WLBKTEO3OA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.988493881 (175h59m11.791958458s) [noise=true , address=45.91.116.5:4298 , piece_id=DUBHMFQQHF23TXRYMRODBVKGVCLAE6TG7IPEKTBHUE72WEX5O2SA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:24.038956565 (175h59m11.842421143s) [noise=true , address=45.91.116.10:4082 , piece_id=UWRIMN2P47HUKBYHCATAARGFTJXFGQZ7KRV5MUMIQ4GQRFQO7D6Q, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:24.589378044 (175h59m12.392842628s) [noise=true , address=145.79.227.18:4005 , piece_id=IYY76ITHCMY4Z22VRD7WRX3Y6BDVLFKOQ2PCQRU7WFBE6VDQKV5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:25.194244661 (175h59m12.99770926s) [noise=true , address=145.79.227.13:4801 , piece_id=2W5CLWLLIUYIFCFUW7BBO5MGURJ6YBUNAQDVJCJ2F2LSVX3SLI5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:25.194299398 (175h59m12.997763991s) [seg_pos={0 0}, upload=154939]: Segment reached optimal threshold of 13 pieces. Sep 30 21:29:25.194653994 (175h59m12.99811859s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk): protocol: read tcp 139.59.144.22:34818->145.79.227.19:4077: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 30 21:29:25.194670346 (175h59m12.998134947s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194691442 (175h59m12.99815604s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: read tcp 139.59.144.22:59584->77.90.31.17:4770: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 30 21:29:25.194706378 (175h59m12.998170983s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194911052 (175h59m12.998375653s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: context canceled; context canceled Sep 30 21:29:25.194921667 (175h59m12.998386269s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194946997 (175h59m12.998411575s) [seg_pos={0 0}, upload=154939]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 30 21:29:25.195081879 (175h59m12.998546481s) [upload=154939]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 30 21:29:25.195089364 (175h59m12.998553965s) [upload=154939]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 30 21:29:25.348637798 (175h59m13.1521024s) [upload=154939]: Done uploading segments... Sep 30 21:29:25.348680254 (175h59m13.152144854s) [upload=154939]: Upload finished. err: <nil> Duration: 2.546577713s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 2.534422908s FAIL (2025-09-30 22:09:22 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 30 21:29:22.797897319 (175h59m10.601361902s) [upload=154939]: Starting upload Sep 30 21:29:22.798006215 (175h59m10.601470808s) [upload=154939]: Done starting upload Sep 30 21:29:22.798081644 (175h59m10.601546229s) [upload=154939]: Uploading segments... Sep 30 21:29:22.798290989 (175h59m10.60175557s) [upload=154939]: Got next segment. Inline: false Sep 30 21:29:22.798331474 (175h59m10.601796067s) [upload=154939]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 30 21:29:22.798339902 (175h59m10.601804489s) [upload=154939]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 21:29:22.915769149 (175h59m10.719233752s) [seg_pos={0 0}, upload=154939]: Begin upload segment... Sep 30 21:29:22.915871743 (175h59m10.719336336s) [seg_pos={0 0}, upload=154939]: Done begin upload segment. Sep 30 21:29:22.915919849 (175h59m10.719384451s) [noise=true , address=45.91.116.4:4940 , piece_id=7DW6Y2G3IUR74PJ2ENFV6XLGTBZTISG4BQFI53KSUJ7HUMYIOSTA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.917360095 (175h59m10.720824697s) [noise=true , address=145.79.227.3:4151 , piece_id=LOVPJ4STVGHTGVOBIS5IMXQDUT7QNS55EFEDPZKD65WLBKTEO3OA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.917523118 (175h59m10.720987717s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918065655 (175h59m10.721530259s) [noise=true , address=145.79.227.13:4801 , piece_id=2W5CLWLLIUYIFCFUW7BBO5MGURJ6YBUNAQDVJCJ2F2LSVX3SLI5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918188240 (175h59m10.721652839s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918605397 (175h59m10.722069997s) [noise=true , address=45.91.116.5:4298 , piece_id=DUBHMFQQHF23TXRYMRODBVKGVCLAE6TG7IPEKTBHUE72WEX5O2SA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918717812 (175h59m10.722182405s) [noise=true , address=145.79.227.16:4910 , piece_id=TBS644O7MTLBU7635SRB2N46WHQK6X5KYRYVITTZZAUFPEKC7GDQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919113523 (175h59m10.722578125s) [noise=true , address=145.79.227.7:4295 , piece_id=VLXVHJOSXZ627O6VCWPEL3H3QPLLSK7OZWGRCSRRT3MIPJDB5YBQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919331578 (175h59m10.722796181s) [noise=true , address=145.79.227.18:4005 , piece_id=IYY76ITHCMY4Z22VRD7WRX3Y6BDVLFKOQ2PCQRU7WFBE6VDQKV5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919626618 (175h59m10.72309122s) [noise=true , address=45.91.116.10:4082 , piece_id=UWRIMN2P47HUKBYHCATAARGFTJXFGQZ7KRV5MUMIQ4GQRFQO7D6Q, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920025044 (175h59m10.723489653s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920253240 (175h59m10.723717844s) [noise=true , address=145.79.227.13:4802 , piece_id=Y5SXFQYQF32UFPHKIMIMI3V7WYDHGSZCPONZTZS5EVVKYSPSQONA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920729685 (175h59m10.724194286s) [noise=true , address=145.79.227.11:4731 , piece_id=EDPUPZOFFO7TQWD7MI5MYMZSDONWB22VGCMZBZPMZA5MB3LOWXAA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.921315566 (175h59m10.724780168s) [noise=true , address=145.79.227.6:4257 , piece_id=5YAEQS2PX6L4EXAKWT33WBMWXFRFDKTFKV7CBCPLO4BYCZKKA53A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.923000023 (175h59m10.726464628s) [noise=true , address=145.79.227.2:4114 , piece_id=V2U5JQ6Y5XJOQSXE6EKKQOA7F3ENKEKWTFOQ65LZV6BBRSK6YD5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.923798468 (175h59m10.727263073s) [noise=true , address=45.91.116.10:4081 , piece_id=WJKNHHM37DDABHFA6LPCF35FUW4SEMNMINZTPXDMJULW7TAL22OQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:23.539151225 (175h59m11.342615808s) [upload=154939]: Next returned nil segment Sep 30 21:29:23.539169161 (175h59m11.34263376s) [upload=154939]: Waiting for error group managing segments... Sep 30 21:29:23.685441552 (175h59m11.488906147s) [noise=true , address=45.91.116.4:4940 , piece_id=7DW6Y2G3IUR74PJ2ENFV6XLGTBZTISG4BQFI53KSUJ7HUMYIOSTA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.688251132 (175h59m11.491715718s) [noise=true , address=145.79.227.7:4295 , piece_id=VLXVHJOSXZ627O6VCWPEL3H3QPLLSK7OZWGRCSRRT3MIPJDB5YBQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.691906915 (175h59m11.495371495s) [noise=true , address=145.79.227.16:4910 , piece_id=TBS644O7MTLBU7635SRB2N46WHQK6X5KYRYVITTZZAUFPEKC7GDQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.698112841 (175h59m11.501577422s) [noise=true , address=145.79.227.6:4257 , piece_id=5YAEQS2PX6L4EXAKWT33WBMWXFRFDKTFKV7CBCPLO4BYCZKKA53A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.698258894 (175h59m11.501723478s) [noise=true , address=145.79.227.2:4114 , piece_id=V2U5JQ6Y5XJOQSXE6EKKQOA7F3ENKEKWTFOQ65LZV6BBRSK6YD5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.702276005 (175h59m11.505740586s) [noise=true , address=45.91.116.10:4081 , piece_id=WJKNHHM37DDABHFA6LPCF35FUW4SEMNMINZTPXDMJULW7TAL22OQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.707342122 (175h59m11.51080671s) [noise=true , address=145.79.227.13:4802 , piece_id=Y5SXFQYQF32UFPHKIMIMI3V7WYDHGSZCPONZTZS5EVVKYSPSQONA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.711442145 (175h59m11.51490673s) [noise=true , address=145.79.227.11:4731 , piece_id=EDPUPZOFFO7TQWD7MI5MYMZSDONWB22VGCMZBZPMZA5MB3LOWXAA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.719319539 (175h59m11.522784123s) [noise=true , address=145.79.227.3:4151 , piece_id=LOVPJ4STVGHTGVOBIS5IMXQDUT7QNS55EFEDPZKD65WLBKTEO3OA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.988493881 (175h59m11.791958458s) [noise=true , address=45.91.116.5:4298 , piece_id=DUBHMFQQHF23TXRYMRODBVKGVCLAE6TG7IPEKTBHUE72WEX5O2SA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:24.038956565 (175h59m11.842421143s) [noise=true , address=45.91.116.10:4082 , piece_id=UWRIMN2P47HUKBYHCATAARGFTJXFGQZ7KRV5MUMIQ4GQRFQO7D6Q, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:24.589378044 (175h59m12.392842628s) [noise=true , address=145.79.227.18:4005 , piece_id=IYY76ITHCMY4Z22VRD7WRX3Y6BDVLFKOQ2PCQRU7WFBE6VDQKV5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:25.194244661 (175h59m12.99770926s) [noise=true , address=145.79.227.13:4801 , piece_id=2W5CLWLLIUYIFCFUW7BBO5MGURJ6YBUNAQDVJCJ2F2LSVX3SLI5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:25.194299398 (175h59m12.997763991s) [seg_pos={0 0}, upload=154939]: Segment reached optimal threshold of 13 pieces. Sep 30 21:29:25.194653994 (175h59m12.99811859s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk): protocol: read tcp 139.59.144.22:34818->145.79.227.19:4077: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 30 21:29:25.194670346 (175h59m12.998134947s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194691442 (175h59m12.99815604s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: read tcp 139.59.144.22:59584->77.90.31.17:4770: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 30 21:29:25.194706378 (175h59m12.998170983s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194911052 (175h59m12.998375653s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: context canceled; context canceled Sep 30 21:29:25.194921667 (175h59m12.998386269s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194946997 (175h59m12.998411575s) [seg_pos={0 0}, upload=154939]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 30 21:29:25.195081879 (175h59m12.998546481s) [upload=154939]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 30 21:29:25.195089364 (175h59m12.998553965s) [upload=154939]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 30 21:29:25.348637798 (175h59m13.1521024s) [upload=154939]: Done uploading segments... Sep 30 21:29:25.348680254 (175h59m13.152144854s) [upload=154939]: Upload finished. err: <nil> Duration: 2.534454806s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 2.392460616s FAIL (2025-09-30 22:08:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 30 21:29:22.797897319 (175h59m10.601361902s) [upload=154939]: Starting upload Sep 30 21:29:22.798006215 (175h59m10.601470808s) [upload=154939]: Done starting upload Sep 30 21:29:22.798081644 (175h59m10.601546229s) [upload=154939]: Uploading segments... Sep 30 21:29:22.798290989 (175h59m10.60175557s) [upload=154939]: Got next segment. Inline: false Sep 30 21:29:22.798331474 (175h59m10.601796067s) [upload=154939]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 30 21:29:22.798339902 (175h59m10.601804489s) [upload=154939]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 21:29:22.915769149 (175h59m10.719233752s) [seg_pos={0 0}, upload=154939]: Begin upload segment... Sep 30 21:29:22.915871743 (175h59m10.719336336s) [seg_pos={0 0}, upload=154939]: Done begin upload segment. Sep 30 21:29:22.915919849 (175h59m10.719384451s) [noise=true , address=45.91.116.4:4940 , piece_id=7DW6Y2G3IUR74PJ2ENFV6XLGTBZTISG4BQFI53KSUJ7HUMYIOSTA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.917360095 (175h59m10.720824697s) [noise=true , address=145.79.227.3:4151 , piece_id=LOVPJ4STVGHTGVOBIS5IMXQDUT7QNS55EFEDPZKD65WLBKTEO3OA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.917523118 (175h59m10.720987717s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918065655 (175h59m10.721530259s) [noise=true , address=145.79.227.13:4801 , piece_id=2W5CLWLLIUYIFCFUW7BBO5MGURJ6YBUNAQDVJCJ2F2LSVX3SLI5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918188240 (175h59m10.721652839s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918605397 (175h59m10.722069997s) [noise=true , address=45.91.116.5:4298 , piece_id=DUBHMFQQHF23TXRYMRODBVKGVCLAE6TG7IPEKTBHUE72WEX5O2SA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918717812 (175h59m10.722182405s) [noise=true , address=145.79.227.16:4910 , piece_id=TBS644O7MTLBU7635SRB2N46WHQK6X5KYRYVITTZZAUFPEKC7GDQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919113523 (175h59m10.722578125s) [noise=true , address=145.79.227.7:4295 , piece_id=VLXVHJOSXZ627O6VCWPEL3H3QPLLSK7OZWGRCSRRT3MIPJDB5YBQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919331578 (175h59m10.722796181s) [noise=true , address=145.79.227.18:4005 , piece_id=IYY76ITHCMY4Z22VRD7WRX3Y6BDVLFKOQ2PCQRU7WFBE6VDQKV5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919626618 (175h59m10.72309122s) [noise=true , address=45.91.116.10:4082 , piece_id=UWRIMN2P47HUKBYHCATAARGFTJXFGQZ7KRV5MUMIQ4GQRFQO7D6Q, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920025044 (175h59m10.723489653s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920253240 (175h59m10.723717844s) [noise=true , address=145.79.227.13:4802 , piece_id=Y5SXFQYQF32UFPHKIMIMI3V7WYDHGSZCPONZTZS5EVVKYSPSQONA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920729685 (175h59m10.724194286s) [noise=true , address=145.79.227.11:4731 , piece_id=EDPUPZOFFO7TQWD7MI5MYMZSDONWB22VGCMZBZPMZA5MB3LOWXAA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.921315566 (175h59m10.724780168s) [noise=true , address=145.79.227.6:4257 , piece_id=5YAEQS2PX6L4EXAKWT33WBMWXFRFDKTFKV7CBCPLO4BYCZKKA53A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.923000023 (175h59m10.726464628s) [noise=true , address=145.79.227.2:4114 , piece_id=V2U5JQ6Y5XJOQSXE6EKKQOA7F3ENKEKWTFOQ65LZV6BBRSK6YD5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.923798468 (175h59m10.727263073s) [noise=true , address=45.91.116.10:4081 , piece_id=WJKNHHM37DDABHFA6LPCF35FUW4SEMNMINZTPXDMJULW7TAL22OQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:23.539151225 (175h59m11.342615808s) [upload=154939]: Next returned nil segment Sep 30 21:29:23.539169161 (175h59m11.34263376s) [upload=154939]: Waiting for error group managing segments... Sep 30 21:29:23.685441552 (175h59m11.488906147s) [noise=true , address=45.91.116.4:4940 , piece_id=7DW6Y2G3IUR74PJ2ENFV6XLGTBZTISG4BQFI53KSUJ7HUMYIOSTA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.688251132 (175h59m11.491715718s) [noise=true , address=145.79.227.7:4295 , piece_id=VLXVHJOSXZ627O6VCWPEL3H3QPLLSK7OZWGRCSRRT3MIPJDB5YBQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.691906915 (175h59m11.495371495s) [noise=true , address=145.79.227.16:4910 , piece_id=TBS644O7MTLBU7635SRB2N46WHQK6X5KYRYVITTZZAUFPEKC7GDQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.698112841 (175h59m11.501577422s) [noise=true , address=145.79.227.6:4257 , piece_id=5YAEQS2PX6L4EXAKWT33WBMWXFRFDKTFKV7CBCPLO4BYCZKKA53A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.698258894 (175h59m11.501723478s) [noise=true , address=145.79.227.2:4114 , piece_id=V2U5JQ6Y5XJOQSXE6EKKQOA7F3ENKEKWTFOQ65LZV6BBRSK6YD5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.702276005 (175h59m11.505740586s) [noise=true , address=45.91.116.10:4081 , piece_id=WJKNHHM37DDABHFA6LPCF35FUW4SEMNMINZTPXDMJULW7TAL22OQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.707342122 (175h59m11.51080671s) [noise=true , address=145.79.227.13:4802 , piece_id=Y5SXFQYQF32UFPHKIMIMI3V7WYDHGSZCPONZTZS5EVVKYSPSQONA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.711442145 (175h59m11.51490673s) [noise=true , address=145.79.227.11:4731 , piece_id=EDPUPZOFFO7TQWD7MI5MYMZSDONWB22VGCMZBZPMZA5MB3LOWXAA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.719319539 (175h59m11.522784123s) [noise=true , address=145.79.227.3:4151 , piece_id=LOVPJ4STVGHTGVOBIS5IMXQDUT7QNS55EFEDPZKD65WLBKTEO3OA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.988493881 (175h59m11.791958458s) [noise=true , address=45.91.116.5:4298 , piece_id=DUBHMFQQHF23TXRYMRODBVKGVCLAE6TG7IPEKTBHUE72WEX5O2SA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:24.038956565 (175h59m11.842421143s) [noise=true , address=45.91.116.10:4082 , piece_id=UWRIMN2P47HUKBYHCATAARGFTJXFGQZ7KRV5MUMIQ4GQRFQO7D6Q, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:24.589378044 (175h59m12.392842628s) [noise=true , address=145.79.227.18:4005 , piece_id=IYY76ITHCMY4Z22VRD7WRX3Y6BDVLFKOQ2PCQRU7WFBE6VDQKV5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:25.194244661 (175h59m12.99770926s) [noise=true , address=145.79.227.13:4801 , piece_id=2W5CLWLLIUYIFCFUW7BBO5MGURJ6YBUNAQDVJCJ2F2LSVX3SLI5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:25.194299398 (175h59m12.997763991s) [seg_pos={0 0}, upload=154939]: Segment reached optimal threshold of 13 pieces. Sep 30 21:29:25.194653994 (175h59m12.99811859s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk): protocol: read tcp 139.59.144.22:34818->145.79.227.19:4077: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 30 21:29:25.194670346 (175h59m12.998134947s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194691442 (175h59m12.99815604s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: read tcp 139.59.144.22:59584->77.90.31.17:4770: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 30 21:29:25.194706378 (175h59m12.998170983s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194911052 (175h59m12.998375653s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: context canceled; context canceled Sep 30 21:29:25.194921667 (175h59m12.998386269s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194946997 (175h59m12.998411575s) [seg_pos={0 0}, upload=154939]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 30 21:29:25.195081879 (175h59m12.998546481s) [upload=154939]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 30 21:29:25.195089364 (175h59m12.998553965s) [upload=154939]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 30 21:29:25.348637798 (175h59m13.1521024s) [upload=154939]: Done uploading segments... Sep 30 21:29:25.348680254 (175h59m13.152144854s) [upload=154939]: Upload finished. err: <nil> Duration: 2.392517156s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 2.41193387s FAIL (2025-09-30 22:08:29 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 30 21:29:22.797897319 (175h59m10.601361902s) [upload=154939]: Starting upload Sep 30 21:29:22.798006215 (175h59m10.601470808s) [upload=154939]: Done starting upload Sep 30 21:29:22.798081644 (175h59m10.601546229s) [upload=154939]: Uploading segments... Sep 30 21:29:22.798290989 (175h59m10.60175557s) [upload=154939]: Got next segment. Inline: false Sep 30 21:29:22.798331474 (175h59m10.601796067s) [upload=154939]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 30 21:29:22.798339902 (175h59m10.601804489s) [upload=154939]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 21:29:22.915769149 (175h59m10.719233752s) [seg_pos={0 0}, upload=154939]: Begin upload segment... Sep 30 21:29:22.915871743 (175h59m10.719336336s) [seg_pos={0 0}, upload=154939]: Done begin upload segment. Sep 30 21:29:22.915919849 (175h59m10.719384451s) [noise=true , address=45.91.116.4:4940 , piece_id=7DW6Y2G3IUR74PJ2ENFV6XLGTBZTISG4BQFI53KSUJ7HUMYIOSTA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.917360095 (175h59m10.720824697s) [noise=true , address=145.79.227.3:4151 , piece_id=LOVPJ4STVGHTGVOBIS5IMXQDUT7QNS55EFEDPZKD65WLBKTEO3OA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.917523118 (175h59m10.720987717s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918065655 (175h59m10.721530259s) [noise=true , address=145.79.227.13:4801 , piece_id=2W5CLWLLIUYIFCFUW7BBO5MGURJ6YBUNAQDVJCJ2F2LSVX3SLI5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918188240 (175h59m10.721652839s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918605397 (175h59m10.722069997s) [noise=true , address=45.91.116.5:4298 , piece_id=DUBHMFQQHF23TXRYMRODBVKGVCLAE6TG7IPEKTBHUE72WEX5O2SA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918717812 (175h59m10.722182405s) [noise=true , address=145.79.227.16:4910 , piece_id=TBS644O7MTLBU7635SRB2N46WHQK6X5KYRYVITTZZAUFPEKC7GDQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919113523 (175h59m10.722578125s) [noise=true , address=145.79.227.7:4295 , piece_id=VLXVHJOSXZ627O6VCWPEL3H3QPLLSK7OZWGRCSRRT3MIPJDB5YBQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919331578 (175h59m10.722796181s) [noise=true , address=145.79.227.18:4005 , piece_id=IYY76ITHCMY4Z22VRD7WRX3Y6BDVLFKOQ2PCQRU7WFBE6VDQKV5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919626618 (175h59m10.72309122s) [noise=true , address=45.91.116.10:4082 , piece_id=UWRIMN2P47HUKBYHCATAARGFTJXFGQZ7KRV5MUMIQ4GQRFQO7D6Q, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920025044 (175h59m10.723489653s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920253240 (175h59m10.723717844s) [noise=true , address=145.79.227.13:4802 , piece_id=Y5SXFQYQF32UFPHKIMIMI3V7WYDHGSZCPONZTZS5EVVKYSPSQONA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920729685 (175h59m10.724194286s) [noise=true , address=145.79.227.11:4731 , piece_id=EDPUPZOFFO7TQWD7MI5MYMZSDONWB22VGCMZBZPMZA5MB3LOWXAA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.921315566 (175h59m10.724780168s) [noise=true , address=145.79.227.6:4257 , piece_id=5YAEQS2PX6L4EXAKWT33WBMWXFRFDKTFKV7CBCPLO4BYCZKKA53A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.923000023 (175h59m10.726464628s) [noise=true , address=145.79.227.2:4114 , piece_id=V2U5JQ6Y5XJOQSXE6EKKQOA7F3ENKEKWTFOQ65LZV6BBRSK6YD5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.923798468 (175h59m10.727263073s) [noise=true , address=45.91.116.10:4081 , piece_id=WJKNHHM37DDABHFA6LPCF35FUW4SEMNMINZTPXDMJULW7TAL22OQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:23.539151225 (175h59m11.342615808s) [upload=154939]: Next returned nil segment Sep 30 21:29:23.539169161 (175h59m11.34263376s) [upload=154939]: Waiting for error group managing segments... Sep 30 21:29:23.685441552 (175h59m11.488906147s) [noise=true , address=45.91.116.4:4940 , piece_id=7DW6Y2G3IUR74PJ2ENFV6XLGTBZTISG4BQFI53KSUJ7HUMYIOSTA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.688251132 (175h59m11.491715718s) [noise=true , address=145.79.227.7:4295 , piece_id=VLXVHJOSXZ627O6VCWPEL3H3QPLLSK7OZWGRCSRRT3MIPJDB5YBQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.691906915 (175h59m11.495371495s) [noise=true , address=145.79.227.16:4910 , piece_id=TBS644O7MTLBU7635SRB2N46WHQK6X5KYRYVITTZZAUFPEKC7GDQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.698112841 (175h59m11.501577422s) [noise=true , address=145.79.227.6:4257 , piece_id=5YAEQS2PX6L4EXAKWT33WBMWXFRFDKTFKV7CBCPLO4BYCZKKA53A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.698258894 (175h59m11.501723478s) [noise=true , address=145.79.227.2:4114 , piece_id=V2U5JQ6Y5XJOQSXE6EKKQOA7F3ENKEKWTFOQ65LZV6BBRSK6YD5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.702276005 (175h59m11.505740586s) [noise=true , address=45.91.116.10:4081 , piece_id=WJKNHHM37DDABHFA6LPCF35FUW4SEMNMINZTPXDMJULW7TAL22OQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.707342122 (175h59m11.51080671s) [noise=true , address=145.79.227.13:4802 , piece_id=Y5SXFQYQF32UFPHKIMIMI3V7WYDHGSZCPONZTZS5EVVKYSPSQONA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.711442145 (175h59m11.51490673s) [noise=true , address=145.79.227.11:4731 , piece_id=EDPUPZOFFO7TQWD7MI5MYMZSDONWB22VGCMZBZPMZA5MB3LOWXAA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.719319539 (175h59m11.522784123s) [noise=true , address=145.79.227.3:4151 , piece_id=LOVPJ4STVGHTGVOBIS5IMXQDUT7QNS55EFEDPZKD65WLBKTEO3OA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.988493881 (175h59m11.791958458s) [noise=true , address=45.91.116.5:4298 , piece_id=DUBHMFQQHF23TXRYMRODBVKGVCLAE6TG7IPEKTBHUE72WEX5O2SA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:24.038956565 (175h59m11.842421143s) [noise=true , address=45.91.116.10:4082 , piece_id=UWRIMN2P47HUKBYHCATAARGFTJXFGQZ7KRV5MUMIQ4GQRFQO7D6Q, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:24.589378044 (175h59m12.392842628s) [noise=true , address=145.79.227.18:4005 , piece_id=IYY76ITHCMY4Z22VRD7WRX3Y6BDVLFKOQ2PCQRU7WFBE6VDQKV5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:25.194244661 (175h59m12.99770926s) [noise=true , address=145.79.227.13:4801 , piece_id=2W5CLWLLIUYIFCFUW7BBO5MGURJ6YBUNAQDVJCJ2F2LSVX3SLI5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:25.194299398 (175h59m12.997763991s) [seg_pos={0 0}, upload=154939]: Segment reached optimal threshold of 13 pieces. Sep 30 21:29:25.194653994 (175h59m12.99811859s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk): protocol: read tcp 139.59.144.22:34818->145.79.227.19:4077: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 30 21:29:25.194670346 (175h59m12.998134947s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194691442 (175h59m12.99815604s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: read tcp 139.59.144.22:59584->77.90.31.17:4770: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 30 21:29:25.194706378 (175h59m12.998170983s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194911052 (175h59m12.998375653s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: context canceled; context canceled Sep 30 21:29:25.194921667 (175h59m12.998386269s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194946997 (175h59m12.998411575s) [seg_pos={0 0}, upload=154939]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 30 21:29:25.195081879 (175h59m12.998546481s) [upload=154939]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 30 21:29:25.195089364 (175h59m12.998553965s) [upload=154939]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 30 21:29:25.348637798 (175h59m13.1521024s) [upload=154939]: Done uploading segments... Sep 30 21:29:25.348680254 (175h59m13.152144854s) [upload=154939]: Upload finished. err: <nil> Duration: 2.411981715s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 2.536644936s FAIL (2025-09-30 22:08:05 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 30 21:29:22.797897319 (175h59m10.601361902s) [upload=154939]: Starting upload Sep 30 21:29:22.798006215 (175h59m10.601470808s) [upload=154939]: Done starting upload Sep 30 21:29:22.798081644 (175h59m10.601546229s) [upload=154939]: Uploading segments... Sep 30 21:29:22.798290989 (175h59m10.60175557s) [upload=154939]: Got next segment. Inline: false Sep 30 21:29:22.798331474 (175h59m10.601796067s) [upload=154939]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 30 21:29:22.798339902 (175h59m10.601804489s) [upload=154939]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 21:29:22.915769149 (175h59m10.719233752s) [seg_pos={0 0}, upload=154939]: Begin upload segment... Sep 30 21:29:22.915871743 (175h59m10.719336336s) [seg_pos={0 0}, upload=154939]: Done begin upload segment. Sep 30 21:29:22.915919849 (175h59m10.719384451s) [noise=true , address=45.91.116.4:4940 , piece_id=7DW6Y2G3IUR74PJ2ENFV6XLGTBZTISG4BQFI53KSUJ7HUMYIOSTA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.917360095 (175h59m10.720824697s) [noise=true , address=145.79.227.3:4151 , piece_id=LOVPJ4STVGHTGVOBIS5IMXQDUT7QNS55EFEDPZKD65WLBKTEO3OA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.917523118 (175h59m10.720987717s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918065655 (175h59m10.721530259s) [noise=true , address=145.79.227.13:4801 , piece_id=2W5CLWLLIUYIFCFUW7BBO5MGURJ6YBUNAQDVJCJ2F2LSVX3SLI5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918188240 (175h59m10.721652839s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918605397 (175h59m10.722069997s) [noise=true , address=45.91.116.5:4298 , piece_id=DUBHMFQQHF23TXRYMRODBVKGVCLAE6TG7IPEKTBHUE72WEX5O2SA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918717812 (175h59m10.722182405s) [noise=true , address=145.79.227.16:4910 , piece_id=TBS644O7MTLBU7635SRB2N46WHQK6X5KYRYVITTZZAUFPEKC7GDQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919113523 (175h59m10.722578125s) [noise=true , address=145.79.227.7:4295 , piece_id=VLXVHJOSXZ627O6VCWPEL3H3QPLLSK7OZWGRCSRRT3MIPJDB5YBQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919331578 (175h59m10.722796181s) [noise=true , address=145.79.227.18:4005 , piece_id=IYY76ITHCMY4Z22VRD7WRX3Y6BDVLFKOQ2PCQRU7WFBE6VDQKV5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919626618 (175h59m10.72309122s) [noise=true , address=45.91.116.10:4082 , piece_id=UWRIMN2P47HUKBYHCATAARGFTJXFGQZ7KRV5MUMIQ4GQRFQO7D6Q, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920025044 (175h59m10.723489653s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920253240 (175h59m10.723717844s) [noise=true , address=145.79.227.13:4802 , piece_id=Y5SXFQYQF32UFPHKIMIMI3V7WYDHGSZCPONZTZS5EVVKYSPSQONA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920729685 (175h59m10.724194286s) [noise=true , address=145.79.227.11:4731 , piece_id=EDPUPZOFFO7TQWD7MI5MYMZSDONWB22VGCMZBZPMZA5MB3LOWXAA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.921315566 (175h59m10.724780168s) [noise=true , address=145.79.227.6:4257 , piece_id=5YAEQS2PX6L4EXAKWT33WBMWXFRFDKTFKV7CBCPLO4BYCZKKA53A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.923000023 (175h59m10.726464628s) [noise=true , address=145.79.227.2:4114 , piece_id=V2U5JQ6Y5XJOQSXE6EKKQOA7F3ENKEKWTFOQ65LZV6BBRSK6YD5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.923798468 (175h59m10.727263073s) [noise=true , address=45.91.116.10:4081 , piece_id=WJKNHHM37DDABHFA6LPCF35FUW4SEMNMINZTPXDMJULW7TAL22OQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:23.539151225 (175h59m11.342615808s) [upload=154939]: Next returned nil segment Sep 30 21:29:23.539169161 (175h59m11.34263376s) [upload=154939]: Waiting for error group managing segments... Sep 30 21:29:23.685441552 (175h59m11.488906147s) [noise=true , address=45.91.116.4:4940 , piece_id=7DW6Y2G3IUR74PJ2ENFV6XLGTBZTISG4BQFI53KSUJ7HUMYIOSTA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.688251132 (175h59m11.491715718s) [noise=true , address=145.79.227.7:4295 , piece_id=VLXVHJOSXZ627O6VCWPEL3H3QPLLSK7OZWGRCSRRT3MIPJDB5YBQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.691906915 (175h59m11.495371495s) [noise=true , address=145.79.227.16:4910 , piece_id=TBS644O7MTLBU7635SRB2N46WHQK6X5KYRYVITTZZAUFPEKC7GDQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.698112841 (175h59m11.501577422s) [noise=true , address=145.79.227.6:4257 , piece_id=5YAEQS2PX6L4EXAKWT33WBMWXFRFDKTFKV7CBCPLO4BYCZKKA53A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.698258894 (175h59m11.501723478s) [noise=true , address=145.79.227.2:4114 , piece_id=V2U5JQ6Y5XJOQSXE6EKKQOA7F3ENKEKWTFOQ65LZV6BBRSK6YD5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.702276005 (175h59m11.505740586s) [noise=true , address=45.91.116.10:4081 , piece_id=WJKNHHM37DDABHFA6LPCF35FUW4SEMNMINZTPXDMJULW7TAL22OQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.707342122 (175h59m11.51080671s) [noise=true , address=145.79.227.13:4802 , piece_id=Y5SXFQYQF32UFPHKIMIMI3V7WYDHGSZCPONZTZS5EVVKYSPSQONA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.711442145 (175h59m11.51490673s) [noise=true , address=145.79.227.11:4731 , piece_id=EDPUPZOFFO7TQWD7MI5MYMZSDONWB22VGCMZBZPMZA5MB3LOWXAA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.719319539 (175h59m11.522784123s) [noise=true , address=145.79.227.3:4151 , piece_id=LOVPJ4STVGHTGVOBIS5IMXQDUT7QNS55EFEDPZKD65WLBKTEO3OA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.988493881 (175h59m11.791958458s) [noise=true , address=45.91.116.5:4298 , piece_id=DUBHMFQQHF23TXRYMRODBVKGVCLAE6TG7IPEKTBHUE72WEX5O2SA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:24.038956565 (175h59m11.842421143s) [noise=true , address=45.91.116.10:4082 , piece_id=UWRIMN2P47HUKBYHCATAARGFTJXFGQZ7KRV5MUMIQ4GQRFQO7D6Q, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:24.589378044 (175h59m12.392842628s) [noise=true , address=145.79.227.18:4005 , piece_id=IYY76ITHCMY4Z22VRD7WRX3Y6BDVLFKOQ2PCQRU7WFBE6VDQKV5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:25.194244661 (175h59m12.99770926s) [noise=true , address=145.79.227.13:4801 , piece_id=2W5CLWLLIUYIFCFUW7BBO5MGURJ6YBUNAQDVJCJ2F2LSVX3SLI5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:25.194299398 (175h59m12.997763991s) [seg_pos={0 0}, upload=154939]: Segment reached optimal threshold of 13 pieces. Sep 30 21:29:25.194653994 (175h59m12.99811859s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk): protocol: read tcp 139.59.144.22:34818->145.79.227.19:4077: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 30 21:29:25.194670346 (175h59m12.998134947s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194691442 (175h59m12.99815604s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: read tcp 139.59.144.22:59584->77.90.31.17:4770: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 30 21:29:25.194706378 (175h59m12.998170983s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194911052 (175h59m12.998375653s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: context canceled; context canceled Sep 30 21:29:25.194921667 (175h59m12.998386269s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194946997 (175h59m12.998411575s) [seg_pos={0 0}, upload=154939]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 30 21:29:25.195081879 (175h59m12.998546481s) [upload=154939]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 30 21:29:25.195089364 (175h59m12.998553965s) [upload=154939]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 30 21:29:25.348637798 (175h59m13.1521024s) [upload=154939]: Done uploading segments... Sep 30 21:29:25.348680254 (175h59m13.152144854s) [upload=154939]: Upload finished. err: <nil> Duration: 2.536680712s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 2.64420294s FAIL (2025-09-30 22:07:28 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 30 21:29:22.797897319 (175h59m10.601361902s) [upload=154939]: Starting upload Sep 30 21:29:22.798006215 (175h59m10.601470808s) [upload=154939]: Done starting upload Sep 30 21:29:22.798081644 (175h59m10.601546229s) [upload=154939]: Uploading segments... Sep 30 21:29:22.798290989 (175h59m10.60175557s) [upload=154939]: Got next segment. Inline: false Sep 30 21:29:22.798331474 (175h59m10.601796067s) [upload=154939]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 30 21:29:22.798339902 (175h59m10.601804489s) [upload=154939]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 21:29:22.915769149 (175h59m10.719233752s) [seg_pos={0 0}, upload=154939]: Begin upload segment... Sep 30 21:29:22.915871743 (175h59m10.719336336s) [seg_pos={0 0}, upload=154939]: Done begin upload segment. Sep 30 21:29:22.915919849 (175h59m10.719384451s) [noise=true , address=45.91.116.4:4940 , piece_id=7DW6Y2G3IUR74PJ2ENFV6XLGTBZTISG4BQFI53KSUJ7HUMYIOSTA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.917360095 (175h59m10.720824697s) [noise=true , address=145.79.227.3:4151 , piece_id=LOVPJ4STVGHTGVOBIS5IMXQDUT7QNS55EFEDPZKD65WLBKTEO3OA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.917523118 (175h59m10.720987717s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918065655 (175h59m10.721530259s) [noise=true , address=145.79.227.13:4801 , piece_id=2W5CLWLLIUYIFCFUW7BBO5MGURJ6YBUNAQDVJCJ2F2LSVX3SLI5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918188240 (175h59m10.721652839s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918605397 (175h59m10.722069997s) [noise=true , address=45.91.116.5:4298 , piece_id=DUBHMFQQHF23TXRYMRODBVKGVCLAE6TG7IPEKTBHUE72WEX5O2SA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918717812 (175h59m10.722182405s) [noise=true , address=145.79.227.16:4910 , piece_id=TBS644O7MTLBU7635SRB2N46WHQK6X5KYRYVITTZZAUFPEKC7GDQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919113523 (175h59m10.722578125s) [noise=true , address=145.79.227.7:4295 , piece_id=VLXVHJOSXZ627O6VCWPEL3H3QPLLSK7OZWGRCSRRT3MIPJDB5YBQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919331578 (175h59m10.722796181s) [noise=true , address=145.79.227.18:4005 , piece_id=IYY76ITHCMY4Z22VRD7WRX3Y6BDVLFKOQ2PCQRU7WFBE6VDQKV5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919626618 (175h59m10.72309122s) [noise=true , address=45.91.116.10:4082 , piece_id=UWRIMN2P47HUKBYHCATAARGFTJXFGQZ7KRV5MUMIQ4GQRFQO7D6Q, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920025044 (175h59m10.723489653s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920253240 (175h59m10.723717844s) [noise=true , address=145.79.227.13:4802 , piece_id=Y5SXFQYQF32UFPHKIMIMI3V7WYDHGSZCPONZTZS5EVVKYSPSQONA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920729685 (175h59m10.724194286s) [noise=true , address=145.79.227.11:4731 , piece_id=EDPUPZOFFO7TQWD7MI5MYMZSDONWB22VGCMZBZPMZA5MB3LOWXAA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.921315566 (175h59m10.724780168s) [noise=true , address=145.79.227.6:4257 , piece_id=5YAEQS2PX6L4EXAKWT33WBMWXFRFDKTFKV7CBCPLO4BYCZKKA53A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.923000023 (175h59m10.726464628s) [noise=true , address=145.79.227.2:4114 , piece_id=V2U5JQ6Y5XJOQSXE6EKKQOA7F3ENKEKWTFOQ65LZV6BBRSK6YD5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.923798468 (175h59m10.727263073s) [noise=true , address=45.91.116.10:4081 , piece_id=WJKNHHM37DDABHFA6LPCF35FUW4SEMNMINZTPXDMJULW7TAL22OQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:23.539151225 (175h59m11.342615808s) [upload=154939]: Next returned nil segment Sep 30 21:29:23.539169161 (175h59m11.34263376s) [upload=154939]: Waiting for error group managing segments... Sep 30 21:29:23.685441552 (175h59m11.488906147s) [noise=true , address=45.91.116.4:4940 , piece_id=7DW6Y2G3IUR74PJ2ENFV6XLGTBZTISG4BQFI53KSUJ7HUMYIOSTA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.688251132 (175h59m11.491715718s) [noise=true , address=145.79.227.7:4295 , piece_id=VLXVHJOSXZ627O6VCWPEL3H3QPLLSK7OZWGRCSRRT3MIPJDB5YBQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.691906915 (175h59m11.495371495s) [noise=true , address=145.79.227.16:4910 , piece_id=TBS644O7MTLBU7635SRB2N46WHQK6X5KYRYVITTZZAUFPEKC7GDQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.698112841 (175h59m11.501577422s) [noise=true , address=145.79.227.6:4257 , piece_id=5YAEQS2PX6L4EXAKWT33WBMWXFRFDKTFKV7CBCPLO4BYCZKKA53A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.698258894 (175h59m11.501723478s) [noise=true , address=145.79.227.2:4114 , piece_id=V2U5JQ6Y5XJOQSXE6EKKQOA7F3ENKEKWTFOQ65LZV6BBRSK6YD5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.702276005 (175h59m11.505740586s) [noise=true , address=45.91.116.10:4081 , piece_id=WJKNHHM37DDABHFA6LPCF35FUW4SEMNMINZTPXDMJULW7TAL22OQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.707342122 (175h59m11.51080671s) [noise=true , address=145.79.227.13:4802 , piece_id=Y5SXFQYQF32UFPHKIMIMI3V7WYDHGSZCPONZTZS5EVVKYSPSQONA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.711442145 (175h59m11.51490673s) [noise=true , address=145.79.227.11:4731 , piece_id=EDPUPZOFFO7TQWD7MI5MYMZSDONWB22VGCMZBZPMZA5MB3LOWXAA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.719319539 (175h59m11.522784123s) [noise=true , address=145.79.227.3:4151 , piece_id=LOVPJ4STVGHTGVOBIS5IMXQDUT7QNS55EFEDPZKD65WLBKTEO3OA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.988493881 (175h59m11.791958458s) [noise=true , address=45.91.116.5:4298 , piece_id=DUBHMFQQHF23TXRYMRODBVKGVCLAE6TG7IPEKTBHUE72WEX5O2SA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:24.038956565 (175h59m11.842421143s) [noise=true , address=45.91.116.10:4082 , piece_id=UWRIMN2P47HUKBYHCATAARGFTJXFGQZ7KRV5MUMIQ4GQRFQO7D6Q, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:24.589378044 (175h59m12.392842628s) [noise=true , address=145.79.227.18:4005 , piece_id=IYY76ITHCMY4Z22VRD7WRX3Y6BDVLFKOQ2PCQRU7WFBE6VDQKV5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:25.194244661 (175h59m12.99770926s) [noise=true , address=145.79.227.13:4801 , piece_id=2W5CLWLLIUYIFCFUW7BBO5MGURJ6YBUNAQDVJCJ2F2LSVX3SLI5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:25.194299398 (175h59m12.997763991s) [seg_pos={0 0}, upload=154939]: Segment reached optimal threshold of 13 pieces. Sep 30 21:29:25.194653994 (175h59m12.99811859s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk): protocol: read tcp 139.59.144.22:34818->145.79.227.19:4077: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 30 21:29:25.194670346 (175h59m12.998134947s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194691442 (175h59m12.99815604s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: read tcp 139.59.144.22:59584->77.90.31.17:4770: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 30 21:29:25.194706378 (175h59m12.998170983s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194911052 (175h59m12.998375653s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: context canceled; context canceled Sep 30 21:29:25.194921667 (175h59m12.998386269s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194946997 (175h59m12.998411575s) [seg_pos={0 0}, upload=154939]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 30 21:29:25.195081879 (175h59m12.998546481s) [upload=154939]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 30 21:29:25.195089364 (175h59m12.998553965s) [upload=154939]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 30 21:29:25.348637798 (175h59m13.1521024s) [upload=154939]: Done uploading segments... Sep 30 21:29:25.348680254 (175h59m13.152144854s) [upload=154939]: Upload finished. err: <nil> Duration: 2.644237341s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 2.486618404s FAIL (2025-09-30 22:07:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 30 21:29:22.797897319 (175h59m10.601361902s) [upload=154939]: Starting upload Sep 30 21:29:22.798006215 (175h59m10.601470808s) [upload=154939]: Done starting upload Sep 30 21:29:22.798081644 (175h59m10.601546229s) [upload=154939]: Uploading segments... Sep 30 21:29:22.798290989 (175h59m10.60175557s) [upload=154939]: Got next segment. Inline: false Sep 30 21:29:22.798331474 (175h59m10.601796067s) [upload=154939]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 30 21:29:22.798339902 (175h59m10.601804489s) [upload=154939]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 21:29:22.915769149 (175h59m10.719233752s) [seg_pos={0 0}, upload=154939]: Begin upload segment... Sep 30 21:29:22.915871743 (175h59m10.719336336s) [seg_pos={0 0}, upload=154939]: Done begin upload segment. Sep 30 21:29:22.915919849 (175h59m10.719384451s) [noise=true , address=45.91.116.4:4940 , piece_id=7DW6Y2G3IUR74PJ2ENFV6XLGTBZTISG4BQFI53KSUJ7HUMYIOSTA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.917360095 (175h59m10.720824697s) [noise=true , address=145.79.227.3:4151 , piece_id=LOVPJ4STVGHTGVOBIS5IMXQDUT7QNS55EFEDPZKD65WLBKTEO3OA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.917523118 (175h59m10.720987717s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918065655 (175h59m10.721530259s) [noise=true , address=145.79.227.13:4801 , piece_id=2W5CLWLLIUYIFCFUW7BBO5MGURJ6YBUNAQDVJCJ2F2LSVX3SLI5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918188240 (175h59m10.721652839s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918605397 (175h59m10.722069997s) [noise=true , address=45.91.116.5:4298 , piece_id=DUBHMFQQHF23TXRYMRODBVKGVCLAE6TG7IPEKTBHUE72WEX5O2SA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918717812 (175h59m10.722182405s) [noise=true , address=145.79.227.16:4910 , piece_id=TBS644O7MTLBU7635SRB2N46WHQK6X5KYRYVITTZZAUFPEKC7GDQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919113523 (175h59m10.722578125s) [noise=true , address=145.79.227.7:4295 , piece_id=VLXVHJOSXZ627O6VCWPEL3H3QPLLSK7OZWGRCSRRT3MIPJDB5YBQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919331578 (175h59m10.722796181s) [noise=true , address=145.79.227.18:4005 , piece_id=IYY76ITHCMY4Z22VRD7WRX3Y6BDVLFKOQ2PCQRU7WFBE6VDQKV5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919626618 (175h59m10.72309122s) [noise=true , address=45.91.116.10:4082 , piece_id=UWRIMN2P47HUKBYHCATAARGFTJXFGQZ7KRV5MUMIQ4GQRFQO7D6Q, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920025044 (175h59m10.723489653s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920253240 (175h59m10.723717844s) [noise=true , address=145.79.227.13:4802 , piece_id=Y5SXFQYQF32UFPHKIMIMI3V7WYDHGSZCPONZTZS5EVVKYSPSQONA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920729685 (175h59m10.724194286s) [noise=true , address=145.79.227.11:4731 , piece_id=EDPUPZOFFO7TQWD7MI5MYMZSDONWB22VGCMZBZPMZA5MB3LOWXAA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.921315566 (175h59m10.724780168s) [noise=true , address=145.79.227.6:4257 , piece_id=5YAEQS2PX6L4EXAKWT33WBMWXFRFDKTFKV7CBCPLO4BYCZKKA53A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.923000023 (175h59m10.726464628s) [noise=true , address=145.79.227.2:4114 , piece_id=V2U5JQ6Y5XJOQSXE6EKKQOA7F3ENKEKWTFOQ65LZV6BBRSK6YD5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.923798468 (175h59m10.727263073s) [noise=true , address=45.91.116.10:4081 , piece_id=WJKNHHM37DDABHFA6LPCF35FUW4SEMNMINZTPXDMJULW7TAL22OQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:23.539151225 (175h59m11.342615808s) [upload=154939]: Next returned nil segment Sep 30 21:29:23.539169161 (175h59m11.34263376s) [upload=154939]: Waiting for error group managing segments... Sep 30 21:29:23.685441552 (175h59m11.488906147s) [noise=true , address=45.91.116.4:4940 , piece_id=7DW6Y2G3IUR74PJ2ENFV6XLGTBZTISG4BQFI53KSUJ7HUMYIOSTA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.688251132 (175h59m11.491715718s) [noise=true , address=145.79.227.7:4295 , piece_id=VLXVHJOSXZ627O6VCWPEL3H3QPLLSK7OZWGRCSRRT3MIPJDB5YBQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.691906915 (175h59m11.495371495s) [noise=true , address=145.79.227.16:4910 , piece_id=TBS644O7MTLBU7635SRB2N46WHQK6X5KYRYVITTZZAUFPEKC7GDQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.698112841 (175h59m11.501577422s) [noise=true , address=145.79.227.6:4257 , piece_id=5YAEQS2PX6L4EXAKWT33WBMWXFRFDKTFKV7CBCPLO4BYCZKKA53A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.698258894 (175h59m11.501723478s) [noise=true , address=145.79.227.2:4114 , piece_id=V2U5JQ6Y5XJOQSXE6EKKQOA7F3ENKEKWTFOQ65LZV6BBRSK6YD5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.702276005 (175h59m11.505740586s) [noise=true , address=45.91.116.10:4081 , piece_id=WJKNHHM37DDABHFA6LPCF35FUW4SEMNMINZTPXDMJULW7TAL22OQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.707342122 (175h59m11.51080671s) [noise=true , address=145.79.227.13:4802 , piece_id=Y5SXFQYQF32UFPHKIMIMI3V7WYDHGSZCPONZTZS5EVVKYSPSQONA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.711442145 (175h59m11.51490673s) [noise=true , address=145.79.227.11:4731 , piece_id=EDPUPZOFFO7TQWD7MI5MYMZSDONWB22VGCMZBZPMZA5MB3LOWXAA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.719319539 (175h59m11.522784123s) [noise=true , address=145.79.227.3:4151 , piece_id=LOVPJ4STVGHTGVOBIS5IMXQDUT7QNS55EFEDPZKD65WLBKTEO3OA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.988493881 (175h59m11.791958458s) [noise=true , address=45.91.116.5:4298 , piece_id=DUBHMFQQHF23TXRYMRODBVKGVCLAE6TG7IPEKTBHUE72WEX5O2SA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:24.038956565 (175h59m11.842421143s) [noise=true , address=45.91.116.10:4082 , piece_id=UWRIMN2P47HUKBYHCATAARGFTJXFGQZ7KRV5MUMIQ4GQRFQO7D6Q, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:24.589378044 (175h59m12.392842628s) [noise=true , address=145.79.227.18:4005 , piece_id=IYY76ITHCMY4Z22VRD7WRX3Y6BDVLFKOQ2PCQRU7WFBE6VDQKV5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:25.194244661 (175h59m12.99770926s) [noise=true , address=145.79.227.13:4801 , piece_id=2W5CLWLLIUYIFCFUW7BBO5MGURJ6YBUNAQDVJCJ2F2LSVX3SLI5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:25.194299398 (175h59m12.997763991s) [seg_pos={0 0}, upload=154939]: Segment reached optimal threshold of 13 pieces. Sep 30 21:29:25.194653994 (175h59m12.99811859s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk): protocol: read tcp 139.59.144.22:34818->145.79.227.19:4077: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 30 21:29:25.194670346 (175h59m12.998134947s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194691442 (175h59m12.99815604s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: read tcp 139.59.144.22:59584->77.90.31.17:4770: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 30 21:29:25.194706378 (175h59m12.998170983s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194911052 (175h59m12.998375653s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: context canceled; context canceled Sep 30 21:29:25.194921667 (175h59m12.998386269s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194946997 (175h59m12.998411575s) [seg_pos={0 0}, upload=154939]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 30 21:29:25.195081879 (175h59m12.998546481s) [upload=154939]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 30 21:29:25.195089364 (175h59m12.998553965s) [upload=154939]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 30 21:29:25.348637798 (175h59m13.1521024s) [upload=154939]: Done uploading segments... Sep 30 21:29:25.348680254 (175h59m13.152144854s) [upload=154939]: Upload finished. err: <nil> Duration: 2.486652135s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 2.606019667s FAIL (2025-09-30 22:06:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 30 21:29:22.797897319 (175h59m10.601361902s) [upload=154939]: Starting upload Sep 30 21:29:22.798006215 (175h59m10.601470808s) [upload=154939]: Done starting upload Sep 30 21:29:22.798081644 (175h59m10.601546229s) [upload=154939]: Uploading segments... Sep 30 21:29:22.798290989 (175h59m10.60175557s) [upload=154939]: Got next segment. Inline: false Sep 30 21:29:22.798331474 (175h59m10.601796067s) [upload=154939]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 30 21:29:22.798339902 (175h59m10.601804489s) [upload=154939]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 21:29:22.915769149 (175h59m10.719233752s) [seg_pos={0 0}, upload=154939]: Begin upload segment... Sep 30 21:29:22.915871743 (175h59m10.719336336s) [seg_pos={0 0}, upload=154939]: Done begin upload segment. Sep 30 21:29:22.915919849 (175h59m10.719384451s) [noise=true , address=45.91.116.4:4940 , piece_id=7DW6Y2G3IUR74PJ2ENFV6XLGTBZTISG4BQFI53KSUJ7HUMYIOSTA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.917360095 (175h59m10.720824697s) [noise=true , address=145.79.227.3:4151 , piece_id=LOVPJ4STVGHTGVOBIS5IMXQDUT7QNS55EFEDPZKD65WLBKTEO3OA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.917523118 (175h59m10.720987717s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918065655 (175h59m10.721530259s) [noise=true , address=145.79.227.13:4801 , piece_id=2W5CLWLLIUYIFCFUW7BBO5MGURJ6YBUNAQDVJCJ2F2LSVX3SLI5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918188240 (175h59m10.721652839s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918605397 (175h59m10.722069997s) [noise=true , address=45.91.116.5:4298 , piece_id=DUBHMFQQHF23TXRYMRODBVKGVCLAE6TG7IPEKTBHUE72WEX5O2SA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918717812 (175h59m10.722182405s) [noise=true , address=145.79.227.16:4910 , piece_id=TBS644O7MTLBU7635SRB2N46WHQK6X5KYRYVITTZZAUFPEKC7GDQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919113523 (175h59m10.722578125s) [noise=true , address=145.79.227.7:4295 , piece_id=VLXVHJOSXZ627O6VCWPEL3H3QPLLSK7OZWGRCSRRT3MIPJDB5YBQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919331578 (175h59m10.722796181s) [noise=true , address=145.79.227.18:4005 , piece_id=IYY76ITHCMY4Z22VRD7WRX3Y6BDVLFKOQ2PCQRU7WFBE6VDQKV5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919626618 (175h59m10.72309122s) [noise=true , address=45.91.116.10:4082 , piece_id=UWRIMN2P47HUKBYHCATAARGFTJXFGQZ7KRV5MUMIQ4GQRFQO7D6Q, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920025044 (175h59m10.723489653s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920253240 (175h59m10.723717844s) [noise=true , address=145.79.227.13:4802 , piece_id=Y5SXFQYQF32UFPHKIMIMI3V7WYDHGSZCPONZTZS5EVVKYSPSQONA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920729685 (175h59m10.724194286s) [noise=true , address=145.79.227.11:4731 , piece_id=EDPUPZOFFO7TQWD7MI5MYMZSDONWB22VGCMZBZPMZA5MB3LOWXAA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.921315566 (175h59m10.724780168s) [noise=true , address=145.79.227.6:4257 , piece_id=5YAEQS2PX6L4EXAKWT33WBMWXFRFDKTFKV7CBCPLO4BYCZKKA53A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.923000023 (175h59m10.726464628s) [noise=true , address=145.79.227.2:4114 , piece_id=V2U5JQ6Y5XJOQSXE6EKKQOA7F3ENKEKWTFOQ65LZV6BBRSK6YD5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.923798468 (175h59m10.727263073s) [noise=true , address=45.91.116.10:4081 , piece_id=WJKNHHM37DDABHFA6LPCF35FUW4SEMNMINZTPXDMJULW7TAL22OQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:23.539151225 (175h59m11.342615808s) [upload=154939]: Next returned nil segment Sep 30 21:29:23.539169161 (175h59m11.34263376s) [upload=154939]: Waiting for error group managing segments... Sep 30 21:29:23.685441552 (175h59m11.488906147s) [noise=true , address=45.91.116.4:4940 , piece_id=7DW6Y2G3IUR74PJ2ENFV6XLGTBZTISG4BQFI53KSUJ7HUMYIOSTA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.688251132 (175h59m11.491715718s) [noise=true , address=145.79.227.7:4295 , piece_id=VLXVHJOSXZ627O6VCWPEL3H3QPLLSK7OZWGRCSRRT3MIPJDB5YBQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.691906915 (175h59m11.495371495s) [noise=true , address=145.79.227.16:4910 , piece_id=TBS644O7MTLBU7635SRB2N46WHQK6X5KYRYVITTZZAUFPEKC7GDQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.698112841 (175h59m11.501577422s) [noise=true , address=145.79.227.6:4257 , piece_id=5YAEQS2PX6L4EXAKWT33WBMWXFRFDKTFKV7CBCPLO4BYCZKKA53A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.698258894 (175h59m11.501723478s) [noise=true , address=145.79.227.2:4114 , piece_id=V2U5JQ6Y5XJOQSXE6EKKQOA7F3ENKEKWTFOQ65LZV6BBRSK6YD5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.702276005 (175h59m11.505740586s) [noise=true , address=45.91.116.10:4081 , piece_id=WJKNHHM37DDABHFA6LPCF35FUW4SEMNMINZTPXDMJULW7TAL22OQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.707342122 (175h59m11.51080671s) [noise=true , address=145.79.227.13:4802 , piece_id=Y5SXFQYQF32UFPHKIMIMI3V7WYDHGSZCPONZTZS5EVVKYSPSQONA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.711442145 (175h59m11.51490673s) [noise=true , address=145.79.227.11:4731 , piece_id=EDPUPZOFFO7TQWD7MI5MYMZSDONWB22VGCMZBZPMZA5MB3LOWXAA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.719319539 (175h59m11.522784123s) [noise=true , address=145.79.227.3:4151 , piece_id=LOVPJ4STVGHTGVOBIS5IMXQDUT7QNS55EFEDPZKD65WLBKTEO3OA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.988493881 (175h59m11.791958458s) [noise=true , address=45.91.116.5:4298 , piece_id=DUBHMFQQHF23TXRYMRODBVKGVCLAE6TG7IPEKTBHUE72WEX5O2SA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:24.038956565 (175h59m11.842421143s) [noise=true , address=45.91.116.10:4082 , piece_id=UWRIMN2P47HUKBYHCATAARGFTJXFGQZ7KRV5MUMIQ4GQRFQO7D6Q, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:24.589378044 (175h59m12.392842628s) [noise=true , address=145.79.227.18:4005 , piece_id=IYY76ITHCMY4Z22VRD7WRX3Y6BDVLFKOQ2PCQRU7WFBE6VDQKV5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:25.194244661 (175h59m12.99770926s) [noise=true , address=145.79.227.13:4801 , piece_id=2W5CLWLLIUYIFCFUW7BBO5MGURJ6YBUNAQDVJCJ2F2LSVX3SLI5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:25.194299398 (175h59m12.997763991s) [seg_pos={0 0}, upload=154939]: Segment reached optimal threshold of 13 pieces. Sep 30 21:29:25.194653994 (175h59m12.99811859s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk): protocol: read tcp 139.59.144.22:34818->145.79.227.19:4077: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 30 21:29:25.194670346 (175h59m12.998134947s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194691442 (175h59m12.99815604s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: read tcp 139.59.144.22:59584->77.90.31.17:4770: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 30 21:29:25.194706378 (175h59m12.998170983s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194911052 (175h59m12.998375653s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: context canceled; context canceled Sep 30 21:29:25.194921667 (175h59m12.998386269s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194946997 (175h59m12.998411575s) [seg_pos={0 0}, upload=154939]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 30 21:29:25.195081879 (175h59m12.998546481s) [upload=154939]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 30 21:29:25.195089364 (175h59m12.998553965s) [upload=154939]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 30 21:29:25.348637798 (175h59m13.1521024s) [upload=154939]: Done uploading segments... Sep 30 21:29:25.348680254 (175h59m13.152144854s) [upload=154939]: Upload finished. err: <nil> Duration: 2.60605171s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 2.63664793s FAIL (2025-09-30 22:05:28 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 30 21:29:22.797897319 (175h59m10.601361902s) [upload=154939]: Starting upload Sep 30 21:29:22.798006215 (175h59m10.601470808s) [upload=154939]: Done starting upload Sep 30 21:29:22.798081644 (175h59m10.601546229s) [upload=154939]: Uploading segments... Sep 30 21:29:22.798290989 (175h59m10.60175557s) [upload=154939]: Got next segment. Inline: false Sep 30 21:29:22.798331474 (175h59m10.601796067s) [upload=154939]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 30 21:29:22.798339902 (175h59m10.601804489s) [upload=154939]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 21:29:22.915769149 (175h59m10.719233752s) [seg_pos={0 0}, upload=154939]: Begin upload segment... Sep 30 21:29:22.915871743 (175h59m10.719336336s) [seg_pos={0 0}, upload=154939]: Done begin upload segment. Sep 30 21:29:22.915919849 (175h59m10.719384451s) [noise=true , address=45.91.116.4:4940 , piece_id=7DW6Y2G3IUR74PJ2ENFV6XLGTBZTISG4BQFI53KSUJ7HUMYIOSTA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.917360095 (175h59m10.720824697s) [noise=true , address=145.79.227.3:4151 , piece_id=LOVPJ4STVGHTGVOBIS5IMXQDUT7QNS55EFEDPZKD65WLBKTEO3OA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.917523118 (175h59m10.720987717s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918065655 (175h59m10.721530259s) [noise=true , address=145.79.227.13:4801 , piece_id=2W5CLWLLIUYIFCFUW7BBO5MGURJ6YBUNAQDVJCJ2F2LSVX3SLI5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918188240 (175h59m10.721652839s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918605397 (175h59m10.722069997s) [noise=true , address=45.91.116.5:4298 , piece_id=DUBHMFQQHF23TXRYMRODBVKGVCLAE6TG7IPEKTBHUE72WEX5O2SA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918717812 (175h59m10.722182405s) [noise=true , address=145.79.227.16:4910 , piece_id=TBS644O7MTLBU7635SRB2N46WHQK6X5KYRYVITTZZAUFPEKC7GDQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919113523 (175h59m10.722578125s) [noise=true , address=145.79.227.7:4295 , piece_id=VLXVHJOSXZ627O6VCWPEL3H3QPLLSK7OZWGRCSRRT3MIPJDB5YBQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919331578 (175h59m10.722796181s) [noise=true , address=145.79.227.18:4005 , piece_id=IYY76ITHCMY4Z22VRD7WRX3Y6BDVLFKOQ2PCQRU7WFBE6VDQKV5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919626618 (175h59m10.72309122s) [noise=true , address=45.91.116.10:4082 , piece_id=UWRIMN2P47HUKBYHCATAARGFTJXFGQZ7KRV5MUMIQ4GQRFQO7D6Q, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920025044 (175h59m10.723489653s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920253240 (175h59m10.723717844s) [noise=true , address=145.79.227.13:4802 , piece_id=Y5SXFQYQF32UFPHKIMIMI3V7WYDHGSZCPONZTZS5EVVKYSPSQONA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920729685 (175h59m10.724194286s) [noise=true , address=145.79.227.11:4731 , piece_id=EDPUPZOFFO7TQWD7MI5MYMZSDONWB22VGCMZBZPMZA5MB3LOWXAA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.921315566 (175h59m10.724780168s) [noise=true , address=145.79.227.6:4257 , piece_id=5YAEQS2PX6L4EXAKWT33WBMWXFRFDKTFKV7CBCPLO4BYCZKKA53A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.923000023 (175h59m10.726464628s) [noise=true , address=145.79.227.2:4114 , piece_id=V2U5JQ6Y5XJOQSXE6EKKQOA7F3ENKEKWTFOQ65LZV6BBRSK6YD5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.923798468 (175h59m10.727263073s) [noise=true , address=45.91.116.10:4081 , piece_id=WJKNHHM37DDABHFA6LPCF35FUW4SEMNMINZTPXDMJULW7TAL22OQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:23.539151225 (175h59m11.342615808s) [upload=154939]: Next returned nil segment Sep 30 21:29:23.539169161 (175h59m11.34263376s) [upload=154939]: Waiting for error group managing segments... Sep 30 21:29:23.685441552 (175h59m11.488906147s) [noise=true , address=45.91.116.4:4940 , piece_id=7DW6Y2G3IUR74PJ2ENFV6XLGTBZTISG4BQFI53KSUJ7HUMYIOSTA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.688251132 (175h59m11.491715718s) [noise=true , address=145.79.227.7:4295 , piece_id=VLXVHJOSXZ627O6VCWPEL3H3QPLLSK7OZWGRCSRRT3MIPJDB5YBQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.691906915 (175h59m11.495371495s) [noise=true , address=145.79.227.16:4910 , piece_id=TBS644O7MTLBU7635SRB2N46WHQK6X5KYRYVITTZZAUFPEKC7GDQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.698112841 (175h59m11.501577422s) [noise=true , address=145.79.227.6:4257 , piece_id=5YAEQS2PX6L4EXAKWT33WBMWXFRFDKTFKV7CBCPLO4BYCZKKA53A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.698258894 (175h59m11.501723478s) [noise=true , address=145.79.227.2:4114 , piece_id=V2U5JQ6Y5XJOQSXE6EKKQOA7F3ENKEKWTFOQ65LZV6BBRSK6YD5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.702276005 (175h59m11.505740586s) [noise=true , address=45.91.116.10:4081 , piece_id=WJKNHHM37DDABHFA6LPCF35FUW4SEMNMINZTPXDMJULW7TAL22OQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.707342122 (175h59m11.51080671s) [noise=true , address=145.79.227.13:4802 , piece_id=Y5SXFQYQF32UFPHKIMIMI3V7WYDHGSZCPONZTZS5EVVKYSPSQONA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.711442145 (175h59m11.51490673s) [noise=true , address=145.79.227.11:4731 , piece_id=EDPUPZOFFO7TQWD7MI5MYMZSDONWB22VGCMZBZPMZA5MB3LOWXAA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.719319539 (175h59m11.522784123s) [noise=true , address=145.79.227.3:4151 , piece_id=LOVPJ4STVGHTGVOBIS5IMXQDUT7QNS55EFEDPZKD65WLBKTEO3OA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.988493881 (175h59m11.791958458s) [noise=true , address=45.91.116.5:4298 , piece_id=DUBHMFQQHF23TXRYMRODBVKGVCLAE6TG7IPEKTBHUE72WEX5O2SA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:24.038956565 (175h59m11.842421143s) [noise=true , address=45.91.116.10:4082 , piece_id=UWRIMN2P47HUKBYHCATAARGFTJXFGQZ7KRV5MUMIQ4GQRFQO7D6Q, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:24.589378044 (175h59m12.392842628s) [noise=true , address=145.79.227.18:4005 , piece_id=IYY76ITHCMY4Z22VRD7WRX3Y6BDVLFKOQ2PCQRU7WFBE6VDQKV5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:25.194244661 (175h59m12.99770926s) [noise=true , address=145.79.227.13:4801 , piece_id=2W5CLWLLIUYIFCFUW7BBO5MGURJ6YBUNAQDVJCJ2F2LSVX3SLI5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:25.194299398 (175h59m12.997763991s) [seg_pos={0 0}, upload=154939]: Segment reached optimal threshold of 13 pieces. Sep 30 21:29:25.194653994 (175h59m12.99811859s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk): protocol: read tcp 139.59.144.22:34818->145.79.227.19:4077: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 30 21:29:25.194670346 (175h59m12.998134947s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194691442 (175h59m12.99815604s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: read tcp 139.59.144.22:59584->77.90.31.17:4770: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 30 21:29:25.194706378 (175h59m12.998170983s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194911052 (175h59m12.998375653s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: context canceled; context canceled Sep 30 21:29:25.194921667 (175h59m12.998386269s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194946997 (175h59m12.998411575s) [seg_pos={0 0}, upload=154939]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 30 21:29:25.195081879 (175h59m12.998546481s) [upload=154939]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 30 21:29:25.195089364 (175h59m12.998553965s) [upload=154939]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 30 21:29:25.348637798 (175h59m13.1521024s) [upload=154939]: Done uploading segments... Sep 30 21:29:25.348680254 (175h59m13.152144854s) [upload=154939]: Upload finished. err: <nil> Duration: 2.636677334s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 2.442041809s FAIL (2025-09-30 22:05:05 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 30 21:29:22.797897319 (175h59m10.601361902s) [upload=154939]: Starting upload Sep 30 21:29:22.798006215 (175h59m10.601470808s) [upload=154939]: Done starting upload Sep 30 21:29:22.798081644 (175h59m10.601546229s) [upload=154939]: Uploading segments... Sep 30 21:29:22.798290989 (175h59m10.60175557s) [upload=154939]: Got next segment. Inline: false Sep 30 21:29:22.798331474 (175h59m10.601796067s) [upload=154939]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 30 21:29:22.798339902 (175h59m10.601804489s) [upload=154939]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 21:29:22.915769149 (175h59m10.719233752s) [seg_pos={0 0}, upload=154939]: Begin upload segment... Sep 30 21:29:22.915871743 (175h59m10.719336336s) [seg_pos={0 0}, upload=154939]: Done begin upload segment. Sep 30 21:29:22.915919849 (175h59m10.719384451s) [noise=true , address=45.91.116.4:4940 , piece_id=7DW6Y2G3IUR74PJ2ENFV6XLGTBZTISG4BQFI53KSUJ7HUMYIOSTA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.917360095 (175h59m10.720824697s) [noise=true , address=145.79.227.3:4151 , piece_id=LOVPJ4STVGHTGVOBIS5IMXQDUT7QNS55EFEDPZKD65WLBKTEO3OA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.917523118 (175h59m10.720987717s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918065655 (175h59m10.721530259s) [noise=true , address=145.79.227.13:4801 , piece_id=2W5CLWLLIUYIFCFUW7BBO5MGURJ6YBUNAQDVJCJ2F2LSVX3SLI5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918188240 (175h59m10.721652839s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918605397 (175h59m10.722069997s) [noise=true , address=45.91.116.5:4298 , piece_id=DUBHMFQQHF23TXRYMRODBVKGVCLAE6TG7IPEKTBHUE72WEX5O2SA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918717812 (175h59m10.722182405s) [noise=true , address=145.79.227.16:4910 , piece_id=TBS644O7MTLBU7635SRB2N46WHQK6X5KYRYVITTZZAUFPEKC7GDQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919113523 (175h59m10.722578125s) [noise=true , address=145.79.227.7:4295 , piece_id=VLXVHJOSXZ627O6VCWPEL3H3QPLLSK7OZWGRCSRRT3MIPJDB5YBQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919331578 (175h59m10.722796181s) [noise=true , address=145.79.227.18:4005 , piece_id=IYY76ITHCMY4Z22VRD7WRX3Y6BDVLFKOQ2PCQRU7WFBE6VDQKV5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919626618 (175h59m10.72309122s) [noise=true , address=45.91.116.10:4082 , piece_id=UWRIMN2P47HUKBYHCATAARGFTJXFGQZ7KRV5MUMIQ4GQRFQO7D6Q, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920025044 (175h59m10.723489653s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920253240 (175h59m10.723717844s) [noise=true , address=145.79.227.13:4802 , piece_id=Y5SXFQYQF32UFPHKIMIMI3V7WYDHGSZCPONZTZS5EVVKYSPSQONA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920729685 (175h59m10.724194286s) [noise=true , address=145.79.227.11:4731 , piece_id=EDPUPZOFFO7TQWD7MI5MYMZSDONWB22VGCMZBZPMZA5MB3LOWXAA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.921315566 (175h59m10.724780168s) [noise=true , address=145.79.227.6:4257 , piece_id=5YAEQS2PX6L4EXAKWT33WBMWXFRFDKTFKV7CBCPLO4BYCZKKA53A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.923000023 (175h59m10.726464628s) [noise=true , address=145.79.227.2:4114 , piece_id=V2U5JQ6Y5XJOQSXE6EKKQOA7F3ENKEKWTFOQ65LZV6BBRSK6YD5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.923798468 (175h59m10.727263073s) [noise=true , address=45.91.116.10:4081 , piece_id=WJKNHHM37DDABHFA6LPCF35FUW4SEMNMINZTPXDMJULW7TAL22OQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:23.539151225 (175h59m11.342615808s) [upload=154939]: Next returned nil segment Sep 30 21:29:23.539169161 (175h59m11.34263376s) [upload=154939]: Waiting for error group managing segments... Sep 30 21:29:23.685441552 (175h59m11.488906147s) [noise=true , address=45.91.116.4:4940 , piece_id=7DW6Y2G3IUR74PJ2ENFV6XLGTBZTISG4BQFI53KSUJ7HUMYIOSTA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.688251132 (175h59m11.491715718s) [noise=true , address=145.79.227.7:4295 , piece_id=VLXVHJOSXZ627O6VCWPEL3H3QPLLSK7OZWGRCSRRT3MIPJDB5YBQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.691906915 (175h59m11.495371495s) [noise=true , address=145.79.227.16:4910 , piece_id=TBS644O7MTLBU7635SRB2N46WHQK6X5KYRYVITTZZAUFPEKC7GDQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.698112841 (175h59m11.501577422s) [noise=true , address=145.79.227.6:4257 , piece_id=5YAEQS2PX6L4EXAKWT33WBMWXFRFDKTFKV7CBCPLO4BYCZKKA53A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.698258894 (175h59m11.501723478s) [noise=true , address=145.79.227.2:4114 , piece_id=V2U5JQ6Y5XJOQSXE6EKKQOA7F3ENKEKWTFOQ65LZV6BBRSK6YD5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.702276005 (175h59m11.505740586s) [noise=true , address=45.91.116.10:4081 , piece_id=WJKNHHM37DDABHFA6LPCF35FUW4SEMNMINZTPXDMJULW7TAL22OQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.707342122 (175h59m11.51080671s) [noise=true , address=145.79.227.13:4802 , piece_id=Y5SXFQYQF32UFPHKIMIMI3V7WYDHGSZCPONZTZS5EVVKYSPSQONA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.711442145 (175h59m11.51490673s) [noise=true , address=145.79.227.11:4731 , piece_id=EDPUPZOFFO7TQWD7MI5MYMZSDONWB22VGCMZBZPMZA5MB3LOWXAA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.719319539 (175h59m11.522784123s) [noise=true , address=145.79.227.3:4151 , piece_id=LOVPJ4STVGHTGVOBIS5IMXQDUT7QNS55EFEDPZKD65WLBKTEO3OA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.988493881 (175h59m11.791958458s) [noise=true , address=45.91.116.5:4298 , piece_id=DUBHMFQQHF23TXRYMRODBVKGVCLAE6TG7IPEKTBHUE72WEX5O2SA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:24.038956565 (175h59m11.842421143s) [noise=true , address=45.91.116.10:4082 , piece_id=UWRIMN2P47HUKBYHCATAARGFTJXFGQZ7KRV5MUMIQ4GQRFQO7D6Q, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:24.589378044 (175h59m12.392842628s) [noise=true , address=145.79.227.18:4005 , piece_id=IYY76ITHCMY4Z22VRD7WRX3Y6BDVLFKOQ2PCQRU7WFBE6VDQKV5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:25.194244661 (175h59m12.99770926s) [noise=true , address=145.79.227.13:4801 , piece_id=2W5CLWLLIUYIFCFUW7BBO5MGURJ6YBUNAQDVJCJ2F2LSVX3SLI5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:25.194299398 (175h59m12.997763991s) [seg_pos={0 0}, upload=154939]: Segment reached optimal threshold of 13 pieces. Sep 30 21:29:25.194653994 (175h59m12.99811859s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk): protocol: read tcp 139.59.144.22:34818->145.79.227.19:4077: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 30 21:29:25.194670346 (175h59m12.998134947s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194691442 (175h59m12.99815604s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: read tcp 139.59.144.22:59584->77.90.31.17:4770: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 30 21:29:25.194706378 (175h59m12.998170983s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194911052 (175h59m12.998375653s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: context canceled; context canceled Sep 30 21:29:25.194921667 (175h59m12.998386269s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194946997 (175h59m12.998411575s) [seg_pos={0 0}, upload=154939]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 30 21:29:25.195081879 (175h59m12.998546481s) [upload=154939]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 30 21:29:25.195089364 (175h59m12.998553965s) [upload=154939]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 30 21:29:25.348637798 (175h59m13.1521024s) [upload=154939]: Done uploading segments... Sep 30 21:29:25.348680254 (175h59m13.152144854s) [upload=154939]: Upload finished. err: <nil> Duration: 2.442072179s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 2.547015941s FAIL (2025-09-30 22:04:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 30 21:29:22.797897319 (175h59m10.601361902s) [upload=154939]: Starting upload Sep 30 21:29:22.798006215 (175h59m10.601470808s) [upload=154939]: Done starting upload Sep 30 21:29:22.798081644 (175h59m10.601546229s) [upload=154939]: Uploading segments... Sep 30 21:29:22.798290989 (175h59m10.60175557s) [upload=154939]: Got next segment. Inline: false Sep 30 21:29:22.798331474 (175h59m10.601796067s) [upload=154939]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 30 21:29:22.798339902 (175h59m10.601804489s) [upload=154939]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 21:29:22.915769149 (175h59m10.719233752s) [seg_pos={0 0}, upload=154939]: Begin upload segment... Sep 30 21:29:22.915871743 (175h59m10.719336336s) [seg_pos={0 0}, upload=154939]: Done begin upload segment. Sep 30 21:29:22.915919849 (175h59m10.719384451s) [noise=true , address=45.91.116.4:4940 , piece_id=7DW6Y2G3IUR74PJ2ENFV6XLGTBZTISG4BQFI53KSUJ7HUMYIOSTA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.917360095 (175h59m10.720824697s) [noise=true , address=145.79.227.3:4151 , piece_id=LOVPJ4STVGHTGVOBIS5IMXQDUT7QNS55EFEDPZKD65WLBKTEO3OA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.917523118 (175h59m10.720987717s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918065655 (175h59m10.721530259s) [noise=true , address=145.79.227.13:4801 , piece_id=2W5CLWLLIUYIFCFUW7BBO5MGURJ6YBUNAQDVJCJ2F2LSVX3SLI5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918188240 (175h59m10.721652839s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918605397 (175h59m10.722069997s) [noise=true , address=45.91.116.5:4298 , piece_id=DUBHMFQQHF23TXRYMRODBVKGVCLAE6TG7IPEKTBHUE72WEX5O2SA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918717812 (175h59m10.722182405s) [noise=true , address=145.79.227.16:4910 , piece_id=TBS644O7MTLBU7635SRB2N46WHQK6X5KYRYVITTZZAUFPEKC7GDQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919113523 (175h59m10.722578125s) [noise=true , address=145.79.227.7:4295 , piece_id=VLXVHJOSXZ627O6VCWPEL3H3QPLLSK7OZWGRCSRRT3MIPJDB5YBQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919331578 (175h59m10.722796181s) [noise=true , address=145.79.227.18:4005 , piece_id=IYY76ITHCMY4Z22VRD7WRX3Y6BDVLFKOQ2PCQRU7WFBE6VDQKV5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919626618 (175h59m10.72309122s) [noise=true , address=45.91.116.10:4082 , piece_id=UWRIMN2P47HUKBYHCATAARGFTJXFGQZ7KRV5MUMIQ4GQRFQO7D6Q, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920025044 (175h59m10.723489653s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920253240 (175h59m10.723717844s) [noise=true , address=145.79.227.13:4802 , piece_id=Y5SXFQYQF32UFPHKIMIMI3V7WYDHGSZCPONZTZS5EVVKYSPSQONA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920729685 (175h59m10.724194286s) [noise=true , address=145.79.227.11:4731 , piece_id=EDPUPZOFFO7TQWD7MI5MYMZSDONWB22VGCMZBZPMZA5MB3LOWXAA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.921315566 (175h59m10.724780168s) [noise=true , address=145.79.227.6:4257 , piece_id=5YAEQS2PX6L4EXAKWT33WBMWXFRFDKTFKV7CBCPLO4BYCZKKA53A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.923000023 (175h59m10.726464628s) [noise=true , address=145.79.227.2:4114 , piece_id=V2U5JQ6Y5XJOQSXE6EKKQOA7F3ENKEKWTFOQ65LZV6BBRSK6YD5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.923798468 (175h59m10.727263073s) [noise=true , address=45.91.116.10:4081 , piece_id=WJKNHHM37DDABHFA6LPCF35FUW4SEMNMINZTPXDMJULW7TAL22OQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:23.539151225 (175h59m11.342615808s) [upload=154939]: Next returned nil segment Sep 30 21:29:23.539169161 (175h59m11.34263376s) [upload=154939]: Waiting for error group managing segments... Sep 30 21:29:23.685441552 (175h59m11.488906147s) [noise=true , address=45.91.116.4:4940 , piece_id=7DW6Y2G3IUR74PJ2ENFV6XLGTBZTISG4BQFI53KSUJ7HUMYIOSTA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.688251132 (175h59m11.491715718s) [noise=true , address=145.79.227.7:4295 , piece_id=VLXVHJOSXZ627O6VCWPEL3H3QPLLSK7OZWGRCSRRT3MIPJDB5YBQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.691906915 (175h59m11.495371495s) [noise=true , address=145.79.227.16:4910 , piece_id=TBS644O7MTLBU7635SRB2N46WHQK6X5KYRYVITTZZAUFPEKC7GDQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.698112841 (175h59m11.501577422s) [noise=true , address=145.79.227.6:4257 , piece_id=5YAEQS2PX6L4EXAKWT33WBMWXFRFDKTFKV7CBCPLO4BYCZKKA53A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.698258894 (175h59m11.501723478s) [noise=true , address=145.79.227.2:4114 , piece_id=V2U5JQ6Y5XJOQSXE6EKKQOA7F3ENKEKWTFOQ65LZV6BBRSK6YD5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.702276005 (175h59m11.505740586s) [noise=true , address=45.91.116.10:4081 , piece_id=WJKNHHM37DDABHFA6LPCF35FUW4SEMNMINZTPXDMJULW7TAL22OQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.707342122 (175h59m11.51080671s) [noise=true , address=145.79.227.13:4802 , piece_id=Y5SXFQYQF32UFPHKIMIMI3V7WYDHGSZCPONZTZS5EVVKYSPSQONA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.711442145 (175h59m11.51490673s) [noise=true , address=145.79.227.11:4731 , piece_id=EDPUPZOFFO7TQWD7MI5MYMZSDONWB22VGCMZBZPMZA5MB3LOWXAA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.719319539 (175h59m11.522784123s) [noise=true , address=145.79.227.3:4151 , piece_id=LOVPJ4STVGHTGVOBIS5IMXQDUT7QNS55EFEDPZKD65WLBKTEO3OA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.988493881 (175h59m11.791958458s) [noise=true , address=45.91.116.5:4298 , piece_id=DUBHMFQQHF23TXRYMRODBVKGVCLAE6TG7IPEKTBHUE72WEX5O2SA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:24.038956565 (175h59m11.842421143s) [noise=true , address=45.91.116.10:4082 , piece_id=UWRIMN2P47HUKBYHCATAARGFTJXFGQZ7KRV5MUMIQ4GQRFQO7D6Q, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:24.589378044 (175h59m12.392842628s) [noise=true , address=145.79.227.18:4005 , piece_id=IYY76ITHCMY4Z22VRD7WRX3Y6BDVLFKOQ2PCQRU7WFBE6VDQKV5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:25.194244661 (175h59m12.99770926s) [noise=true , address=145.79.227.13:4801 , piece_id=2W5CLWLLIUYIFCFUW7BBO5MGURJ6YBUNAQDVJCJ2F2LSVX3SLI5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:25.194299398 (175h59m12.997763991s) [seg_pos={0 0}, upload=154939]: Segment reached optimal threshold of 13 pieces. Sep 30 21:29:25.194653994 (175h59m12.99811859s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk): protocol: read tcp 139.59.144.22:34818->145.79.227.19:4077: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 30 21:29:25.194670346 (175h59m12.998134947s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194691442 (175h59m12.99815604s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: read tcp 139.59.144.22:59584->77.90.31.17:4770: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 30 21:29:25.194706378 (175h59m12.998170983s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194911052 (175h59m12.998375653s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: context canceled; context canceled Sep 30 21:29:25.194921667 (175h59m12.998386269s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194946997 (175h59m12.998411575s) [seg_pos={0 0}, upload=154939]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 30 21:29:25.195081879 (175h59m12.998546481s) [upload=154939]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 30 21:29:25.195089364 (175h59m12.998553965s) [upload=154939]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 30 21:29:25.348637798 (175h59m13.1521024s) [upload=154939]: Done uploading segments... Sep 30 21:29:25.348680254 (175h59m13.152144854s) [upload=154939]: Upload finished. err: <nil> Duration: 2.547065573s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 2.61255624s FAIL (2025-09-30 22:03:37 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 30 21:29:22.797897319 (175h59m10.601361902s) [upload=154939]: Starting upload Sep 30 21:29:22.798006215 (175h59m10.601470808s) [upload=154939]: Done starting upload Sep 30 21:29:22.798081644 (175h59m10.601546229s) [upload=154939]: Uploading segments... Sep 30 21:29:22.798290989 (175h59m10.60175557s) [upload=154939]: Got next segment. Inline: false Sep 30 21:29:22.798331474 (175h59m10.601796067s) [upload=154939]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 30 21:29:22.798339902 (175h59m10.601804489s) [upload=154939]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 21:29:22.915769149 (175h59m10.719233752s) [seg_pos={0 0}, upload=154939]: Begin upload segment... Sep 30 21:29:22.915871743 (175h59m10.719336336s) [seg_pos={0 0}, upload=154939]: Done begin upload segment. Sep 30 21:29:22.915919849 (175h59m10.719384451s) [noise=true , address=45.91.116.4:4940 , piece_id=7DW6Y2G3IUR74PJ2ENFV6XLGTBZTISG4BQFI53KSUJ7HUMYIOSTA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.917360095 (175h59m10.720824697s) [noise=true , address=145.79.227.3:4151 , piece_id=LOVPJ4STVGHTGVOBIS5IMXQDUT7QNS55EFEDPZKD65WLBKTEO3OA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.917523118 (175h59m10.720987717s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918065655 (175h59m10.721530259s) [noise=true , address=145.79.227.13:4801 , piece_id=2W5CLWLLIUYIFCFUW7BBO5MGURJ6YBUNAQDVJCJ2F2LSVX3SLI5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918188240 (175h59m10.721652839s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918605397 (175h59m10.722069997s) [noise=true , address=45.91.116.5:4298 , piece_id=DUBHMFQQHF23TXRYMRODBVKGVCLAE6TG7IPEKTBHUE72WEX5O2SA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.918717812 (175h59m10.722182405s) [noise=true , address=145.79.227.16:4910 , piece_id=TBS644O7MTLBU7635SRB2N46WHQK6X5KYRYVITTZZAUFPEKC7GDQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919113523 (175h59m10.722578125s) [noise=true , address=145.79.227.7:4295 , piece_id=VLXVHJOSXZ627O6VCWPEL3H3QPLLSK7OZWGRCSRRT3MIPJDB5YBQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919331578 (175h59m10.722796181s) [noise=true , address=145.79.227.18:4005 , piece_id=IYY76ITHCMY4Z22VRD7WRX3Y6BDVLFKOQ2PCQRU7WFBE6VDQKV5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.919626618 (175h59m10.72309122s) [noise=true , address=45.91.116.10:4082 , piece_id=UWRIMN2P47HUKBYHCATAARGFTJXFGQZ7KRV5MUMIQ4GQRFQO7D6Q, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920025044 (175h59m10.723489653s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920253240 (175h59m10.723717844s) [noise=true , address=145.79.227.13:4802 , piece_id=Y5SXFQYQF32UFPHKIMIMI3V7WYDHGSZCPONZTZS5EVVKYSPSQONA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.920729685 (175h59m10.724194286s) [noise=true , address=145.79.227.11:4731 , piece_id=EDPUPZOFFO7TQWD7MI5MYMZSDONWB22VGCMZBZPMZA5MB3LOWXAA, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.921315566 (175h59m10.724780168s) [noise=true , address=145.79.227.6:4257 , piece_id=5YAEQS2PX6L4EXAKWT33WBMWXFRFDKTFKV7CBCPLO4BYCZKKA53A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.923000023 (175h59m10.726464628s) [noise=true , address=145.79.227.2:4114 , piece_id=V2U5JQ6Y5XJOQSXE6EKKQOA7F3ENKEKWTFOQ65LZV6BBRSK6YD5A, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:22.923798468 (175h59m10.727263073s) [noise=true , address=45.91.116.10:4081 , piece_id=WJKNHHM37DDABHFA6LPCF35FUW4SEMNMINZTPXDMJULW7TAL22OQ, seg_pos={0 0}, upload=154939]: Uploading piece... Sep 30 21:29:23.539151225 (175h59m11.342615808s) [upload=154939]: Next returned nil segment Sep 30 21:29:23.539169161 (175h59m11.34263376s) [upload=154939]: Waiting for error group managing segments... Sep 30 21:29:23.685441552 (175h59m11.488906147s) [noise=true , address=45.91.116.4:4940 , piece_id=7DW6Y2G3IUR74PJ2ENFV6XLGTBZTISG4BQFI53KSUJ7HUMYIOSTA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.688251132 (175h59m11.491715718s) [noise=true , address=145.79.227.7:4295 , piece_id=VLXVHJOSXZ627O6VCWPEL3H3QPLLSK7OZWGRCSRRT3MIPJDB5YBQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.691906915 (175h59m11.495371495s) [noise=true , address=145.79.227.16:4910 , piece_id=TBS644O7MTLBU7635SRB2N46WHQK6X5KYRYVITTZZAUFPEKC7GDQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.698112841 (175h59m11.501577422s) [noise=true , address=145.79.227.6:4257 , piece_id=5YAEQS2PX6L4EXAKWT33WBMWXFRFDKTFKV7CBCPLO4BYCZKKA53A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.698258894 (175h59m11.501723478s) [noise=true , address=145.79.227.2:4114 , piece_id=V2U5JQ6Y5XJOQSXE6EKKQOA7F3ENKEKWTFOQ65LZV6BBRSK6YD5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.702276005 (175h59m11.505740586s) [noise=true , address=45.91.116.10:4081 , piece_id=WJKNHHM37DDABHFA6LPCF35FUW4SEMNMINZTPXDMJULW7TAL22OQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.707342122 (175h59m11.51080671s) [noise=true , address=145.79.227.13:4802 , piece_id=Y5SXFQYQF32UFPHKIMIMI3V7WYDHGSZCPONZTZS5EVVKYSPSQONA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.711442145 (175h59m11.51490673s) [noise=true , address=145.79.227.11:4731 , piece_id=EDPUPZOFFO7TQWD7MI5MYMZSDONWB22VGCMZBZPMZA5MB3LOWXAA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.719319539 (175h59m11.522784123s) [noise=true , address=145.79.227.3:4151 , piece_id=LOVPJ4STVGHTGVOBIS5IMXQDUT7QNS55EFEDPZKD65WLBKTEO3OA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:23.988493881 (175h59m11.791958458s) [noise=true , address=45.91.116.5:4298 , piece_id=DUBHMFQQHF23TXRYMRODBVKGVCLAE6TG7IPEKTBHUE72WEX5O2SA, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:24.038956565 (175h59m11.842421143s) [noise=true , address=45.91.116.10:4082 , piece_id=UWRIMN2P47HUKBYHCATAARGFTJXFGQZ7KRV5MUMIQ4GQRFQO7D6Q, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:24.589378044 (175h59m12.392842628s) [noise=true , address=145.79.227.18:4005 , piece_id=IYY76ITHCMY4Z22VRD7WRX3Y6BDVLFKOQ2PCQRU7WFBE6VDQKV5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:25.194244661 (175h59m12.99770926s) [noise=true , address=145.79.227.13:4801 , piece_id=2W5CLWLLIUYIFCFUW7BBO5MGURJ6YBUNAQDVJCJ2F2LSVX3SLI5A, seg_pos={0 0}, upload=154939]: Done uploading piece. err: <nil> Sep 30 21:29:25.194299398 (175h59m12.997763991s) [seg_pos={0 0}, upload=154939]: Segment reached optimal threshold of 13 pieces. Sep 30 21:29:25.194653994 (175h59m12.99811859s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk): protocol: read tcp 139.59.144.22:34818->145.79.227.19:4077: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 30 21:29:25.194670346 (175h59m12.998134947s) [noise=true , address=145.79.227.19:4077 , piece_id=RXBHUITPAT44CYFM7EERUMQ6VZXJH5POUVI7Z2LFHQ3CR57UINDQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194691442 (175h59m12.99815604s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: read tcp 139.59.144.22:59584->77.90.31.17:4770: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 30 21:29:25.194706378 (175h59m12.998170983s) [noise=true , address=77.90.31.17:4770 , piece_id=HP4IPR5XEHIHHPYLVTXLDKBMS6KBQY7A6YWVTAV2BMHMSUYI4CAQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194911052 (175h59m12.998375653s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: context canceled; context canceled Sep 30 21:29:25.194921667 (175h59m12.998386269s) [noise=true , address=145.79.227.16:4912 , piece_id=3YQRDFPWCLI52GJIOYFBWSH2CWKIJW3I2C4S3Q27EJQHXHZT7BHQ, seg_pos={0 0}, upload=154939]: Long tail cancellation due to optimal threshold Sep 30 21:29:25.194946997 (175h59m12.998411575s) [seg_pos={0 0}, upload=154939]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 30 21:29:25.195081879 (175h59m12.998546481s) [upload=154939]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 30 21:29:25.195089364 (175h59m12.998553965s) [upload=154939]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 30 21:29:25.348637798 (175h59m13.1521024s) [upload=154939]: Done uploading segments... Sep 30 21:29:25.348680254 (175h59m13.152144854s) [upload=154939]: Upload finished. err: <nil> Duration: 2.612598936s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)