Orbiter History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 887.537452ms FAIL (2025-09-07 04:54:39 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 887.715833ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 188.414436ms FAIL (2025-09-07 04:53:49 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 188.448995ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 899.735209ms FAIL (2025-09-07 04:53:26 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 899.782665ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 9.83717597s FAIL (2025-09-07 04:53:09 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 9.837207756s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 1.073737954s FAIL (2025-09-07 04:52:45 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.073773154s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 850.48554ms FAIL (2025-09-07 04:52:25 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 850.541727ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 1.031724886s FAIL (2025-09-07 04:52:10 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.03178005s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 199.9693ms FAIL (2025-09-07 04:51:58 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 199.994747ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 791.485098ms FAIL (2025-09-07 04:51:33 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 791.597831ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 159.123251ms FAIL (2025-09-07 04:51:02 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 159.166152ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 1.128617863s FAIL (2025-09-07 04:50:55 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.128701248s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 149.639494ms FAIL (2025-09-07 04:50:46 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 149.697155ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 147.079218ms FAIL (2025-09-07 04:50:16 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 147.104237ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 929.914731ms FAIL (2025-09-07 04:49:48 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 929.962744ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 144.123317ms FAIL (2025-09-07 04:49:37 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 144.228415ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 1.006009683s FAIL (2025-09-07 04:49:09 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.00604829s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 141.818432ms FAIL (2025-09-07 04:48:51 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 141.845076ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 1.063581516s FAIL (2025-09-07 04:48:34 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.063618712s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 162.940168ms FAIL (2025-09-07 04:47:48 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 162.975542ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 26.857774527s FAIL (2025-09-07 04:47:37 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 26.857874285s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 1.099142551s FAIL (2025-09-07 04:47:24 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.099172961s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 808.511588ms FAIL (2025-09-07 04:46:26 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 808.552085ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 167.287395ms FAIL (2025-09-07 04:46:01 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 167.336056ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 909.80808ms FAIL (2025-09-07 04:45:24 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 909.850783ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 30.000218961s FAIL (2025-09-07 04:45:21 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 30.000241732s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 134.38659ms FAIL (2025-09-07 04:44:36 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 134.423872ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 16.834864468s FAIL (2025-09-07 04:44:34 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 16.834897239s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 1.031424959s FAIL (2025-09-07 04:43:59 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.031557353s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 149.66003ms FAIL (2025-09-07 04:43:58 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 149.75969ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 20.301975923s FAIL (2025-09-07 04:43:38 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 20.302034704s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.004480426s FAIL (2025-09-07 04:43:22 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 30.00450252s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.001472243s FAIL (2025-09-07 04:42:38 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 30.001496377s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.451455425s FAIL (2025-09-07 01:41:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.452438079s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 2.47872695s FAIL (2025-09-07 01:39:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.478764083s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 2.556022586s FAIL (2025-09-07 01:39:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.556073103s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 2.366271451s FAIL (2025-09-07 01:38:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.366325078s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 2.545223452s FAIL (2025-09-07 01:37:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.545260436s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 2.4658288s FAIL (2025-09-07 01:37:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.465855961s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 2.493382942s FAIL (2025-09-07 01:36:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.49342643s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 2.364127135s FAIL (2025-09-07 01:35:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.364159949s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 2.344814169s FAIL (2025-09-07 01:34:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.344855479s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 2.542453334s FAIL (2025-09-07 01:33:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.542496365s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 2.423280579s FAIL (2025-09-07 01:33:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.423320791s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 2.370344743s FAIL (2025-09-07 01:32:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.370383808s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 2.365464901s FAIL (2025-09-07 01:32:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.365515759s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 2.420010406s FAIL (2025-09-07 01:31:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.420029881s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 2.477471745s FAIL (2025-09-07 01:30:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.47753258s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 2.299360297s FAIL (2025-09-07 01:29:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.299385551s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 2.325855788s FAIL (2025-09-07 01:29:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.32589003s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 2.427762574s FAIL (2025-09-07 01:28:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.427794298s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)