Orbiter History

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 1.535753905s trace: https://jaeger.dev.storj.io/trace/4627cf09dd7a1c19 FAIL (2025-12-22 22:12:12 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 1.535790267s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 1.500955507s trace: https://jaeger.dev.storj.io/trace/4627cf09dd7a1c19 FAIL (2025-12-22 22:11:03 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 1.500983241s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 1.502813467s trace: https://jaeger.dev.storj.io/trace/4627cf09dd7a1c19 FAIL (2025-12-22 22:10:00 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 1.502859063s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 1.48095061s trace: https://jaeger.dev.storj.io/trace/4627cf09dd7a1c19 FAIL (2025-12-22 22:08:47 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 1.480984048s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 1.487355819s trace: https://jaeger.dev.storj.io/trace/4627cf09dd7a1c19 FAIL (2025-12-22 22:08:20 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 1.487384769s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 1.503564945s trace: https://jaeger.dev.storj.io/trace/4627cf09dd7a1c19 FAIL (2025-12-22 22:07:39 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 1.503599199s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 1.486215676s trace: https://jaeger.dev.storj.io/trace/4627cf09dd7a1c19 FAIL (2025-12-22 22:06:56 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 1.486254637s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 1.495295929s trace: https://jaeger.dev.storj.io/trace/4627cf09dd7a1c19 FAIL (2025-12-22 22:06:34 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 1.495351556s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 1.497847548s trace: https://jaeger.dev.storj.io/trace/4627cf09dd7a1c19 FAIL (2025-12-22 22:05:31 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 1.497874494s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 1.567651178s trace: https://jaeger.dev.storj.io/trace/4627cf09dd7a1c19 FAIL (2025-12-22 22:05:00 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 1.567699721s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 1.541507846s trace: https://jaeger.dev.storj.io/trace/4627cf09dd7a1c19 FAIL (2025-12-22 22:04:31 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 1.541617534s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 1.503946624s trace: https://jaeger.dev.storj.io/trace/4627cf09dd7a1c19 FAIL (2025-12-22 22:03:30 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 1.503996983s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 1.396221302s trace: https://jaeger.dev.storj.io/trace/4627cf09dd7a1c19 FAIL (2025-12-22 22:02:29 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 1.396252017s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 1.455616945s trace: https://jaeger.dev.storj.io/trace/4627cf09dd7a1c19 FAIL (2025-12-22 22:01:42 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 1.455671298s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 1.413016533s trace: https://jaeger.dev.storj.io/trace/4627cf09dd7a1c19 FAIL (2025-12-22 22:01:08 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 1.413059127s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 1.396138032s trace: https://jaeger.dev.storj.io/trace/4627cf09dd7a1c19 FAIL (2025-12-22 22:00:52 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 1.396175503s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 1.439306632s trace: https://jaeger.dev.storj.io/trace/4627cf09dd7a1c19 FAIL (2025-12-22 22:00:28 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 1.439340745s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 1.3798586s trace: https://jaeger.dev.storj.io/trace/4627cf09dd7a1c19 FAIL (2025-12-22 21:59:54 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 1.379890955s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 1.40969532s trace: https://jaeger.dev.storj.io/trace/4627cf09dd7a1c19 FAIL (2025-12-22 21:59:11 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 1.409721668s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 1.569567441s trace: https://jaeger.dev.storj.io/trace/4627cf09dd7a1c19 FAIL (2025-12-22 21:58:07 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 1.569598144s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 1.508154084s trace: https://jaeger.dev.storj.io/trace/4627cf09dd7a1c19 FAIL (2025-12-22 21:57:23 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 1.508179338s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 1.475071854s trace: https://jaeger.dev.storj.io/trace/4627cf09dd7a1c19 FAIL (2025-12-22 21:57:00 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 1.475101608s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 1.501900809s trace: https://jaeger.dev.storj.io/trace/4627cf09dd7a1c19 FAIL (2025-12-22 21:56:39 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 1.50192663s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 1.483237144s trace: https://jaeger.dev.storj.io/trace/4627cf09dd7a1c19 FAIL (2025-12-22 21:55:45 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 1.483269372s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 1.51056238s trace: https://jaeger.dev.storj.io/trace/4627cf09dd7a1c19 FAIL (2025-12-22 21:55:12 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 1.510595764s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 1.502730718s trace: https://jaeger.dev.storj.io/trace/4627cf09dd7a1c19 FAIL (2025-12-22 21:54:26 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 1.50276308s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 1.510905355s trace: https://jaeger.dev.storj.io/trace/4627cf09dd7a1c19 FAIL (2025-12-22 21:53:59 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 1.510939093s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.001520361s trace: https://jaeger.dev.storj.io/trace/4627cf09dd7a1c19 FAIL (2025-12-22 21:53:30 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 30.001541584s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.535105318s trace: https://jaeger.dev.storj.io/trace/5403b15b90934969 FAIL (2025-12-22 07:32:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 22 06:47:13.390879689 (399h18m53.371232325s) [upload=351224]: Starting upload Dec 22 06:47:13.390933946 (399h18m53.371286588s) [upload=351224]: Done starting upload Dec 22 06:47:13.391022385 (399h18m53.371375007s) [upload=351224]: Uploading segments... Dec 22 06:47:13.391257160 (399h18m53.371609787s) [upload=351224]: Got next segment. Inline: false Dec 22 06:47:13.391304735 (399h18m53.371657378s) [upload=351224]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 06:47:13.391309842 (399h18m53.371662478s) [upload=351224]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 06:47:13.503578512 (399h18m53.48393115s) [seg_pos={0 0}, upload=351224]: Begin upload segment... Dec 22 06:47:13.503670819 (399h18m53.484023465s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504640710 (399h18m53.484993414s) [seg_pos={0 0}, upload=351224]: Done begin upload segment. Dec 22 06:47:13.504785964 (399h18m53.485138878s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504916823 (399h18m53.485269465s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505227903 (399h18m53.485580539s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505250192 (399h18m53.485602828s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505519011 (399h18m53.485871647s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505579890 (399h18m53.485932526s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505839462 (399h18m53.486192109s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506000955 (399h18m53.486353591s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506194887 (399h18m53.486547527s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506408314 (399h18m53.486760957s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506480917 (399h18m53.486833557s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506749655 (399h18m53.48710229s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506759456 (399h18m53.487112092s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.507955888 (399h18m53.488308525s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.509580758 (399h18m53.489933403s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:14.116474867 (399h18m54.096827503s) [upload=351224]: Next returned nil segment Dec 22 06:47:14.116487753 (399h18m54.096840382s) [upload=351224]: Waiting for error group managing segments... Dec 22 06:47:14.252424715 (399h18m54.232777352s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.259199346 (399h18m54.239551983s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.263033341 (399h18m54.243385978s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.268392055 (399h18m54.2487447s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272217620 (399h18m54.252570256s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272621367 (399h18m54.252974009s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.274087214 (399h18m54.254439851s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.281307529 (399h18m54.261660168s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.290977855 (399h18m54.2713305s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.305725196 (399h18m54.28607784s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:21.393190149 (399h19m1.373542789s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:22.349672394 (399h19m2.33002503s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:43.106830367 (399h19m23.087183004s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:17o881pUMuG7K2dv83k7RkrJCrmM7kgUiWesgam4GEw21KVUn7, address:145.79.227.29:4942): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107030968 (399h19m23.087383607s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107221616 (399h19m23.087574252s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107779626 (399h19m23.088132267s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107809812 (399h19m23.088162439s) [seg_pos={0 0}, upload=351224]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 22 06:47:43.213405603 (399h19m23.193758247s) [upload=351224]: Done uploading segments... Dec 22 06:47:43.213448935 (399h19m23.193801581s) [upload=351224]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.535227076s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 2.391934758s trace: https://jaeger.dev.storj.io/trace/5403b15b90934969 FAIL (2025-12-22 07:31:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 22 06:47:13.390879689 (399h18m53.371232325s) [upload=351224]: Starting upload Dec 22 06:47:13.390933946 (399h18m53.371286588s) [upload=351224]: Done starting upload Dec 22 06:47:13.391022385 (399h18m53.371375007s) [upload=351224]: Uploading segments... Dec 22 06:47:13.391257160 (399h18m53.371609787s) [upload=351224]: Got next segment. Inline: false Dec 22 06:47:13.391304735 (399h18m53.371657378s) [upload=351224]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 06:47:13.391309842 (399h18m53.371662478s) [upload=351224]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 06:47:13.503578512 (399h18m53.48393115s) [seg_pos={0 0}, upload=351224]: Begin upload segment... Dec 22 06:47:13.503670819 (399h18m53.484023465s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504640710 (399h18m53.484993414s) [seg_pos={0 0}, upload=351224]: Done begin upload segment. Dec 22 06:47:13.504785964 (399h18m53.485138878s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504916823 (399h18m53.485269465s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505227903 (399h18m53.485580539s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505250192 (399h18m53.485602828s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505519011 (399h18m53.485871647s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505579890 (399h18m53.485932526s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505839462 (399h18m53.486192109s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506000955 (399h18m53.486353591s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506194887 (399h18m53.486547527s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506408314 (399h18m53.486760957s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506480917 (399h18m53.486833557s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506749655 (399h18m53.48710229s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506759456 (399h18m53.487112092s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.507955888 (399h18m53.488308525s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.509580758 (399h18m53.489933403s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:14.116474867 (399h18m54.096827503s) [upload=351224]: Next returned nil segment Dec 22 06:47:14.116487753 (399h18m54.096840382s) [upload=351224]: Waiting for error group managing segments... Dec 22 06:47:14.252424715 (399h18m54.232777352s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.259199346 (399h18m54.239551983s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.263033341 (399h18m54.243385978s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.268392055 (399h18m54.2487447s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272217620 (399h18m54.252570256s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272621367 (399h18m54.252974009s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.274087214 (399h18m54.254439851s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.281307529 (399h18m54.261660168s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.290977855 (399h18m54.2713305s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.305725196 (399h18m54.28607784s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:21.393190149 (399h19m1.373542789s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:22.349672394 (399h19m2.33002503s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:43.106830367 (399h19m23.087183004s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:17o881pUMuG7K2dv83k7RkrJCrmM7kgUiWesgam4GEw21KVUn7, address:145.79.227.29:4942): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107030968 (399h19m23.087383607s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107221616 (399h19m23.087574252s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107779626 (399h19m23.088132267s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107809812 (399h19m23.088162439s) [seg_pos={0 0}, upload=351224]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 22 06:47:43.213405603 (399h19m23.193758247s) [upload=351224]: Done uploading segments... Dec 22 06:47:43.213448935 (399h19m23.193801581s) [upload=351224]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.391961281s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 2.359217826s trace: https://jaeger.dev.storj.io/trace/5403b15b90934969 FAIL (2025-12-22 07:31:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 22 06:47:13.390879689 (399h18m53.371232325s) [upload=351224]: Starting upload Dec 22 06:47:13.390933946 (399h18m53.371286588s) [upload=351224]: Done starting upload Dec 22 06:47:13.391022385 (399h18m53.371375007s) [upload=351224]: Uploading segments... Dec 22 06:47:13.391257160 (399h18m53.371609787s) [upload=351224]: Got next segment. Inline: false Dec 22 06:47:13.391304735 (399h18m53.371657378s) [upload=351224]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 06:47:13.391309842 (399h18m53.371662478s) [upload=351224]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 06:47:13.503578512 (399h18m53.48393115s) [seg_pos={0 0}, upload=351224]: Begin upload segment... Dec 22 06:47:13.503670819 (399h18m53.484023465s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504640710 (399h18m53.484993414s) [seg_pos={0 0}, upload=351224]: Done begin upload segment. Dec 22 06:47:13.504785964 (399h18m53.485138878s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504916823 (399h18m53.485269465s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505227903 (399h18m53.485580539s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505250192 (399h18m53.485602828s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505519011 (399h18m53.485871647s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505579890 (399h18m53.485932526s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505839462 (399h18m53.486192109s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506000955 (399h18m53.486353591s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506194887 (399h18m53.486547527s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506408314 (399h18m53.486760957s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506480917 (399h18m53.486833557s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506749655 (399h18m53.48710229s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506759456 (399h18m53.487112092s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.507955888 (399h18m53.488308525s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.509580758 (399h18m53.489933403s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:14.116474867 (399h18m54.096827503s) [upload=351224]: Next returned nil segment Dec 22 06:47:14.116487753 (399h18m54.096840382s) [upload=351224]: Waiting for error group managing segments... Dec 22 06:47:14.252424715 (399h18m54.232777352s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.259199346 (399h18m54.239551983s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.263033341 (399h18m54.243385978s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.268392055 (399h18m54.2487447s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272217620 (399h18m54.252570256s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272621367 (399h18m54.252974009s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.274087214 (399h18m54.254439851s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.281307529 (399h18m54.261660168s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.290977855 (399h18m54.2713305s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.305725196 (399h18m54.28607784s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:21.393190149 (399h19m1.373542789s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:22.349672394 (399h19m2.33002503s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:43.106830367 (399h19m23.087183004s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:17o881pUMuG7K2dv83k7RkrJCrmM7kgUiWesgam4GEw21KVUn7, address:145.79.227.29:4942): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107030968 (399h19m23.087383607s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107221616 (399h19m23.087574252s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107779626 (399h19m23.088132267s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107809812 (399h19m23.088162439s) [seg_pos={0 0}, upload=351224]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 22 06:47:43.213405603 (399h19m23.193758247s) [upload=351224]: Done uploading segments... Dec 22 06:47:43.213448935 (399h19m23.193801581s) [upload=351224]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.359240282s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 2.364850737s trace: https://jaeger.dev.storj.io/trace/5403b15b90934969 FAIL (2025-12-22 07:30:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 22 06:47:13.390879689 (399h18m53.371232325s) [upload=351224]: Starting upload Dec 22 06:47:13.390933946 (399h18m53.371286588s) [upload=351224]: Done starting upload Dec 22 06:47:13.391022385 (399h18m53.371375007s) [upload=351224]: Uploading segments... Dec 22 06:47:13.391257160 (399h18m53.371609787s) [upload=351224]: Got next segment. Inline: false Dec 22 06:47:13.391304735 (399h18m53.371657378s) [upload=351224]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 06:47:13.391309842 (399h18m53.371662478s) [upload=351224]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 06:47:13.503578512 (399h18m53.48393115s) [seg_pos={0 0}, upload=351224]: Begin upload segment... Dec 22 06:47:13.503670819 (399h18m53.484023465s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504640710 (399h18m53.484993414s) [seg_pos={0 0}, upload=351224]: Done begin upload segment. Dec 22 06:47:13.504785964 (399h18m53.485138878s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504916823 (399h18m53.485269465s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505227903 (399h18m53.485580539s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505250192 (399h18m53.485602828s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505519011 (399h18m53.485871647s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505579890 (399h18m53.485932526s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505839462 (399h18m53.486192109s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506000955 (399h18m53.486353591s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506194887 (399h18m53.486547527s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506408314 (399h18m53.486760957s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506480917 (399h18m53.486833557s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506749655 (399h18m53.48710229s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506759456 (399h18m53.487112092s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.507955888 (399h18m53.488308525s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.509580758 (399h18m53.489933403s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:14.116474867 (399h18m54.096827503s) [upload=351224]: Next returned nil segment Dec 22 06:47:14.116487753 (399h18m54.096840382s) [upload=351224]: Waiting for error group managing segments... Dec 22 06:47:14.252424715 (399h18m54.232777352s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.259199346 (399h18m54.239551983s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.263033341 (399h18m54.243385978s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.268392055 (399h18m54.2487447s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272217620 (399h18m54.252570256s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272621367 (399h18m54.252974009s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.274087214 (399h18m54.254439851s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.281307529 (399h18m54.261660168s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.290977855 (399h18m54.2713305s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.305725196 (399h18m54.28607784s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:21.393190149 (399h19m1.373542789s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:22.349672394 (399h19m2.33002503s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:43.106830367 (399h19m23.087183004s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:17o881pUMuG7K2dv83k7RkrJCrmM7kgUiWesgam4GEw21KVUn7, address:145.79.227.29:4942): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107030968 (399h19m23.087383607s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107221616 (399h19m23.087574252s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107779626 (399h19m23.088132267s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107809812 (399h19m23.088162439s) [seg_pos={0 0}, upload=351224]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 22 06:47:43.213405603 (399h19m23.193758247s) [upload=351224]: Done uploading segments... Dec 22 06:47:43.213448935 (399h19m23.193801581s) [upload=351224]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.364890547s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 2.343558698s trace: https://jaeger.dev.storj.io/trace/5403b15b90934969 FAIL (2025-12-22 07:29:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 22 06:47:13.390879689 (399h18m53.371232325s) [upload=351224]: Starting upload Dec 22 06:47:13.390933946 (399h18m53.371286588s) [upload=351224]: Done starting upload Dec 22 06:47:13.391022385 (399h18m53.371375007s) [upload=351224]: Uploading segments... Dec 22 06:47:13.391257160 (399h18m53.371609787s) [upload=351224]: Got next segment. Inline: false Dec 22 06:47:13.391304735 (399h18m53.371657378s) [upload=351224]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 06:47:13.391309842 (399h18m53.371662478s) [upload=351224]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 06:47:13.503578512 (399h18m53.48393115s) [seg_pos={0 0}, upload=351224]: Begin upload segment... Dec 22 06:47:13.503670819 (399h18m53.484023465s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504640710 (399h18m53.484993414s) [seg_pos={0 0}, upload=351224]: Done begin upload segment. Dec 22 06:47:13.504785964 (399h18m53.485138878s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504916823 (399h18m53.485269465s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505227903 (399h18m53.485580539s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505250192 (399h18m53.485602828s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505519011 (399h18m53.485871647s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505579890 (399h18m53.485932526s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505839462 (399h18m53.486192109s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506000955 (399h18m53.486353591s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506194887 (399h18m53.486547527s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506408314 (399h18m53.486760957s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506480917 (399h18m53.486833557s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506749655 (399h18m53.48710229s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506759456 (399h18m53.487112092s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.507955888 (399h18m53.488308525s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.509580758 (399h18m53.489933403s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:14.116474867 (399h18m54.096827503s) [upload=351224]: Next returned nil segment Dec 22 06:47:14.116487753 (399h18m54.096840382s) [upload=351224]: Waiting for error group managing segments... Dec 22 06:47:14.252424715 (399h18m54.232777352s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.259199346 (399h18m54.239551983s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.263033341 (399h18m54.243385978s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.268392055 (399h18m54.2487447s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272217620 (399h18m54.252570256s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272621367 (399h18m54.252974009s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.274087214 (399h18m54.254439851s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.281307529 (399h18m54.261660168s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.290977855 (399h18m54.2713305s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.305725196 (399h18m54.28607784s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:21.393190149 (399h19m1.373542789s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:22.349672394 (399h19m2.33002503s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:43.106830367 (399h19m23.087183004s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:17o881pUMuG7K2dv83k7RkrJCrmM7kgUiWesgam4GEw21KVUn7, address:145.79.227.29:4942): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107030968 (399h19m23.087383607s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107221616 (399h19m23.087574252s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107779626 (399h19m23.088132267s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107809812 (399h19m23.088162439s) [seg_pos={0 0}, upload=351224]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 22 06:47:43.213405603 (399h19m23.193758247s) [upload=351224]: Done uploading segments... Dec 22 06:47:43.213448935 (399h19m23.193801581s) [upload=351224]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.343587151s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 2.315148743s trace: https://jaeger.dev.storj.io/trace/5403b15b90934969 FAIL (2025-12-22 07:28:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 22 06:47:13.390879689 (399h18m53.371232325s) [upload=351224]: Starting upload Dec 22 06:47:13.390933946 (399h18m53.371286588s) [upload=351224]: Done starting upload Dec 22 06:47:13.391022385 (399h18m53.371375007s) [upload=351224]: Uploading segments... Dec 22 06:47:13.391257160 (399h18m53.371609787s) [upload=351224]: Got next segment. Inline: false Dec 22 06:47:13.391304735 (399h18m53.371657378s) [upload=351224]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 06:47:13.391309842 (399h18m53.371662478s) [upload=351224]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 06:47:13.503578512 (399h18m53.48393115s) [seg_pos={0 0}, upload=351224]: Begin upload segment... Dec 22 06:47:13.503670819 (399h18m53.484023465s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504640710 (399h18m53.484993414s) [seg_pos={0 0}, upload=351224]: Done begin upload segment. Dec 22 06:47:13.504785964 (399h18m53.485138878s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504916823 (399h18m53.485269465s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505227903 (399h18m53.485580539s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505250192 (399h18m53.485602828s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505519011 (399h18m53.485871647s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505579890 (399h18m53.485932526s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505839462 (399h18m53.486192109s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506000955 (399h18m53.486353591s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506194887 (399h18m53.486547527s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506408314 (399h18m53.486760957s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506480917 (399h18m53.486833557s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506749655 (399h18m53.48710229s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506759456 (399h18m53.487112092s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.507955888 (399h18m53.488308525s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.509580758 (399h18m53.489933403s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:14.116474867 (399h18m54.096827503s) [upload=351224]: Next returned nil segment Dec 22 06:47:14.116487753 (399h18m54.096840382s) [upload=351224]: Waiting for error group managing segments... Dec 22 06:47:14.252424715 (399h18m54.232777352s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.259199346 (399h18m54.239551983s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.263033341 (399h18m54.243385978s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.268392055 (399h18m54.2487447s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272217620 (399h18m54.252570256s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272621367 (399h18m54.252974009s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.274087214 (399h18m54.254439851s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.281307529 (399h18m54.261660168s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.290977855 (399h18m54.2713305s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.305725196 (399h18m54.28607784s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:21.393190149 (399h19m1.373542789s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:22.349672394 (399h19m2.33002503s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:43.106830367 (399h19m23.087183004s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:17o881pUMuG7K2dv83k7RkrJCrmM7kgUiWesgam4GEw21KVUn7, address:145.79.227.29:4942): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107030968 (399h19m23.087383607s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107221616 (399h19m23.087574252s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107779626 (399h19m23.088132267s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107809812 (399h19m23.088162439s) [seg_pos={0 0}, upload=351224]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 22 06:47:43.213405603 (399h19m23.193758247s) [upload=351224]: Done uploading segments... Dec 22 06:47:43.213448935 (399h19m23.193801581s) [upload=351224]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.315173915s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 2.308859808s trace: https://jaeger.dev.storj.io/trace/5403b15b90934969 FAIL (2025-12-22 07:27:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 22 06:47:13.390879689 (399h18m53.371232325s) [upload=351224]: Starting upload Dec 22 06:47:13.390933946 (399h18m53.371286588s) [upload=351224]: Done starting upload Dec 22 06:47:13.391022385 (399h18m53.371375007s) [upload=351224]: Uploading segments... Dec 22 06:47:13.391257160 (399h18m53.371609787s) [upload=351224]: Got next segment. Inline: false Dec 22 06:47:13.391304735 (399h18m53.371657378s) [upload=351224]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 06:47:13.391309842 (399h18m53.371662478s) [upload=351224]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 06:47:13.503578512 (399h18m53.48393115s) [seg_pos={0 0}, upload=351224]: Begin upload segment... Dec 22 06:47:13.503670819 (399h18m53.484023465s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504640710 (399h18m53.484993414s) [seg_pos={0 0}, upload=351224]: Done begin upload segment. Dec 22 06:47:13.504785964 (399h18m53.485138878s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504916823 (399h18m53.485269465s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505227903 (399h18m53.485580539s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505250192 (399h18m53.485602828s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505519011 (399h18m53.485871647s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505579890 (399h18m53.485932526s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505839462 (399h18m53.486192109s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506000955 (399h18m53.486353591s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506194887 (399h18m53.486547527s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506408314 (399h18m53.486760957s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506480917 (399h18m53.486833557s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506749655 (399h18m53.48710229s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506759456 (399h18m53.487112092s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.507955888 (399h18m53.488308525s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.509580758 (399h18m53.489933403s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:14.116474867 (399h18m54.096827503s) [upload=351224]: Next returned nil segment Dec 22 06:47:14.116487753 (399h18m54.096840382s) [upload=351224]: Waiting for error group managing segments... Dec 22 06:47:14.252424715 (399h18m54.232777352s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.259199346 (399h18m54.239551983s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.263033341 (399h18m54.243385978s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.268392055 (399h18m54.2487447s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272217620 (399h18m54.252570256s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272621367 (399h18m54.252974009s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.274087214 (399h18m54.254439851s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.281307529 (399h18m54.261660168s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.290977855 (399h18m54.2713305s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.305725196 (399h18m54.28607784s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:21.393190149 (399h19m1.373542789s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:22.349672394 (399h19m2.33002503s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:43.106830367 (399h19m23.087183004s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:17o881pUMuG7K2dv83k7RkrJCrmM7kgUiWesgam4GEw21KVUn7, address:145.79.227.29:4942): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107030968 (399h19m23.087383607s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107221616 (399h19m23.087574252s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107779626 (399h19m23.088132267s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107809812 (399h19m23.088162439s) [seg_pos={0 0}, upload=351224]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 22 06:47:43.213405603 (399h19m23.193758247s) [upload=351224]: Done uploading segments... Dec 22 06:47:43.213448935 (399h19m23.193801581s) [upload=351224]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.308886506s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 2.378997516s trace: https://jaeger.dev.storj.io/trace/5403b15b90934969 FAIL (2025-12-22 07:26:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 22 06:47:13.390879689 (399h18m53.371232325s) [upload=351224]: Starting upload Dec 22 06:47:13.390933946 (399h18m53.371286588s) [upload=351224]: Done starting upload Dec 22 06:47:13.391022385 (399h18m53.371375007s) [upload=351224]: Uploading segments... Dec 22 06:47:13.391257160 (399h18m53.371609787s) [upload=351224]: Got next segment. Inline: false Dec 22 06:47:13.391304735 (399h18m53.371657378s) [upload=351224]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 06:47:13.391309842 (399h18m53.371662478s) [upload=351224]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 06:47:13.503578512 (399h18m53.48393115s) [seg_pos={0 0}, upload=351224]: Begin upload segment... Dec 22 06:47:13.503670819 (399h18m53.484023465s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504640710 (399h18m53.484993414s) [seg_pos={0 0}, upload=351224]: Done begin upload segment. Dec 22 06:47:13.504785964 (399h18m53.485138878s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504916823 (399h18m53.485269465s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505227903 (399h18m53.485580539s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505250192 (399h18m53.485602828s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505519011 (399h18m53.485871647s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505579890 (399h18m53.485932526s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505839462 (399h18m53.486192109s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506000955 (399h18m53.486353591s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506194887 (399h18m53.486547527s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506408314 (399h18m53.486760957s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506480917 (399h18m53.486833557s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506749655 (399h18m53.48710229s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506759456 (399h18m53.487112092s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.507955888 (399h18m53.488308525s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.509580758 (399h18m53.489933403s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:14.116474867 (399h18m54.096827503s) [upload=351224]: Next returned nil segment Dec 22 06:47:14.116487753 (399h18m54.096840382s) [upload=351224]: Waiting for error group managing segments... Dec 22 06:47:14.252424715 (399h18m54.232777352s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.259199346 (399h18m54.239551983s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.263033341 (399h18m54.243385978s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.268392055 (399h18m54.2487447s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272217620 (399h18m54.252570256s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272621367 (399h18m54.252974009s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.274087214 (399h18m54.254439851s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.281307529 (399h18m54.261660168s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.290977855 (399h18m54.2713305s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.305725196 (399h18m54.28607784s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:21.393190149 (399h19m1.373542789s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:22.349672394 (399h19m2.33002503s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:43.106830367 (399h19m23.087183004s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:17o881pUMuG7K2dv83k7RkrJCrmM7kgUiWesgam4GEw21KVUn7, address:145.79.227.29:4942): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107030968 (399h19m23.087383607s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107221616 (399h19m23.087574252s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107779626 (399h19m23.088132267s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107809812 (399h19m23.088162439s) [seg_pos={0 0}, upload=351224]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 22 06:47:43.213405603 (399h19m23.193758247s) [upload=351224]: Done uploading segments... Dec 22 06:47:43.213448935 (399h19m23.193801581s) [upload=351224]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.379026436s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 2.35046794s trace: https://jaeger.dev.storj.io/trace/5403b15b90934969 FAIL (2025-12-22 07:26:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 22 06:47:13.390879689 (399h18m53.371232325s) [upload=351224]: Starting upload Dec 22 06:47:13.390933946 (399h18m53.371286588s) [upload=351224]: Done starting upload Dec 22 06:47:13.391022385 (399h18m53.371375007s) [upload=351224]: Uploading segments... Dec 22 06:47:13.391257160 (399h18m53.371609787s) [upload=351224]: Got next segment. Inline: false Dec 22 06:47:13.391304735 (399h18m53.371657378s) [upload=351224]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 06:47:13.391309842 (399h18m53.371662478s) [upload=351224]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 06:47:13.503578512 (399h18m53.48393115s) [seg_pos={0 0}, upload=351224]: Begin upload segment... Dec 22 06:47:13.503670819 (399h18m53.484023465s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504640710 (399h18m53.484993414s) [seg_pos={0 0}, upload=351224]: Done begin upload segment. Dec 22 06:47:13.504785964 (399h18m53.485138878s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504916823 (399h18m53.485269465s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505227903 (399h18m53.485580539s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505250192 (399h18m53.485602828s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505519011 (399h18m53.485871647s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505579890 (399h18m53.485932526s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505839462 (399h18m53.486192109s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506000955 (399h18m53.486353591s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506194887 (399h18m53.486547527s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506408314 (399h18m53.486760957s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506480917 (399h18m53.486833557s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506749655 (399h18m53.48710229s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506759456 (399h18m53.487112092s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.507955888 (399h18m53.488308525s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.509580758 (399h18m53.489933403s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:14.116474867 (399h18m54.096827503s) [upload=351224]: Next returned nil segment Dec 22 06:47:14.116487753 (399h18m54.096840382s) [upload=351224]: Waiting for error group managing segments... Dec 22 06:47:14.252424715 (399h18m54.232777352s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.259199346 (399h18m54.239551983s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.263033341 (399h18m54.243385978s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.268392055 (399h18m54.2487447s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272217620 (399h18m54.252570256s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272621367 (399h18m54.252974009s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.274087214 (399h18m54.254439851s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.281307529 (399h18m54.261660168s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.290977855 (399h18m54.2713305s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.305725196 (399h18m54.28607784s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:21.393190149 (399h19m1.373542789s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:22.349672394 (399h19m2.33002503s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:43.106830367 (399h19m23.087183004s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:17o881pUMuG7K2dv83k7RkrJCrmM7kgUiWesgam4GEw21KVUn7, address:145.79.227.29:4942): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107030968 (399h19m23.087383607s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107221616 (399h19m23.087574252s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107779626 (399h19m23.088132267s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107809812 (399h19m23.088162439s) [seg_pos={0 0}, upload=351224]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 22 06:47:43.213405603 (399h19m23.193758247s) [upload=351224]: Done uploading segments... Dec 22 06:47:43.213448935 (399h19m23.193801581s) [upload=351224]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.350491803s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 2.383853887s trace: https://jaeger.dev.storj.io/trace/5403b15b90934969 FAIL (2025-12-22 07:25:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 22 06:47:13.390879689 (399h18m53.371232325s) [upload=351224]: Starting upload Dec 22 06:47:13.390933946 (399h18m53.371286588s) [upload=351224]: Done starting upload Dec 22 06:47:13.391022385 (399h18m53.371375007s) [upload=351224]: Uploading segments... Dec 22 06:47:13.391257160 (399h18m53.371609787s) [upload=351224]: Got next segment. Inline: false Dec 22 06:47:13.391304735 (399h18m53.371657378s) [upload=351224]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 06:47:13.391309842 (399h18m53.371662478s) [upload=351224]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 06:47:13.503578512 (399h18m53.48393115s) [seg_pos={0 0}, upload=351224]: Begin upload segment... Dec 22 06:47:13.503670819 (399h18m53.484023465s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504640710 (399h18m53.484993414s) [seg_pos={0 0}, upload=351224]: Done begin upload segment. Dec 22 06:47:13.504785964 (399h18m53.485138878s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504916823 (399h18m53.485269465s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505227903 (399h18m53.485580539s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505250192 (399h18m53.485602828s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505519011 (399h18m53.485871647s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505579890 (399h18m53.485932526s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505839462 (399h18m53.486192109s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506000955 (399h18m53.486353591s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506194887 (399h18m53.486547527s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506408314 (399h18m53.486760957s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506480917 (399h18m53.486833557s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506749655 (399h18m53.48710229s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506759456 (399h18m53.487112092s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.507955888 (399h18m53.488308525s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.509580758 (399h18m53.489933403s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:14.116474867 (399h18m54.096827503s) [upload=351224]: Next returned nil segment Dec 22 06:47:14.116487753 (399h18m54.096840382s) [upload=351224]: Waiting for error group managing segments... Dec 22 06:47:14.252424715 (399h18m54.232777352s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.259199346 (399h18m54.239551983s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.263033341 (399h18m54.243385978s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.268392055 (399h18m54.2487447s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272217620 (399h18m54.252570256s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272621367 (399h18m54.252974009s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.274087214 (399h18m54.254439851s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.281307529 (399h18m54.261660168s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.290977855 (399h18m54.2713305s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.305725196 (399h18m54.28607784s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:21.393190149 (399h19m1.373542789s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:22.349672394 (399h19m2.33002503s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:43.106830367 (399h19m23.087183004s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:17o881pUMuG7K2dv83k7RkrJCrmM7kgUiWesgam4GEw21KVUn7, address:145.79.227.29:4942): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107030968 (399h19m23.087383607s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107221616 (399h19m23.087574252s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107779626 (399h19m23.088132267s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107809812 (399h19m23.088162439s) [seg_pos={0 0}, upload=351224]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 22 06:47:43.213405603 (399h19m23.193758247s) [upload=351224]: Done uploading segments... Dec 22 06:47:43.213448935 (399h19m23.193801581s) [upload=351224]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.383882567s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 2.418858278s trace: https://jaeger.dev.storj.io/trace/5403b15b90934969 FAIL (2025-12-22 07:25:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 22 06:47:13.390879689 (399h18m53.371232325s) [upload=351224]: Starting upload Dec 22 06:47:13.390933946 (399h18m53.371286588s) [upload=351224]: Done starting upload Dec 22 06:47:13.391022385 (399h18m53.371375007s) [upload=351224]: Uploading segments... Dec 22 06:47:13.391257160 (399h18m53.371609787s) [upload=351224]: Got next segment. Inline: false Dec 22 06:47:13.391304735 (399h18m53.371657378s) [upload=351224]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 06:47:13.391309842 (399h18m53.371662478s) [upload=351224]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 06:47:13.503578512 (399h18m53.48393115s) [seg_pos={0 0}, upload=351224]: Begin upload segment... Dec 22 06:47:13.503670819 (399h18m53.484023465s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504640710 (399h18m53.484993414s) [seg_pos={0 0}, upload=351224]: Done begin upload segment. Dec 22 06:47:13.504785964 (399h18m53.485138878s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504916823 (399h18m53.485269465s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505227903 (399h18m53.485580539s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505250192 (399h18m53.485602828s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505519011 (399h18m53.485871647s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505579890 (399h18m53.485932526s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505839462 (399h18m53.486192109s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506000955 (399h18m53.486353591s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506194887 (399h18m53.486547527s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506408314 (399h18m53.486760957s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506480917 (399h18m53.486833557s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506749655 (399h18m53.48710229s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506759456 (399h18m53.487112092s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.507955888 (399h18m53.488308525s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.509580758 (399h18m53.489933403s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:14.116474867 (399h18m54.096827503s) [upload=351224]: Next returned nil segment Dec 22 06:47:14.116487753 (399h18m54.096840382s) [upload=351224]: Waiting for error group managing segments... Dec 22 06:47:14.252424715 (399h18m54.232777352s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.259199346 (399h18m54.239551983s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.263033341 (399h18m54.243385978s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.268392055 (399h18m54.2487447s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272217620 (399h18m54.252570256s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272621367 (399h18m54.252974009s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.274087214 (399h18m54.254439851s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.281307529 (399h18m54.261660168s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.290977855 (399h18m54.2713305s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.305725196 (399h18m54.28607784s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:21.393190149 (399h19m1.373542789s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:22.349672394 (399h19m2.33002503s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:43.106830367 (399h19m23.087183004s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:17o881pUMuG7K2dv83k7RkrJCrmM7kgUiWesgam4GEw21KVUn7, address:145.79.227.29:4942): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107030968 (399h19m23.087383607s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107221616 (399h19m23.087574252s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107779626 (399h19m23.088132267s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107809812 (399h19m23.088162439s) [seg_pos={0 0}, upload=351224]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 22 06:47:43.213405603 (399h19m23.193758247s) [upload=351224]: Done uploading segments... Dec 22 06:47:43.213448935 (399h19m23.193801581s) [upload=351224]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.41887848s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 2.345790421s trace: https://jaeger.dev.storj.io/trace/5403b15b90934969 FAIL (2025-12-22 07:24:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 22 06:47:13.390879689 (399h18m53.371232325s) [upload=351224]: Starting upload Dec 22 06:47:13.390933946 (399h18m53.371286588s) [upload=351224]: Done starting upload Dec 22 06:47:13.391022385 (399h18m53.371375007s) [upload=351224]: Uploading segments... Dec 22 06:47:13.391257160 (399h18m53.371609787s) [upload=351224]: Got next segment. Inline: false Dec 22 06:47:13.391304735 (399h18m53.371657378s) [upload=351224]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 06:47:13.391309842 (399h18m53.371662478s) [upload=351224]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 06:47:13.503578512 (399h18m53.48393115s) [seg_pos={0 0}, upload=351224]: Begin upload segment... Dec 22 06:47:13.503670819 (399h18m53.484023465s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504640710 (399h18m53.484993414s) [seg_pos={0 0}, upload=351224]: Done begin upload segment. Dec 22 06:47:13.504785964 (399h18m53.485138878s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504916823 (399h18m53.485269465s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505227903 (399h18m53.485580539s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505250192 (399h18m53.485602828s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505519011 (399h18m53.485871647s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505579890 (399h18m53.485932526s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505839462 (399h18m53.486192109s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506000955 (399h18m53.486353591s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506194887 (399h18m53.486547527s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506408314 (399h18m53.486760957s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506480917 (399h18m53.486833557s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506749655 (399h18m53.48710229s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506759456 (399h18m53.487112092s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.507955888 (399h18m53.488308525s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.509580758 (399h18m53.489933403s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:14.116474867 (399h18m54.096827503s) [upload=351224]: Next returned nil segment Dec 22 06:47:14.116487753 (399h18m54.096840382s) [upload=351224]: Waiting for error group managing segments... Dec 22 06:47:14.252424715 (399h18m54.232777352s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.259199346 (399h18m54.239551983s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.263033341 (399h18m54.243385978s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.268392055 (399h18m54.2487447s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272217620 (399h18m54.252570256s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272621367 (399h18m54.252974009s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.274087214 (399h18m54.254439851s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.281307529 (399h18m54.261660168s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.290977855 (399h18m54.2713305s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.305725196 (399h18m54.28607784s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:21.393190149 (399h19m1.373542789s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:22.349672394 (399h19m2.33002503s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:43.106830367 (399h19m23.087183004s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:17o881pUMuG7K2dv83k7RkrJCrmM7kgUiWesgam4GEw21KVUn7, address:145.79.227.29:4942): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107030968 (399h19m23.087383607s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107221616 (399h19m23.087574252s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107779626 (399h19m23.088132267s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107809812 (399h19m23.088162439s) [seg_pos={0 0}, upload=351224]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 22 06:47:43.213405603 (399h19m23.193758247s) [upload=351224]: Done uploading segments... Dec 22 06:47:43.213448935 (399h19m23.193801581s) [upload=351224]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.345816534s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 2.396032643s trace: https://jaeger.dev.storj.io/trace/5403b15b90934969 FAIL (2025-12-22 07:23:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 22 06:47:13.390879689 (399h18m53.371232325s) [upload=351224]: Starting upload Dec 22 06:47:13.390933946 (399h18m53.371286588s) [upload=351224]: Done starting upload Dec 22 06:47:13.391022385 (399h18m53.371375007s) [upload=351224]: Uploading segments... Dec 22 06:47:13.391257160 (399h18m53.371609787s) [upload=351224]: Got next segment. Inline: false Dec 22 06:47:13.391304735 (399h18m53.371657378s) [upload=351224]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 06:47:13.391309842 (399h18m53.371662478s) [upload=351224]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 06:47:13.503578512 (399h18m53.48393115s) [seg_pos={0 0}, upload=351224]: Begin upload segment... Dec 22 06:47:13.503670819 (399h18m53.484023465s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504640710 (399h18m53.484993414s) [seg_pos={0 0}, upload=351224]: Done begin upload segment. Dec 22 06:47:13.504785964 (399h18m53.485138878s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504916823 (399h18m53.485269465s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505227903 (399h18m53.485580539s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505250192 (399h18m53.485602828s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505519011 (399h18m53.485871647s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505579890 (399h18m53.485932526s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505839462 (399h18m53.486192109s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506000955 (399h18m53.486353591s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506194887 (399h18m53.486547527s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506408314 (399h18m53.486760957s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506480917 (399h18m53.486833557s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506749655 (399h18m53.48710229s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506759456 (399h18m53.487112092s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.507955888 (399h18m53.488308525s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.509580758 (399h18m53.489933403s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:14.116474867 (399h18m54.096827503s) [upload=351224]: Next returned nil segment Dec 22 06:47:14.116487753 (399h18m54.096840382s) [upload=351224]: Waiting for error group managing segments... Dec 22 06:47:14.252424715 (399h18m54.232777352s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.259199346 (399h18m54.239551983s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.263033341 (399h18m54.243385978s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.268392055 (399h18m54.2487447s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272217620 (399h18m54.252570256s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272621367 (399h18m54.252974009s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.274087214 (399h18m54.254439851s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.281307529 (399h18m54.261660168s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.290977855 (399h18m54.2713305s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.305725196 (399h18m54.28607784s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:21.393190149 (399h19m1.373542789s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:22.349672394 (399h19m2.33002503s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:43.106830367 (399h19m23.087183004s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:17o881pUMuG7K2dv83k7RkrJCrmM7kgUiWesgam4GEw21KVUn7, address:145.79.227.29:4942): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107030968 (399h19m23.087383607s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107221616 (399h19m23.087574252s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107779626 (399h19m23.088132267s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107809812 (399h19m23.088162439s) [seg_pos={0 0}, upload=351224]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 22 06:47:43.213405603 (399h19m23.193758247s) [upload=351224]: Done uploading segments... Dec 22 06:47:43.213448935 (399h19m23.193801581s) [upload=351224]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.396204173s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 2.480235171s trace: https://jaeger.dev.storj.io/trace/5403b15b90934969 FAIL (2025-12-22 07:23:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 22 06:47:13.390879689 (399h18m53.371232325s) [upload=351224]: Starting upload Dec 22 06:47:13.390933946 (399h18m53.371286588s) [upload=351224]: Done starting upload Dec 22 06:47:13.391022385 (399h18m53.371375007s) [upload=351224]: Uploading segments... Dec 22 06:47:13.391257160 (399h18m53.371609787s) [upload=351224]: Got next segment. Inline: false Dec 22 06:47:13.391304735 (399h18m53.371657378s) [upload=351224]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 06:47:13.391309842 (399h18m53.371662478s) [upload=351224]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 06:47:13.503578512 (399h18m53.48393115s) [seg_pos={0 0}, upload=351224]: Begin upload segment... Dec 22 06:47:13.503670819 (399h18m53.484023465s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504640710 (399h18m53.484993414s) [seg_pos={0 0}, upload=351224]: Done begin upload segment. Dec 22 06:47:13.504785964 (399h18m53.485138878s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504916823 (399h18m53.485269465s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505227903 (399h18m53.485580539s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505250192 (399h18m53.485602828s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505519011 (399h18m53.485871647s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505579890 (399h18m53.485932526s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505839462 (399h18m53.486192109s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506000955 (399h18m53.486353591s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506194887 (399h18m53.486547527s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506408314 (399h18m53.486760957s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506480917 (399h18m53.486833557s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506749655 (399h18m53.48710229s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506759456 (399h18m53.487112092s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.507955888 (399h18m53.488308525s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.509580758 (399h18m53.489933403s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:14.116474867 (399h18m54.096827503s) [upload=351224]: Next returned nil segment Dec 22 06:47:14.116487753 (399h18m54.096840382s) [upload=351224]: Waiting for error group managing segments... Dec 22 06:47:14.252424715 (399h18m54.232777352s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.259199346 (399h18m54.239551983s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.263033341 (399h18m54.243385978s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.268392055 (399h18m54.2487447s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272217620 (399h18m54.252570256s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272621367 (399h18m54.252974009s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.274087214 (399h18m54.254439851s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.281307529 (399h18m54.261660168s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.290977855 (399h18m54.2713305s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.305725196 (399h18m54.28607784s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:21.393190149 (399h19m1.373542789s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:22.349672394 (399h19m2.33002503s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:43.106830367 (399h19m23.087183004s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:17o881pUMuG7K2dv83k7RkrJCrmM7kgUiWesgam4GEw21KVUn7, address:145.79.227.29:4942): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107030968 (399h19m23.087383607s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107221616 (399h19m23.087574252s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107779626 (399h19m23.088132267s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107809812 (399h19m23.088162439s) [seg_pos={0 0}, upload=351224]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 22 06:47:43.213405603 (399h19m23.193758247s) [upload=351224]: Done uploading segments... Dec 22 06:47:43.213448935 (399h19m23.193801581s) [upload=351224]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.480264302s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 2.390313455s trace: https://jaeger.dev.storj.io/trace/5403b15b90934969 FAIL (2025-12-22 07:22:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 22 06:47:13.390879689 (399h18m53.371232325s) [upload=351224]: Starting upload Dec 22 06:47:13.390933946 (399h18m53.371286588s) [upload=351224]: Done starting upload Dec 22 06:47:13.391022385 (399h18m53.371375007s) [upload=351224]: Uploading segments... Dec 22 06:47:13.391257160 (399h18m53.371609787s) [upload=351224]: Got next segment. Inline: false Dec 22 06:47:13.391304735 (399h18m53.371657378s) [upload=351224]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 06:47:13.391309842 (399h18m53.371662478s) [upload=351224]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 06:47:13.503578512 (399h18m53.48393115s) [seg_pos={0 0}, upload=351224]: Begin upload segment... Dec 22 06:47:13.503670819 (399h18m53.484023465s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504640710 (399h18m53.484993414s) [seg_pos={0 0}, upload=351224]: Done begin upload segment. Dec 22 06:47:13.504785964 (399h18m53.485138878s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504916823 (399h18m53.485269465s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505227903 (399h18m53.485580539s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505250192 (399h18m53.485602828s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505519011 (399h18m53.485871647s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505579890 (399h18m53.485932526s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505839462 (399h18m53.486192109s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506000955 (399h18m53.486353591s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506194887 (399h18m53.486547527s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506408314 (399h18m53.486760957s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506480917 (399h18m53.486833557s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506749655 (399h18m53.48710229s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506759456 (399h18m53.487112092s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.507955888 (399h18m53.488308525s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.509580758 (399h18m53.489933403s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:14.116474867 (399h18m54.096827503s) [upload=351224]: Next returned nil segment Dec 22 06:47:14.116487753 (399h18m54.096840382s) [upload=351224]: Waiting for error group managing segments... Dec 22 06:47:14.252424715 (399h18m54.232777352s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.259199346 (399h18m54.239551983s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.263033341 (399h18m54.243385978s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.268392055 (399h18m54.2487447s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272217620 (399h18m54.252570256s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272621367 (399h18m54.252974009s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.274087214 (399h18m54.254439851s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.281307529 (399h18m54.261660168s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.290977855 (399h18m54.2713305s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.305725196 (399h18m54.28607784s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:21.393190149 (399h19m1.373542789s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:22.349672394 (399h19m2.33002503s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:43.106830367 (399h19m23.087183004s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:17o881pUMuG7K2dv83k7RkrJCrmM7kgUiWesgam4GEw21KVUn7, address:145.79.227.29:4942): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107030968 (399h19m23.087383607s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107221616 (399h19m23.087574252s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107779626 (399h19m23.088132267s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107809812 (399h19m23.088162439s) [seg_pos={0 0}, upload=351224]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 22 06:47:43.213405603 (399h19m23.193758247s) [upload=351224]: Done uploading segments... Dec 22 06:47:43.213448935 (399h19m23.193801581s) [upload=351224]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.390377993s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 2.363616282s trace: https://jaeger.dev.storj.io/trace/5403b15b90934969 FAIL (2025-12-22 07:21:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 22 06:47:13.390879689 (399h18m53.371232325s) [upload=351224]: Starting upload Dec 22 06:47:13.390933946 (399h18m53.371286588s) [upload=351224]: Done starting upload Dec 22 06:47:13.391022385 (399h18m53.371375007s) [upload=351224]: Uploading segments... Dec 22 06:47:13.391257160 (399h18m53.371609787s) [upload=351224]: Got next segment. Inline: false Dec 22 06:47:13.391304735 (399h18m53.371657378s) [upload=351224]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 06:47:13.391309842 (399h18m53.371662478s) [upload=351224]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 06:47:13.503578512 (399h18m53.48393115s) [seg_pos={0 0}, upload=351224]: Begin upload segment... Dec 22 06:47:13.503670819 (399h18m53.484023465s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504640710 (399h18m53.484993414s) [seg_pos={0 0}, upload=351224]: Done begin upload segment. Dec 22 06:47:13.504785964 (399h18m53.485138878s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504916823 (399h18m53.485269465s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505227903 (399h18m53.485580539s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505250192 (399h18m53.485602828s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505519011 (399h18m53.485871647s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505579890 (399h18m53.485932526s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505839462 (399h18m53.486192109s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506000955 (399h18m53.486353591s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506194887 (399h18m53.486547527s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506408314 (399h18m53.486760957s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506480917 (399h18m53.486833557s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506749655 (399h18m53.48710229s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506759456 (399h18m53.487112092s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.507955888 (399h18m53.488308525s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.509580758 (399h18m53.489933403s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:14.116474867 (399h18m54.096827503s) [upload=351224]: Next returned nil segment Dec 22 06:47:14.116487753 (399h18m54.096840382s) [upload=351224]: Waiting for error group managing segments... Dec 22 06:47:14.252424715 (399h18m54.232777352s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.259199346 (399h18m54.239551983s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.263033341 (399h18m54.243385978s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.268392055 (399h18m54.2487447s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272217620 (399h18m54.252570256s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272621367 (399h18m54.252974009s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.274087214 (399h18m54.254439851s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.281307529 (399h18m54.261660168s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.290977855 (399h18m54.2713305s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.305725196 (399h18m54.28607784s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:21.393190149 (399h19m1.373542789s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:22.349672394 (399h19m2.33002503s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:43.106830367 (399h19m23.087183004s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:17o881pUMuG7K2dv83k7RkrJCrmM7kgUiWesgam4GEw21KVUn7, address:145.79.227.29:4942): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107030968 (399h19m23.087383607s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107221616 (399h19m23.087574252s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107779626 (399h19m23.088132267s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107809812 (399h19m23.088162439s) [seg_pos={0 0}, upload=351224]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 22 06:47:43.213405603 (399h19m23.193758247s) [upload=351224]: Done uploading segments... Dec 22 06:47:43.213448935 (399h19m23.193801581s) [upload=351224]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.363729686s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 2.352800124s trace: https://jaeger.dev.storj.io/trace/5403b15b90934969 FAIL (2025-12-22 07:20:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 22 06:47:13.390879689 (399h18m53.371232325s) [upload=351224]: Starting upload Dec 22 06:47:13.390933946 (399h18m53.371286588s) [upload=351224]: Done starting upload Dec 22 06:47:13.391022385 (399h18m53.371375007s) [upload=351224]: Uploading segments... Dec 22 06:47:13.391257160 (399h18m53.371609787s) [upload=351224]: Got next segment. Inline: false Dec 22 06:47:13.391304735 (399h18m53.371657378s) [upload=351224]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 06:47:13.391309842 (399h18m53.371662478s) [upload=351224]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 06:47:13.503578512 (399h18m53.48393115s) [seg_pos={0 0}, upload=351224]: Begin upload segment... Dec 22 06:47:13.503670819 (399h18m53.484023465s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504640710 (399h18m53.484993414s) [seg_pos={0 0}, upload=351224]: Done begin upload segment. Dec 22 06:47:13.504785964 (399h18m53.485138878s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504916823 (399h18m53.485269465s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505227903 (399h18m53.485580539s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505250192 (399h18m53.485602828s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505519011 (399h18m53.485871647s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505579890 (399h18m53.485932526s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505839462 (399h18m53.486192109s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506000955 (399h18m53.486353591s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506194887 (399h18m53.486547527s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506408314 (399h18m53.486760957s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506480917 (399h18m53.486833557s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506749655 (399h18m53.48710229s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506759456 (399h18m53.487112092s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.507955888 (399h18m53.488308525s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.509580758 (399h18m53.489933403s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:14.116474867 (399h18m54.096827503s) [upload=351224]: Next returned nil segment Dec 22 06:47:14.116487753 (399h18m54.096840382s) [upload=351224]: Waiting for error group managing segments... Dec 22 06:47:14.252424715 (399h18m54.232777352s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.259199346 (399h18m54.239551983s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.263033341 (399h18m54.243385978s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.268392055 (399h18m54.2487447s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272217620 (399h18m54.252570256s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272621367 (399h18m54.252974009s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.274087214 (399h18m54.254439851s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.281307529 (399h18m54.261660168s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.290977855 (399h18m54.2713305s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.305725196 (399h18m54.28607784s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:21.393190149 (399h19m1.373542789s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:22.349672394 (399h19m2.33002503s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:43.106830367 (399h19m23.087183004s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:17o881pUMuG7K2dv83k7RkrJCrmM7kgUiWesgam4GEw21KVUn7, address:145.79.227.29:4942): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107030968 (399h19m23.087383607s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107221616 (399h19m23.087574252s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107779626 (399h19m23.088132267s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107809812 (399h19m23.088162439s) [seg_pos={0 0}, upload=351224]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 22 06:47:43.213405603 (399h19m23.193758247s) [upload=351224]: Done uploading segments... Dec 22 06:47:43.213448935 (399h19m23.193801581s) [upload=351224]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.352826641s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 2.30318653s trace: https://jaeger.dev.storj.io/trace/5403b15b90934969 FAIL (2025-12-22 07:19:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 22 06:47:13.390879689 (399h18m53.371232325s) [upload=351224]: Starting upload Dec 22 06:47:13.390933946 (399h18m53.371286588s) [upload=351224]: Done starting upload Dec 22 06:47:13.391022385 (399h18m53.371375007s) [upload=351224]: Uploading segments... Dec 22 06:47:13.391257160 (399h18m53.371609787s) [upload=351224]: Got next segment. Inline: false Dec 22 06:47:13.391304735 (399h18m53.371657378s) [upload=351224]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 06:47:13.391309842 (399h18m53.371662478s) [upload=351224]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 06:47:13.503578512 (399h18m53.48393115s) [seg_pos={0 0}, upload=351224]: Begin upload segment... Dec 22 06:47:13.503670819 (399h18m53.484023465s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504640710 (399h18m53.484993414s) [seg_pos={0 0}, upload=351224]: Done begin upload segment. Dec 22 06:47:13.504785964 (399h18m53.485138878s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504916823 (399h18m53.485269465s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505227903 (399h18m53.485580539s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505250192 (399h18m53.485602828s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505519011 (399h18m53.485871647s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505579890 (399h18m53.485932526s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505839462 (399h18m53.486192109s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506000955 (399h18m53.486353591s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506194887 (399h18m53.486547527s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506408314 (399h18m53.486760957s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506480917 (399h18m53.486833557s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506749655 (399h18m53.48710229s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506759456 (399h18m53.487112092s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.507955888 (399h18m53.488308525s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.509580758 (399h18m53.489933403s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:14.116474867 (399h18m54.096827503s) [upload=351224]: Next returned nil segment Dec 22 06:47:14.116487753 (399h18m54.096840382s) [upload=351224]: Waiting for error group managing segments... Dec 22 06:47:14.252424715 (399h18m54.232777352s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.259199346 (399h18m54.239551983s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.263033341 (399h18m54.243385978s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.268392055 (399h18m54.2487447s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272217620 (399h18m54.252570256s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272621367 (399h18m54.252974009s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.274087214 (399h18m54.254439851s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.281307529 (399h18m54.261660168s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.290977855 (399h18m54.2713305s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.305725196 (399h18m54.28607784s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:21.393190149 (399h19m1.373542789s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:22.349672394 (399h19m2.33002503s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:43.106830367 (399h19m23.087183004s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:17o881pUMuG7K2dv83k7RkrJCrmM7kgUiWesgam4GEw21KVUn7, address:145.79.227.29:4942): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107030968 (399h19m23.087383607s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107221616 (399h19m23.087574252s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107779626 (399h19m23.088132267s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107809812 (399h19m23.088162439s) [seg_pos={0 0}, upload=351224]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 22 06:47:43.213405603 (399h19m23.193758247s) [upload=351224]: Done uploading segments... Dec 22 06:47:43.213448935 (399h19m23.193801581s) [upload=351224]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.303212236s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 2.355397127s trace: https://jaeger.dev.storj.io/trace/5403b15b90934969 FAIL (2025-12-22 07:19:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 22 06:47:13.390879689 (399h18m53.371232325s) [upload=351224]: Starting upload Dec 22 06:47:13.390933946 (399h18m53.371286588s) [upload=351224]: Done starting upload Dec 22 06:47:13.391022385 (399h18m53.371375007s) [upload=351224]: Uploading segments... Dec 22 06:47:13.391257160 (399h18m53.371609787s) [upload=351224]: Got next segment. Inline: false Dec 22 06:47:13.391304735 (399h18m53.371657378s) [upload=351224]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 06:47:13.391309842 (399h18m53.371662478s) [upload=351224]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 06:47:13.503578512 (399h18m53.48393115s) [seg_pos={0 0}, upload=351224]: Begin upload segment... Dec 22 06:47:13.503670819 (399h18m53.484023465s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504640710 (399h18m53.484993414s) [seg_pos={0 0}, upload=351224]: Done begin upload segment. Dec 22 06:47:13.504785964 (399h18m53.485138878s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504916823 (399h18m53.485269465s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505227903 (399h18m53.485580539s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505250192 (399h18m53.485602828s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505519011 (399h18m53.485871647s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505579890 (399h18m53.485932526s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505839462 (399h18m53.486192109s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506000955 (399h18m53.486353591s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506194887 (399h18m53.486547527s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506408314 (399h18m53.486760957s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506480917 (399h18m53.486833557s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506749655 (399h18m53.48710229s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506759456 (399h18m53.487112092s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.507955888 (399h18m53.488308525s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.509580758 (399h18m53.489933403s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:14.116474867 (399h18m54.096827503s) [upload=351224]: Next returned nil segment Dec 22 06:47:14.116487753 (399h18m54.096840382s) [upload=351224]: Waiting for error group managing segments... Dec 22 06:47:14.252424715 (399h18m54.232777352s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.259199346 (399h18m54.239551983s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.263033341 (399h18m54.243385978s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.268392055 (399h18m54.2487447s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272217620 (399h18m54.252570256s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272621367 (399h18m54.252974009s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.274087214 (399h18m54.254439851s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.281307529 (399h18m54.261660168s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.290977855 (399h18m54.2713305s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.305725196 (399h18m54.28607784s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:21.393190149 (399h19m1.373542789s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:22.349672394 (399h19m2.33002503s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:43.106830367 (399h19m23.087183004s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:17o881pUMuG7K2dv83k7RkrJCrmM7kgUiWesgam4GEw21KVUn7, address:145.79.227.29:4942): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107030968 (399h19m23.087383607s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107221616 (399h19m23.087574252s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107779626 (399h19m23.088132267s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107809812 (399h19m23.088162439s) [seg_pos={0 0}, upload=351224]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 22 06:47:43.213405603 (399h19m23.193758247s) [upload=351224]: Done uploading segments... Dec 22 06:47:43.213448935 (399h19m23.193801581s) [upload=351224]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.355422757s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 2.293766023s trace: https://jaeger.dev.storj.io/trace/5403b15b90934969 FAIL (2025-12-22 07:18:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 22 06:47:13.390879689 (399h18m53.371232325s) [upload=351224]: Starting upload Dec 22 06:47:13.390933946 (399h18m53.371286588s) [upload=351224]: Done starting upload Dec 22 06:47:13.391022385 (399h18m53.371375007s) [upload=351224]: Uploading segments... Dec 22 06:47:13.391257160 (399h18m53.371609787s) [upload=351224]: Got next segment. Inline: false Dec 22 06:47:13.391304735 (399h18m53.371657378s) [upload=351224]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 06:47:13.391309842 (399h18m53.371662478s) [upload=351224]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 06:47:13.503578512 (399h18m53.48393115s) [seg_pos={0 0}, upload=351224]: Begin upload segment... Dec 22 06:47:13.503670819 (399h18m53.484023465s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504640710 (399h18m53.484993414s) [seg_pos={0 0}, upload=351224]: Done begin upload segment. Dec 22 06:47:13.504785964 (399h18m53.485138878s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504916823 (399h18m53.485269465s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505227903 (399h18m53.485580539s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505250192 (399h18m53.485602828s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505519011 (399h18m53.485871647s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505579890 (399h18m53.485932526s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505839462 (399h18m53.486192109s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506000955 (399h18m53.486353591s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506194887 (399h18m53.486547527s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506408314 (399h18m53.486760957s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506480917 (399h18m53.486833557s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506749655 (399h18m53.48710229s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506759456 (399h18m53.487112092s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.507955888 (399h18m53.488308525s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.509580758 (399h18m53.489933403s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:14.116474867 (399h18m54.096827503s) [upload=351224]: Next returned nil segment Dec 22 06:47:14.116487753 (399h18m54.096840382s) [upload=351224]: Waiting for error group managing segments... Dec 22 06:47:14.252424715 (399h18m54.232777352s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.259199346 (399h18m54.239551983s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.263033341 (399h18m54.243385978s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.268392055 (399h18m54.2487447s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272217620 (399h18m54.252570256s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272621367 (399h18m54.252974009s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.274087214 (399h18m54.254439851s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.281307529 (399h18m54.261660168s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.290977855 (399h18m54.2713305s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.305725196 (399h18m54.28607784s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:21.393190149 (399h19m1.373542789s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:22.349672394 (399h19m2.33002503s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:43.106830367 (399h19m23.087183004s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:17o881pUMuG7K2dv83k7RkrJCrmM7kgUiWesgam4GEw21KVUn7, address:145.79.227.29:4942): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107030968 (399h19m23.087383607s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107221616 (399h19m23.087574252s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107779626 (399h19m23.088132267s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107809812 (399h19m23.088162439s) [seg_pos={0 0}, upload=351224]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 22 06:47:43.213405603 (399h19m23.193758247s) [upload=351224]: Done uploading segments... Dec 22 06:47:43.213448935 (399h19m23.193801581s) [upload=351224]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.29381447s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 2.442088608s trace: https://jaeger.dev.storj.io/trace/5403b15b90934969 FAIL (2025-12-22 07:17:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 22 06:47:13.390879689 (399h18m53.371232325s) [upload=351224]: Starting upload Dec 22 06:47:13.390933946 (399h18m53.371286588s) [upload=351224]: Done starting upload Dec 22 06:47:13.391022385 (399h18m53.371375007s) [upload=351224]: Uploading segments... Dec 22 06:47:13.391257160 (399h18m53.371609787s) [upload=351224]: Got next segment. Inline: false Dec 22 06:47:13.391304735 (399h18m53.371657378s) [upload=351224]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 06:47:13.391309842 (399h18m53.371662478s) [upload=351224]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 06:47:13.503578512 (399h18m53.48393115s) [seg_pos={0 0}, upload=351224]: Begin upload segment... Dec 22 06:47:13.503670819 (399h18m53.484023465s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504640710 (399h18m53.484993414s) [seg_pos={0 0}, upload=351224]: Done begin upload segment. Dec 22 06:47:13.504785964 (399h18m53.485138878s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504916823 (399h18m53.485269465s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505227903 (399h18m53.485580539s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505250192 (399h18m53.485602828s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505519011 (399h18m53.485871647s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505579890 (399h18m53.485932526s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505839462 (399h18m53.486192109s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506000955 (399h18m53.486353591s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506194887 (399h18m53.486547527s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506408314 (399h18m53.486760957s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506480917 (399h18m53.486833557s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506749655 (399h18m53.48710229s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506759456 (399h18m53.487112092s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.507955888 (399h18m53.488308525s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.509580758 (399h18m53.489933403s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:14.116474867 (399h18m54.096827503s) [upload=351224]: Next returned nil segment Dec 22 06:47:14.116487753 (399h18m54.096840382s) [upload=351224]: Waiting for error group managing segments... Dec 22 06:47:14.252424715 (399h18m54.232777352s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.259199346 (399h18m54.239551983s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.263033341 (399h18m54.243385978s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.268392055 (399h18m54.2487447s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272217620 (399h18m54.252570256s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272621367 (399h18m54.252974009s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.274087214 (399h18m54.254439851s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.281307529 (399h18m54.261660168s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.290977855 (399h18m54.2713305s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.305725196 (399h18m54.28607784s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:21.393190149 (399h19m1.373542789s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:22.349672394 (399h19m2.33002503s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:43.106830367 (399h19m23.087183004s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:17o881pUMuG7K2dv83k7RkrJCrmM7kgUiWesgam4GEw21KVUn7, address:145.79.227.29:4942): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107030968 (399h19m23.087383607s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107221616 (399h19m23.087574252s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107779626 (399h19m23.088132267s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107809812 (399h19m23.088162439s) [seg_pos={0 0}, upload=351224]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 22 06:47:43.213405603 (399h19m23.193758247s) [upload=351224]: Done uploading segments... Dec 22 06:47:43.213448935 (399h19m23.193801581s) [upload=351224]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.442115131s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 2.356463619s trace: https://jaeger.dev.storj.io/trace/5403b15b90934969 FAIL (2025-12-22 07:16:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 22 06:47:13.390879689 (399h18m53.371232325s) [upload=351224]: Starting upload Dec 22 06:47:13.390933946 (399h18m53.371286588s) [upload=351224]: Done starting upload Dec 22 06:47:13.391022385 (399h18m53.371375007s) [upload=351224]: Uploading segments... Dec 22 06:47:13.391257160 (399h18m53.371609787s) [upload=351224]: Got next segment. Inline: false Dec 22 06:47:13.391304735 (399h18m53.371657378s) [upload=351224]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 06:47:13.391309842 (399h18m53.371662478s) [upload=351224]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 06:47:13.503578512 (399h18m53.48393115s) [seg_pos={0 0}, upload=351224]: Begin upload segment... Dec 22 06:47:13.503670819 (399h18m53.484023465s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504640710 (399h18m53.484993414s) [seg_pos={0 0}, upload=351224]: Done begin upload segment. Dec 22 06:47:13.504785964 (399h18m53.485138878s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.504916823 (399h18m53.485269465s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505227903 (399h18m53.485580539s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505250192 (399h18m53.485602828s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505519011 (399h18m53.485871647s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505579890 (399h18m53.485932526s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.505839462 (399h18m53.486192109s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506000955 (399h18m53.486353591s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506194887 (399h18m53.486547527s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506408314 (399h18m53.486760957s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506480917 (399h18m53.486833557s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506749655 (399h18m53.48710229s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.506759456 (399h18m53.487112092s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.507955888 (399h18m53.488308525s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:13.509580758 (399h18m53.489933403s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Uploading piece... Dec 22 06:47:14.116474867 (399h18m54.096827503s) [upload=351224]: Next returned nil segment Dec 22 06:47:14.116487753 (399h18m54.096840382s) [upload=351224]: Waiting for error group managing segments... Dec 22 06:47:14.252424715 (399h18m54.232777352s) [noise=true , address=77.90.31.19:4842 , piece_id=EM2LSIEISDYOD4M6GEBXZ3FS3TFSFTPNVBGWL63GY3KNSRNDB7LQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.259199346 (399h18m54.239551983s) [noise=true , address=45.91.116.6:4976 , piece_id=U4OR7ZIHIU5GSFU7V2CI4D4BVBNHFK6IXQZPEJXO5EZYFU2Q5HSA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.263033341 (399h18m54.243385978s) [noise=true , address=145.79.227.16:4910 , piece_id=XVCKNOPRCVY5DHZ4O65RJCEWHOAGFO7QNUL5NQX67FCAXFWGFFBA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.268392055 (399h18m54.2487447s) [noise=true , address=145.79.227.45:4230 , piece_id=XCWRUKHSKBWUNMQ7PAGAZCUGU2JGYHR3YN7A73VXSSZGH4SRHDXQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272217620 (399h18m54.252570256s) [noise=true , address=145.79.227.34:4905 , piece_id=FMMFST4XFUFCC45LJASZWNRU2GKJOLYFFBDXZCSHPULN7PYRA34Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.272621367 (399h18m54.252974009s) [noise=true , address=145.79.227.4:4187 , piece_id=7FXNQSCCEXKMRNQ4TMIETZMXSL34LGIHZW43RRQUYLBVC6JC2RRA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.274087214 (399h18m54.254439851s) [noise=true , address=77.90.31.14:4662 , piece_id=QHEQJTJU43V4QJSUJH5UQRUNO7BIG5TGVMFVUZEAAGSCTV4LJGOQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.281307529 (399h18m54.261660168s) [noise=true , address=45.91.116.5:4298 , piece_id=Z3EZHUWSEDLLQO45UJQWEGDI2T2EGFIN5QK44KEJKMFHJS47YKGQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.290977855 (399h18m54.2713305s) [noise=true , address=145.79.227.51:4410 , piece_id=CFOA5CQ4SQYWJB64G22GSFUU42E2PTGYSSVM53WXFUHOUVP6J47A, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:14.305725196 (399h18m54.28607784s) [noise=true , address=145.79.227.41:4086 , piece_id=ADAWH4SVXEJ5HVMLBN7QXLN5CTJ4XYCUSDTEMBDTRPBMJYO47SIQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:21.393190149 (399h19m1.373542789s) [noise=true , address=145.79.227.23:4547 , piece_id=WPZ3MCPYCISSFJFCU5ROE2QG6NIDAMJKL2XXCPKD6VFKCNLMC7LA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:22.349672394 (399h19m2.33002503s) [noise=true , address=145.79.227.28:4727 , piece_id=77AWXI32H7JJCVHHC23OXHYURQYJUN7EDEOZGYI2TBKGBBLHJCEA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: <nil> Dec 22 06:47:43.106830367 (399h19m23.087183004s) [noise=true , address=145.79.227.29:4942 , piece_id=GSFL36URKGAC7IFVCLJW4EVRLUONYQR6OAFHSR6LCCT4KGG74QFA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:17o881pUMuG7K2dv83k7RkrJCrmM7kgUiWesgam4GEw21KVUn7, address:145.79.227.29:4942): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107030968 (399h19m23.087383607s) [noise=true , address=145.79.227.40:4049 , piece_id=JDNXRRGRNTGSM4WN3IZTBTLVI3C5LMIXICTEVE6B2DOLEXY2JG2Q, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107221616 (399h19m23.087574252s) [noise=true , address=145.79.227.8:4329 , piece_id=CIDYWZXFMW6QVG7F3W36JOIVFL6L5WA4VPYHKZN7BZ2AO4FPCHCQ, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107779626 (399h19m23.088132267s) [noise=true , address=145.79.227.32:4835 , piece_id=FSFZYWEJHZZSWVCEEFGNXPOVLV2HNWOVJKV3DGPQ36PLIF3P2MWA, seg_pos={0 0}, upload=351224]: Done uploading piece. err: ecclient: upload failed (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y, address:145.79.227.32:4835): protocol: expected piece hash; context deadline exceeded; EOF Dec 22 06:47:43.107809812 (399h19m23.088162439s) [seg_pos={0 0}, upload=351224]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 22 06:47:43.213405603 (399h19m23.193758247s) [upload=351224]: Done uploading segments... Dec 22 06:47:43.213448935 (399h19m23.193801581s) [upload=351224]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.356497416s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)