Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 2.494857089s trace: https://jaeger.dev.storj.io/trace/6086529041566513 FAIL (2025-12-08 19:02:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 8 18:54:57.674974846 (75h26m37.655327486s) [upload=66264]: Starting upload Dec 8 18:54:57.675038729 (75h26m37.655391369s) [upload=66264]: Done starting upload Dec 8 18:54:57.675330133 (75h26m37.655682761s) [upload=66264]: Uploading segments... Dec 8 18:54:57.675406307 (75h26m37.655758943s) [upload=66264]: Got next segment. Inline: false Dec 8 18:54:57.675485211 (75h26m37.655837857s) [upload=66264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 8 18:54:57.675493430 (75h26m37.655846073s) [upload=66264]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 8 18:54:57.785146518 (75h26m37.765499144s) [seg_pos={0 0}, upload=66264]: Begin upload segment... Dec 8 18:54:57.785240478 (75h26m37.765593118s) [seg_pos={0 0}, upload=66264]: Done begin upload segment. Dec 8 18:54:57.785485278 (75h26m37.765837924s) [noise=true , address=145.79.227.32:4835 , piece_id=AFJ6KUBQP5LSAJUSUN7DOSNJ5TBWBBQRRYBTARKHKMOCWGDHDRJQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.785679403 (75h26m37.766032052s) [noise=true , address=77.90.31.12:4590 , piece_id=P6H4KDKDRZZWGB3FVK4KB42KORBAHHHJES65VCTFD72D4JH5VJVA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.786641527 (75h26m37.766994177s) [noise=true , address=145.79.227.30:4763 , piece_id=5YDXU37WXFV36D2XH76XQ6BNY45SYZYTWGV3HBEFNVVT5LBW3NNQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787110826 (75h26m37.767463468s) [noise=true , address=77.90.31.16:4735 , piece_id=P77WZF5A3CNM7ZXREICQQUGBDYJPHYQIQ4K23NKEXXBY4XFZV73Q, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787261208 (75h26m37.767613854s) [noise=true , address=145.79.227.20:4439 , piece_id=HDPIFW4OJ6CY5LKLJ57IPHRPFDTXHVDTRJ7FKAXDY7PER6QPXQAA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787489023 (75h26m37.76784167s) [noise=true , address=77.90.31.11:4554 , piece_id=MJEVYVGP5NQZENUOE3NNBBS3BYC52642XB5OGF2K6J7YHNYORNPQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787843253 (75h26m37.768195901s) [noise=true , address=145.79.227.41:4085 , piece_id=HKKDK2YHRBBULRLBYAVC7QZQR2HGFH4DXZDJAS6EYR654U7SV2RA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787852122 (75h26m37.768204761s) [noise=true , address=145.79.227.25:4619 , piece_id=IPCP5WU2OE2JZDF47UKPU7GFPEIKLLBETHPV6UOQ7EYZG7QXUZJQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.788192899 (75h26m37.768545534s) [noise=true , address=145.79.227.21:4474 , piece_id=4PXKGNK6IBHWO7TKWG2O3BJI5P4MSYUWZD4NJ7JMOXP2TCY3VVWA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.788412033 (75h26m37.76876468s) [noise=true , address=145.79.227.27:4690 , piece_id=S6DIBWGQA3NAM3E3AS5QIB7QJL2SYSDFGANJYOZA3MTT6MQZS7AQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.788516029 (75h26m37.768868665s) [noise=true , address=145.79.227.22:4511 , piece_id=HE3JRGJIDNHPZFAXR2Y3HOXYZSCOKQ4OP6VPCLBLLZUFMBYU3WOA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.788933691 (75h26m37.769286334s) [noise=true , address=145.79.227.33:4871 , piece_id=HCUMHPIMTFBWYNYL5NHFR5AQSYSIDIN2D7EO7ZYHG5GUCAYMLAPQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.789013511 (75h26m37.769366149s) [noise=true , address=77.90.31.17:4772 , piece_id=Q4QNOKUPLIVMP7DVRAQSYKVCBIFYVVL7CYTVOKYYLQZSXD2JMRCQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.789551986 (75h26m37.769904633s) [noise=true , address=77.90.31.21:4878 , piece_id=7PHBVJ5HENMIOKLSRV34B6LOBJUGG6O3LUY4J3M6EFHV57JPXLSA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.790354423 (75h26m37.770707062s) [noise=true , address=77.90.31.12:4589 , piece_id=XA2OR2VJMLK6LGEABP7UBIQNZG6BXVMUX5XP7SVMLBEKJE5YDP7Q, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.790934325 (75h26m37.771286971s) [noise=true , address=145.79.227.39:4014 , piece_id=XPW63KAMFGYPJULSRSYPO3V5PIJX5DH7BLGFCDSYRSU46AXCYYRA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:58.172348027 (75h26m38.15270068s) [noise=true , address=77.90.31.12:4590 , piece_id=P6H4KDKDRZZWGB3FVK4KB42KORBAHHHJES65VCTFD72D4JH5VJVA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000 Dec 8 18:54:58.172831033 (75h26m38.153183679s) [noise=true , address=145.79.227.30:4763 , piece_id=5YDXU37WXFV36D2XH76XQ6BNY45SYZYTWGV3HBEFNVVT5LBW3NNQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: ecclient: upload failed (node:12Xy6Y953cfdoyywnf98XTrbd9WuDZNEV8SRhkBKbZRHrU2ymPg, address:145.79.227.30:4763): protocol: storage node overloaded, request limit: 2000 Dec 8 18:54:58.178590990 (75h26m38.158943635s) [noise=true , address=77.90.31.12:4589 , piece_id=XA2OR2VJMLK6LGEABP7UBIQNZG6BXVMUX5XP7SVMLBEKJE5YDP7Q, seg_pos={0 0}, upload=66264]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000 Dec 8 18:54:58.392481216 (75h26m38.372833849s) [upload=66264]: Next returned nil segment Dec 8 18:54:58.392499713 (75h26m38.372852355s) [upload=66264]: Waiting for error group managing segments... Dec 8 18:54:58.531890881 (75h26m38.512243518s) [noise=true , address=77.90.31.11:4554 , piece_id=MJEVYVGP5NQZENUOE3NNBBS3BYC52642XB5OGF2K6J7YHNYORNPQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.532848367 (75h26m38.513201014s) [noise=true , address=77.90.31.17:4772 , piece_id=Q4QNOKUPLIVMP7DVRAQSYKVCBIFYVVL7CYTVOKYYLQZSXD2JMRCQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.532895902 (75h26m38.513248545s) [noise=true , address=145.79.227.41:4085 , piece_id=HKKDK2YHRBBULRLBYAVC7QZQR2HGFH4DXZDJAS6EYR654U7SV2RA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.536318192 (75h26m38.516670836s) [noise=true , address=145.79.227.27:4690 , piece_id=S6DIBWGQA3NAM3E3AS5QIB7QJL2SYSDFGANJYOZA3MTT6MQZS7AQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.538479769 (75h26m38.518832418s) [noise=true , address=145.79.227.21:4474 , piece_id=4PXKGNK6IBHWO7TKWG2O3BJI5P4MSYUWZD4NJ7JMOXP2TCY3VVWA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.539308517 (75h26m38.519661165s) [noise=true , address=145.79.227.39:4014 , piece_id=XPW63KAMFGYPJULSRSYPO3V5PIJX5DH7BLGFCDSYRSU46AXCYYRA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.541778178 (75h26m38.522130824s) [noise=true , address=77.90.31.21:4878 , piece_id=7PHBVJ5HENMIOKLSRV34B6LOBJUGG6O3LUY4J3M6EFHV57JPXLSA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.553574259 (75h26m38.533926895s) [noise=true , address=145.79.227.25:4619 , piece_id=IPCP5WU2OE2JZDF47UKPU7GFPEIKLLBETHPV6UOQ7EYZG7QXUZJQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.555434734 (75h26m38.535787374s) [noise=true , address=145.79.227.20:4439 , piece_id=HDPIFW4OJ6CY5LKLJ57IPHRPFDTXHVDTRJ7FKAXDY7PER6QPXQAA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.556116312 (75h26m38.536468949s) [noise=true , address=145.79.227.32:4835 , piece_id=AFJ6KUBQP5LSAJUSUN7DOSNJ5TBWBBQRRYBTARKHKMOCWGDHDRJQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.556487637 (75h26m38.536840274s) [noise=true , address=145.79.227.22:4511 , piece_id=HE3JRGJIDNHPZFAXR2Y3HOXYZSCOKQ4OP6VPCLBLLZUFMBYU3WOA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.714141565 (75h26m38.694494204s) [noise=true , address=145.79.227.33:4871 , piece_id=HCUMHPIMTFBWYNYL5NHFR5AQSYSIDIN2D7EO7ZYHG5GUCAYMLAPQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:55:27.392456241 (75h27m7.372808888s) [noise=true , address=77.90.31.16:4735 , piece_id=P77WZF5A3CNM7ZXREICQQUGBDYJPHYQIQ4K23NKEXXBY4XFZV73Q, seg_pos={0 0}, upload=66264]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: expected piece hash; context deadline exceeded; EOF Dec 8 18:55:27.392541526 (75h27m7.372894167s) [seg_pos={0 0}, upload=66264]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 8 18:55:27.504922383 (75h27m7.485275021s) [upload=66264]: Done uploading segments... Dec 8 18:55:27.504964045 (75h27m7.485316691s) [upload=66264]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.49488425s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 2.485258899s trace: https://jaeger.dev.storj.io/trace/655d188af1aa270 FAIL (2025-12-08 19:01:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.485288276s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 2.656214074s trace: https://jaeger.dev.storj.io/trace/655d188af1aa270 FAIL (2025-12-08 19:01:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.656257839s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 2.43680789s trace: https://jaeger.dev.storj.io/trace/6086529041566513 FAIL (2025-12-08 19:01:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 8 18:54:57.674974846 (75h26m37.655327486s) [upload=66264]: Starting upload Dec 8 18:54:57.675038729 (75h26m37.655391369s) [upload=66264]: Done starting upload Dec 8 18:54:57.675330133 (75h26m37.655682761s) [upload=66264]: Uploading segments... Dec 8 18:54:57.675406307 (75h26m37.655758943s) [upload=66264]: Got next segment. Inline: false Dec 8 18:54:57.675485211 (75h26m37.655837857s) [upload=66264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 8 18:54:57.675493430 (75h26m37.655846073s) [upload=66264]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 8 18:54:57.785146518 (75h26m37.765499144s) [seg_pos={0 0}, upload=66264]: Begin upload segment... Dec 8 18:54:57.785240478 (75h26m37.765593118s) [seg_pos={0 0}, upload=66264]: Done begin upload segment. Dec 8 18:54:57.785485278 (75h26m37.765837924s) [noise=true , address=145.79.227.32:4835 , piece_id=AFJ6KUBQP5LSAJUSUN7DOSNJ5TBWBBQRRYBTARKHKMOCWGDHDRJQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.785679403 (75h26m37.766032052s) [noise=true , address=77.90.31.12:4590 , piece_id=P6H4KDKDRZZWGB3FVK4KB42KORBAHHHJES65VCTFD72D4JH5VJVA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.786641527 (75h26m37.766994177s) [noise=true , address=145.79.227.30:4763 , piece_id=5YDXU37WXFV36D2XH76XQ6BNY45SYZYTWGV3HBEFNVVT5LBW3NNQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787110826 (75h26m37.767463468s) [noise=true , address=77.90.31.16:4735 , piece_id=P77WZF5A3CNM7ZXREICQQUGBDYJPHYQIQ4K23NKEXXBY4XFZV73Q, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787261208 (75h26m37.767613854s) [noise=true , address=145.79.227.20:4439 , piece_id=HDPIFW4OJ6CY5LKLJ57IPHRPFDTXHVDTRJ7FKAXDY7PER6QPXQAA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787489023 (75h26m37.76784167s) [noise=true , address=77.90.31.11:4554 , piece_id=MJEVYVGP5NQZENUOE3NNBBS3BYC52642XB5OGF2K6J7YHNYORNPQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787843253 (75h26m37.768195901s) [noise=true , address=145.79.227.41:4085 , piece_id=HKKDK2YHRBBULRLBYAVC7QZQR2HGFH4DXZDJAS6EYR654U7SV2RA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787852122 (75h26m37.768204761s) [noise=true , address=145.79.227.25:4619 , piece_id=IPCP5WU2OE2JZDF47UKPU7GFPEIKLLBETHPV6UOQ7EYZG7QXUZJQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.788192899 (75h26m37.768545534s) [noise=true , address=145.79.227.21:4474 , piece_id=4PXKGNK6IBHWO7TKWG2O3BJI5P4MSYUWZD4NJ7JMOXP2TCY3VVWA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.788412033 (75h26m37.76876468s) [noise=true , address=145.79.227.27:4690 , piece_id=S6DIBWGQA3NAM3E3AS5QIB7QJL2SYSDFGANJYOZA3MTT6MQZS7AQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.788516029 (75h26m37.768868665s) [noise=true , address=145.79.227.22:4511 , piece_id=HE3JRGJIDNHPZFAXR2Y3HOXYZSCOKQ4OP6VPCLBLLZUFMBYU3WOA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.788933691 (75h26m37.769286334s) [noise=true , address=145.79.227.33:4871 , piece_id=HCUMHPIMTFBWYNYL5NHFR5AQSYSIDIN2D7EO7ZYHG5GUCAYMLAPQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.789013511 (75h26m37.769366149s) [noise=true , address=77.90.31.17:4772 , piece_id=Q4QNOKUPLIVMP7DVRAQSYKVCBIFYVVL7CYTVOKYYLQZSXD2JMRCQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.789551986 (75h26m37.769904633s) [noise=true , address=77.90.31.21:4878 , piece_id=7PHBVJ5HENMIOKLSRV34B6LOBJUGG6O3LUY4J3M6EFHV57JPXLSA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.790354423 (75h26m37.770707062s) [noise=true , address=77.90.31.12:4589 , piece_id=XA2OR2VJMLK6LGEABP7UBIQNZG6BXVMUX5XP7SVMLBEKJE5YDP7Q, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.790934325 (75h26m37.771286971s) [noise=true , address=145.79.227.39:4014 , piece_id=XPW63KAMFGYPJULSRSYPO3V5PIJX5DH7BLGFCDSYRSU46AXCYYRA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:58.172348027 (75h26m38.15270068s) [noise=true , address=77.90.31.12:4590 , piece_id=P6H4KDKDRZZWGB3FVK4KB42KORBAHHHJES65VCTFD72D4JH5VJVA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000 Dec 8 18:54:58.172831033 (75h26m38.153183679s) [noise=true , address=145.79.227.30:4763 , piece_id=5YDXU37WXFV36D2XH76XQ6BNY45SYZYTWGV3HBEFNVVT5LBW3NNQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: ecclient: upload failed (node:12Xy6Y953cfdoyywnf98XTrbd9WuDZNEV8SRhkBKbZRHrU2ymPg, address:145.79.227.30:4763): protocol: storage node overloaded, request limit: 2000 Dec 8 18:54:58.178590990 (75h26m38.158943635s) [noise=true , address=77.90.31.12:4589 , piece_id=XA2OR2VJMLK6LGEABP7UBIQNZG6BXVMUX5XP7SVMLBEKJE5YDP7Q, seg_pos={0 0}, upload=66264]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000 Dec 8 18:54:58.392481216 (75h26m38.372833849s) [upload=66264]: Next returned nil segment Dec 8 18:54:58.392499713 (75h26m38.372852355s) [upload=66264]: Waiting for error group managing segments... Dec 8 18:54:58.531890881 (75h26m38.512243518s) [noise=true , address=77.90.31.11:4554 , piece_id=MJEVYVGP5NQZENUOE3NNBBS3BYC52642XB5OGF2K6J7YHNYORNPQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.532848367 (75h26m38.513201014s) [noise=true , address=77.90.31.17:4772 , piece_id=Q4QNOKUPLIVMP7DVRAQSYKVCBIFYVVL7CYTVOKYYLQZSXD2JMRCQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.532895902 (75h26m38.513248545s) [noise=true , address=145.79.227.41:4085 , piece_id=HKKDK2YHRBBULRLBYAVC7QZQR2HGFH4DXZDJAS6EYR654U7SV2RA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.536318192 (75h26m38.516670836s) [noise=true , address=145.79.227.27:4690 , piece_id=S6DIBWGQA3NAM3E3AS5QIB7QJL2SYSDFGANJYOZA3MTT6MQZS7AQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.538479769 (75h26m38.518832418s) [noise=true , address=145.79.227.21:4474 , piece_id=4PXKGNK6IBHWO7TKWG2O3BJI5P4MSYUWZD4NJ7JMOXP2TCY3VVWA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.539308517 (75h26m38.519661165s) [noise=true , address=145.79.227.39:4014 , piece_id=XPW63KAMFGYPJULSRSYPO3V5PIJX5DH7BLGFCDSYRSU46AXCYYRA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.541778178 (75h26m38.522130824s) [noise=true , address=77.90.31.21:4878 , piece_id=7PHBVJ5HENMIOKLSRV34B6LOBJUGG6O3LUY4J3M6EFHV57JPXLSA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.553574259 (75h26m38.533926895s) [noise=true , address=145.79.227.25:4619 , piece_id=IPCP5WU2OE2JZDF47UKPU7GFPEIKLLBETHPV6UOQ7EYZG7QXUZJQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.555434734 (75h26m38.535787374s) [noise=true , address=145.79.227.20:4439 , piece_id=HDPIFW4OJ6CY5LKLJ57IPHRPFDTXHVDTRJ7FKAXDY7PER6QPXQAA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.556116312 (75h26m38.536468949s) [noise=true , address=145.79.227.32:4835 , piece_id=AFJ6KUBQP5LSAJUSUN7DOSNJ5TBWBBQRRYBTARKHKMOCWGDHDRJQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.556487637 (75h26m38.536840274s) [noise=true , address=145.79.227.22:4511 , piece_id=HE3JRGJIDNHPZFAXR2Y3HOXYZSCOKQ4OP6VPCLBLLZUFMBYU3WOA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.714141565 (75h26m38.694494204s) [noise=true , address=145.79.227.33:4871 , piece_id=HCUMHPIMTFBWYNYL5NHFR5AQSYSIDIN2D7EO7ZYHG5GUCAYMLAPQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:55:27.392456241 (75h27m7.372808888s) [noise=true , address=77.90.31.16:4735 , piece_id=P77WZF5A3CNM7ZXREICQQUGBDYJPHYQIQ4K23NKEXXBY4XFZV73Q, seg_pos={0 0}, upload=66264]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: expected piece hash; context deadline exceeded; EOF Dec 8 18:55:27.392541526 (75h27m7.372894167s) [seg_pos={0 0}, upload=66264]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 8 18:55:27.504922383 (75h27m7.485275021s) [upload=66264]: Done uploading segments... Dec 8 18:55:27.504964045 (75h27m7.485316691s) [upload=66264]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.436830998s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 2.402391231s trace: https://jaeger.dev.storj.io/trace/6086529041566513 FAIL (2025-12-08 19:00:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 8 18:54:57.674974846 (75h26m37.655327486s) [upload=66264]: Starting upload Dec 8 18:54:57.675038729 (75h26m37.655391369s) [upload=66264]: Done starting upload Dec 8 18:54:57.675330133 (75h26m37.655682761s) [upload=66264]: Uploading segments... Dec 8 18:54:57.675406307 (75h26m37.655758943s) [upload=66264]: Got next segment. Inline: false Dec 8 18:54:57.675485211 (75h26m37.655837857s) [upload=66264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 8 18:54:57.675493430 (75h26m37.655846073s) [upload=66264]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 8 18:54:57.785146518 (75h26m37.765499144s) [seg_pos={0 0}, upload=66264]: Begin upload segment... Dec 8 18:54:57.785240478 (75h26m37.765593118s) [seg_pos={0 0}, upload=66264]: Done begin upload segment. Dec 8 18:54:57.785485278 (75h26m37.765837924s) [noise=true , address=145.79.227.32:4835 , piece_id=AFJ6KUBQP5LSAJUSUN7DOSNJ5TBWBBQRRYBTARKHKMOCWGDHDRJQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.785679403 (75h26m37.766032052s) [noise=true , address=77.90.31.12:4590 , piece_id=P6H4KDKDRZZWGB3FVK4KB42KORBAHHHJES65VCTFD72D4JH5VJVA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.786641527 (75h26m37.766994177s) [noise=true , address=145.79.227.30:4763 , piece_id=5YDXU37WXFV36D2XH76XQ6BNY45SYZYTWGV3HBEFNVVT5LBW3NNQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787110826 (75h26m37.767463468s) [noise=true , address=77.90.31.16:4735 , piece_id=P77WZF5A3CNM7ZXREICQQUGBDYJPHYQIQ4K23NKEXXBY4XFZV73Q, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787261208 (75h26m37.767613854s) [noise=true , address=145.79.227.20:4439 , piece_id=HDPIFW4OJ6CY5LKLJ57IPHRPFDTXHVDTRJ7FKAXDY7PER6QPXQAA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787489023 (75h26m37.76784167s) [noise=true , address=77.90.31.11:4554 , piece_id=MJEVYVGP5NQZENUOE3NNBBS3BYC52642XB5OGF2K6J7YHNYORNPQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787843253 (75h26m37.768195901s) [noise=true , address=145.79.227.41:4085 , piece_id=HKKDK2YHRBBULRLBYAVC7QZQR2HGFH4DXZDJAS6EYR654U7SV2RA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787852122 (75h26m37.768204761s) [noise=true , address=145.79.227.25:4619 , piece_id=IPCP5WU2OE2JZDF47UKPU7GFPEIKLLBETHPV6UOQ7EYZG7QXUZJQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.788192899 (75h26m37.768545534s) [noise=true , address=145.79.227.21:4474 , piece_id=4PXKGNK6IBHWO7TKWG2O3BJI5P4MSYUWZD4NJ7JMOXP2TCY3VVWA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.788412033 (75h26m37.76876468s) [noise=true , address=145.79.227.27:4690 , piece_id=S6DIBWGQA3NAM3E3AS5QIB7QJL2SYSDFGANJYOZA3MTT6MQZS7AQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.788516029 (75h26m37.768868665s) [noise=true , address=145.79.227.22:4511 , piece_id=HE3JRGJIDNHPZFAXR2Y3HOXYZSCOKQ4OP6VPCLBLLZUFMBYU3WOA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.788933691 (75h26m37.769286334s) [noise=true , address=145.79.227.33:4871 , piece_id=HCUMHPIMTFBWYNYL5NHFR5AQSYSIDIN2D7EO7ZYHG5GUCAYMLAPQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.789013511 (75h26m37.769366149s) [noise=true , address=77.90.31.17:4772 , piece_id=Q4QNOKUPLIVMP7DVRAQSYKVCBIFYVVL7CYTVOKYYLQZSXD2JMRCQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.789551986 (75h26m37.769904633s) [noise=true , address=77.90.31.21:4878 , piece_id=7PHBVJ5HENMIOKLSRV34B6LOBJUGG6O3LUY4J3M6EFHV57JPXLSA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.790354423 (75h26m37.770707062s) [noise=true , address=77.90.31.12:4589 , piece_id=XA2OR2VJMLK6LGEABP7UBIQNZG6BXVMUX5XP7SVMLBEKJE5YDP7Q, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.790934325 (75h26m37.771286971s) [noise=true , address=145.79.227.39:4014 , piece_id=XPW63KAMFGYPJULSRSYPO3V5PIJX5DH7BLGFCDSYRSU46AXCYYRA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:58.172348027 (75h26m38.15270068s) [noise=true , address=77.90.31.12:4590 , piece_id=P6H4KDKDRZZWGB3FVK4KB42KORBAHHHJES65VCTFD72D4JH5VJVA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000 Dec 8 18:54:58.172831033 (75h26m38.153183679s) [noise=true , address=145.79.227.30:4763 , piece_id=5YDXU37WXFV36D2XH76XQ6BNY45SYZYTWGV3HBEFNVVT5LBW3NNQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: ecclient: upload failed (node:12Xy6Y953cfdoyywnf98XTrbd9WuDZNEV8SRhkBKbZRHrU2ymPg, address:145.79.227.30:4763): protocol: storage node overloaded, request limit: 2000 Dec 8 18:54:58.178590990 (75h26m38.158943635s) [noise=true , address=77.90.31.12:4589 , piece_id=XA2OR2VJMLK6LGEABP7UBIQNZG6BXVMUX5XP7SVMLBEKJE5YDP7Q, seg_pos={0 0}, upload=66264]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000 Dec 8 18:54:58.392481216 (75h26m38.372833849s) [upload=66264]: Next returned nil segment Dec 8 18:54:58.392499713 (75h26m38.372852355s) [upload=66264]: Waiting for error group managing segments... Dec 8 18:54:58.531890881 (75h26m38.512243518s) [noise=true , address=77.90.31.11:4554 , piece_id=MJEVYVGP5NQZENUOE3NNBBS3BYC52642XB5OGF2K6J7YHNYORNPQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.532848367 (75h26m38.513201014s) [noise=true , address=77.90.31.17:4772 , piece_id=Q4QNOKUPLIVMP7DVRAQSYKVCBIFYVVL7CYTVOKYYLQZSXD2JMRCQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.532895902 (75h26m38.513248545s) [noise=true , address=145.79.227.41:4085 , piece_id=HKKDK2YHRBBULRLBYAVC7QZQR2HGFH4DXZDJAS6EYR654U7SV2RA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.536318192 (75h26m38.516670836s) [noise=true , address=145.79.227.27:4690 , piece_id=S6DIBWGQA3NAM3E3AS5QIB7QJL2SYSDFGANJYOZA3MTT6MQZS7AQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.538479769 (75h26m38.518832418s) [noise=true , address=145.79.227.21:4474 , piece_id=4PXKGNK6IBHWO7TKWG2O3BJI5P4MSYUWZD4NJ7JMOXP2TCY3VVWA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.539308517 (75h26m38.519661165s) [noise=true , address=145.79.227.39:4014 , piece_id=XPW63KAMFGYPJULSRSYPO3V5PIJX5DH7BLGFCDSYRSU46AXCYYRA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.541778178 (75h26m38.522130824s) [noise=true , address=77.90.31.21:4878 , piece_id=7PHBVJ5HENMIOKLSRV34B6LOBJUGG6O3LUY4J3M6EFHV57JPXLSA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.553574259 (75h26m38.533926895s) [noise=true , address=145.79.227.25:4619 , piece_id=IPCP5WU2OE2JZDF47UKPU7GFPEIKLLBETHPV6UOQ7EYZG7QXUZJQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.555434734 (75h26m38.535787374s) [noise=true , address=145.79.227.20:4439 , piece_id=HDPIFW4OJ6CY5LKLJ57IPHRPFDTXHVDTRJ7FKAXDY7PER6QPXQAA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.556116312 (75h26m38.536468949s) [noise=true , address=145.79.227.32:4835 , piece_id=AFJ6KUBQP5LSAJUSUN7DOSNJ5TBWBBQRRYBTARKHKMOCWGDHDRJQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.556487637 (75h26m38.536840274s) [noise=true , address=145.79.227.22:4511 , piece_id=HE3JRGJIDNHPZFAXR2Y3HOXYZSCOKQ4OP6VPCLBLLZUFMBYU3WOA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.714141565 (75h26m38.694494204s) [noise=true , address=145.79.227.33:4871 , piece_id=HCUMHPIMTFBWYNYL5NHFR5AQSYSIDIN2D7EO7ZYHG5GUCAYMLAPQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:55:27.392456241 (75h27m7.372808888s) [noise=true , address=77.90.31.16:4735 , piece_id=P77WZF5A3CNM7ZXREICQQUGBDYJPHYQIQ4K23NKEXXBY4XFZV73Q, seg_pos={0 0}, upload=66264]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: expected piece hash; context deadline exceeded; EOF Dec 8 18:55:27.392541526 (75h27m7.372894167s) [seg_pos={0 0}, upload=66264]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 8 18:55:27.504922383 (75h27m7.485275021s) [upload=66264]: Done uploading segments... Dec 8 18:55:27.504964045 (75h27m7.485316691s) [upload=66264]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.402409389s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 2.898137075s trace: https://jaeger.dev.storj.io/trace/6086529041566513 FAIL (2025-12-08 19:00:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 8 18:54:57.674974846 (75h26m37.655327486s) [upload=66264]: Starting upload Dec 8 18:54:57.675038729 (75h26m37.655391369s) [upload=66264]: Done starting upload Dec 8 18:54:57.675330133 (75h26m37.655682761s) [upload=66264]: Uploading segments... Dec 8 18:54:57.675406307 (75h26m37.655758943s) [upload=66264]: Got next segment. Inline: false Dec 8 18:54:57.675485211 (75h26m37.655837857s) [upload=66264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 8 18:54:57.675493430 (75h26m37.655846073s) [upload=66264]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 8 18:54:57.785146518 (75h26m37.765499144s) [seg_pos={0 0}, upload=66264]: Begin upload segment... Dec 8 18:54:57.785240478 (75h26m37.765593118s) [seg_pos={0 0}, upload=66264]: Done begin upload segment. Dec 8 18:54:57.785485278 (75h26m37.765837924s) [noise=true , address=145.79.227.32:4835 , piece_id=AFJ6KUBQP5LSAJUSUN7DOSNJ5TBWBBQRRYBTARKHKMOCWGDHDRJQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.785679403 (75h26m37.766032052s) [noise=true , address=77.90.31.12:4590 , piece_id=P6H4KDKDRZZWGB3FVK4KB42KORBAHHHJES65VCTFD72D4JH5VJVA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.786641527 (75h26m37.766994177s) [noise=true , address=145.79.227.30:4763 , piece_id=5YDXU37WXFV36D2XH76XQ6BNY45SYZYTWGV3HBEFNVVT5LBW3NNQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787110826 (75h26m37.767463468s) [noise=true , address=77.90.31.16:4735 , piece_id=P77WZF5A3CNM7ZXREICQQUGBDYJPHYQIQ4K23NKEXXBY4XFZV73Q, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787261208 (75h26m37.767613854s) [noise=true , address=145.79.227.20:4439 , piece_id=HDPIFW4OJ6CY5LKLJ57IPHRPFDTXHVDTRJ7FKAXDY7PER6QPXQAA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787489023 (75h26m37.76784167s) [noise=true , address=77.90.31.11:4554 , piece_id=MJEVYVGP5NQZENUOE3NNBBS3BYC52642XB5OGF2K6J7YHNYORNPQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787843253 (75h26m37.768195901s) [noise=true , address=145.79.227.41:4085 , piece_id=HKKDK2YHRBBULRLBYAVC7QZQR2HGFH4DXZDJAS6EYR654U7SV2RA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787852122 (75h26m37.768204761s) [noise=true , address=145.79.227.25:4619 , piece_id=IPCP5WU2OE2JZDF47UKPU7GFPEIKLLBETHPV6UOQ7EYZG7QXUZJQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.788192899 (75h26m37.768545534s) [noise=true , address=145.79.227.21:4474 , piece_id=4PXKGNK6IBHWO7TKWG2O3BJI5P4MSYUWZD4NJ7JMOXP2TCY3VVWA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.788412033 (75h26m37.76876468s) [noise=true , address=145.79.227.27:4690 , piece_id=S6DIBWGQA3NAM3E3AS5QIB7QJL2SYSDFGANJYOZA3MTT6MQZS7AQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.788516029 (75h26m37.768868665s) [noise=true , address=145.79.227.22:4511 , piece_id=HE3JRGJIDNHPZFAXR2Y3HOXYZSCOKQ4OP6VPCLBLLZUFMBYU3WOA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.788933691 (75h26m37.769286334s) [noise=true , address=145.79.227.33:4871 , piece_id=HCUMHPIMTFBWYNYL5NHFR5AQSYSIDIN2D7EO7ZYHG5GUCAYMLAPQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.789013511 (75h26m37.769366149s) [noise=true , address=77.90.31.17:4772 , piece_id=Q4QNOKUPLIVMP7DVRAQSYKVCBIFYVVL7CYTVOKYYLQZSXD2JMRCQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.789551986 (75h26m37.769904633s) [noise=true , address=77.90.31.21:4878 , piece_id=7PHBVJ5HENMIOKLSRV34B6LOBJUGG6O3LUY4J3M6EFHV57JPXLSA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.790354423 (75h26m37.770707062s) [noise=true , address=77.90.31.12:4589 , piece_id=XA2OR2VJMLK6LGEABP7UBIQNZG6BXVMUX5XP7SVMLBEKJE5YDP7Q, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.790934325 (75h26m37.771286971s) [noise=true , address=145.79.227.39:4014 , piece_id=XPW63KAMFGYPJULSRSYPO3V5PIJX5DH7BLGFCDSYRSU46AXCYYRA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:58.172348027 (75h26m38.15270068s) [noise=true , address=77.90.31.12:4590 , piece_id=P6H4KDKDRZZWGB3FVK4KB42KORBAHHHJES65VCTFD72D4JH5VJVA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000 Dec 8 18:54:58.172831033 (75h26m38.153183679s) [noise=true , address=145.79.227.30:4763 , piece_id=5YDXU37WXFV36D2XH76XQ6BNY45SYZYTWGV3HBEFNVVT5LBW3NNQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: ecclient: upload failed (node:12Xy6Y953cfdoyywnf98XTrbd9WuDZNEV8SRhkBKbZRHrU2ymPg, address:145.79.227.30:4763): protocol: storage node overloaded, request limit: 2000 Dec 8 18:54:58.178590990 (75h26m38.158943635s) [noise=true , address=77.90.31.12:4589 , piece_id=XA2OR2VJMLK6LGEABP7UBIQNZG6BXVMUX5XP7SVMLBEKJE5YDP7Q, seg_pos={0 0}, upload=66264]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000 Dec 8 18:54:58.392481216 (75h26m38.372833849s) [upload=66264]: Next returned nil segment Dec 8 18:54:58.392499713 (75h26m38.372852355s) [upload=66264]: Waiting for error group managing segments... Dec 8 18:54:58.531890881 (75h26m38.512243518s) [noise=true , address=77.90.31.11:4554 , piece_id=MJEVYVGP5NQZENUOE3NNBBS3BYC52642XB5OGF2K6J7YHNYORNPQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.532848367 (75h26m38.513201014s) [noise=true , address=77.90.31.17:4772 , piece_id=Q4QNOKUPLIVMP7DVRAQSYKVCBIFYVVL7CYTVOKYYLQZSXD2JMRCQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.532895902 (75h26m38.513248545s) [noise=true , address=145.79.227.41:4085 , piece_id=HKKDK2YHRBBULRLBYAVC7QZQR2HGFH4DXZDJAS6EYR654U7SV2RA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.536318192 (75h26m38.516670836s) [noise=true , address=145.79.227.27:4690 , piece_id=S6DIBWGQA3NAM3E3AS5QIB7QJL2SYSDFGANJYOZA3MTT6MQZS7AQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.538479769 (75h26m38.518832418s) [noise=true , address=145.79.227.21:4474 , piece_id=4PXKGNK6IBHWO7TKWG2O3BJI5P4MSYUWZD4NJ7JMOXP2TCY3VVWA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.539308517 (75h26m38.519661165s) [noise=true , address=145.79.227.39:4014 , piece_id=XPW63KAMFGYPJULSRSYPO3V5PIJX5DH7BLGFCDSYRSU46AXCYYRA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.541778178 (75h26m38.522130824s) [noise=true , address=77.90.31.21:4878 , piece_id=7PHBVJ5HENMIOKLSRV34B6LOBJUGG6O3LUY4J3M6EFHV57JPXLSA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.553574259 (75h26m38.533926895s) [noise=true , address=145.79.227.25:4619 , piece_id=IPCP5WU2OE2JZDF47UKPU7GFPEIKLLBETHPV6UOQ7EYZG7QXUZJQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.555434734 (75h26m38.535787374s) [noise=true , address=145.79.227.20:4439 , piece_id=HDPIFW4OJ6CY5LKLJ57IPHRPFDTXHVDTRJ7FKAXDY7PER6QPXQAA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.556116312 (75h26m38.536468949s) [noise=true , address=145.79.227.32:4835 , piece_id=AFJ6KUBQP5LSAJUSUN7DOSNJ5TBWBBQRRYBTARKHKMOCWGDHDRJQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.556487637 (75h26m38.536840274s) [noise=true , address=145.79.227.22:4511 , piece_id=HE3JRGJIDNHPZFAXR2Y3HOXYZSCOKQ4OP6VPCLBLLZUFMBYU3WOA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.714141565 (75h26m38.694494204s) [noise=true , address=145.79.227.33:4871 , piece_id=HCUMHPIMTFBWYNYL5NHFR5AQSYSIDIN2D7EO7ZYHG5GUCAYMLAPQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:55:27.392456241 (75h27m7.372808888s) [noise=true , address=77.90.31.16:4735 , piece_id=P77WZF5A3CNM7ZXREICQQUGBDYJPHYQIQ4K23NKEXXBY4XFZV73Q, seg_pos={0 0}, upload=66264]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: expected piece hash; context deadline exceeded; EOF Dec 8 18:55:27.392541526 (75h27m7.372894167s) [seg_pos={0 0}, upload=66264]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 8 18:55:27.504922383 (75h27m7.485275021s) [upload=66264]: Done uploading segments... Dec 8 18:55:27.504964045 (75h27m7.485316691s) [upload=66264]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.898238634s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 2.8088949s trace: https://jaeger.dev.storj.io/trace/655d188af1aa270 FAIL (2025-12-08 19:00:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.809015399s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 2.829767865s trace: https://jaeger.dev.storj.io/trace/655d188af1aa270 FAIL (2025-12-08 18:59:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.829790184s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 2.304831991s trace: https://jaeger.dev.storj.io/trace/6086529041566513 FAIL (2025-12-08 18:59:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 8 18:54:57.674974846 (75h26m37.655327486s) [upload=66264]: Starting upload Dec 8 18:54:57.675038729 (75h26m37.655391369s) [upload=66264]: Done starting upload Dec 8 18:54:57.675330133 (75h26m37.655682761s) [upload=66264]: Uploading segments... Dec 8 18:54:57.675406307 (75h26m37.655758943s) [upload=66264]: Got next segment. Inline: false Dec 8 18:54:57.675485211 (75h26m37.655837857s) [upload=66264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 8 18:54:57.675493430 (75h26m37.655846073s) [upload=66264]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 8 18:54:57.785146518 (75h26m37.765499144s) [seg_pos={0 0}, upload=66264]: Begin upload segment... Dec 8 18:54:57.785240478 (75h26m37.765593118s) [seg_pos={0 0}, upload=66264]: Done begin upload segment. Dec 8 18:54:57.785485278 (75h26m37.765837924s) [noise=true , address=145.79.227.32:4835 , piece_id=AFJ6KUBQP5LSAJUSUN7DOSNJ5TBWBBQRRYBTARKHKMOCWGDHDRJQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.785679403 (75h26m37.766032052s) [noise=true , address=77.90.31.12:4590 , piece_id=P6H4KDKDRZZWGB3FVK4KB42KORBAHHHJES65VCTFD72D4JH5VJVA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.786641527 (75h26m37.766994177s) [noise=true , address=145.79.227.30:4763 , piece_id=5YDXU37WXFV36D2XH76XQ6BNY45SYZYTWGV3HBEFNVVT5LBW3NNQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787110826 (75h26m37.767463468s) [noise=true , address=77.90.31.16:4735 , piece_id=P77WZF5A3CNM7ZXREICQQUGBDYJPHYQIQ4K23NKEXXBY4XFZV73Q, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787261208 (75h26m37.767613854s) [noise=true , address=145.79.227.20:4439 , piece_id=HDPIFW4OJ6CY5LKLJ57IPHRPFDTXHVDTRJ7FKAXDY7PER6QPXQAA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787489023 (75h26m37.76784167s) [noise=true , address=77.90.31.11:4554 , piece_id=MJEVYVGP5NQZENUOE3NNBBS3BYC52642XB5OGF2K6J7YHNYORNPQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787843253 (75h26m37.768195901s) [noise=true , address=145.79.227.41:4085 , piece_id=HKKDK2YHRBBULRLBYAVC7QZQR2HGFH4DXZDJAS6EYR654U7SV2RA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787852122 (75h26m37.768204761s) [noise=true , address=145.79.227.25:4619 , piece_id=IPCP5WU2OE2JZDF47UKPU7GFPEIKLLBETHPV6UOQ7EYZG7QXUZJQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.788192899 (75h26m37.768545534s) [noise=true , address=145.79.227.21:4474 , piece_id=4PXKGNK6IBHWO7TKWG2O3BJI5P4MSYUWZD4NJ7JMOXP2TCY3VVWA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.788412033 (75h26m37.76876468s) [noise=true , address=145.79.227.27:4690 , piece_id=S6DIBWGQA3NAM3E3AS5QIB7QJL2SYSDFGANJYOZA3MTT6MQZS7AQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.788516029 (75h26m37.768868665s) [noise=true , address=145.79.227.22:4511 , piece_id=HE3JRGJIDNHPZFAXR2Y3HOXYZSCOKQ4OP6VPCLBLLZUFMBYU3WOA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.788933691 (75h26m37.769286334s) [noise=true , address=145.79.227.33:4871 , piece_id=HCUMHPIMTFBWYNYL5NHFR5AQSYSIDIN2D7EO7ZYHG5GUCAYMLAPQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.789013511 (75h26m37.769366149s) [noise=true , address=77.90.31.17:4772 , piece_id=Q4QNOKUPLIVMP7DVRAQSYKVCBIFYVVL7CYTVOKYYLQZSXD2JMRCQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.789551986 (75h26m37.769904633s) [noise=true , address=77.90.31.21:4878 , piece_id=7PHBVJ5HENMIOKLSRV34B6LOBJUGG6O3LUY4J3M6EFHV57JPXLSA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.790354423 (75h26m37.770707062s) [noise=true , address=77.90.31.12:4589 , piece_id=XA2OR2VJMLK6LGEABP7UBIQNZG6BXVMUX5XP7SVMLBEKJE5YDP7Q, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.790934325 (75h26m37.771286971s) [noise=true , address=145.79.227.39:4014 , piece_id=XPW63KAMFGYPJULSRSYPO3V5PIJX5DH7BLGFCDSYRSU46AXCYYRA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:58.172348027 (75h26m38.15270068s) [noise=true , address=77.90.31.12:4590 , piece_id=P6H4KDKDRZZWGB3FVK4KB42KORBAHHHJES65VCTFD72D4JH5VJVA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000 Dec 8 18:54:58.172831033 (75h26m38.153183679s) [noise=true , address=145.79.227.30:4763 , piece_id=5YDXU37WXFV36D2XH76XQ6BNY45SYZYTWGV3HBEFNVVT5LBW3NNQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: ecclient: upload failed (node:12Xy6Y953cfdoyywnf98XTrbd9WuDZNEV8SRhkBKbZRHrU2ymPg, address:145.79.227.30:4763): protocol: storage node overloaded, request limit: 2000 Dec 8 18:54:58.178590990 (75h26m38.158943635s) [noise=true , address=77.90.31.12:4589 , piece_id=XA2OR2VJMLK6LGEABP7UBIQNZG6BXVMUX5XP7SVMLBEKJE5YDP7Q, seg_pos={0 0}, upload=66264]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000 Dec 8 18:54:58.392481216 (75h26m38.372833849s) [upload=66264]: Next returned nil segment Dec 8 18:54:58.392499713 (75h26m38.372852355s) [upload=66264]: Waiting for error group managing segments... Dec 8 18:54:58.531890881 (75h26m38.512243518s) [noise=true , address=77.90.31.11:4554 , piece_id=MJEVYVGP5NQZENUOE3NNBBS3BYC52642XB5OGF2K6J7YHNYORNPQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.532848367 (75h26m38.513201014s) [noise=true , address=77.90.31.17:4772 , piece_id=Q4QNOKUPLIVMP7DVRAQSYKVCBIFYVVL7CYTVOKYYLQZSXD2JMRCQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.532895902 (75h26m38.513248545s) [noise=true , address=145.79.227.41:4085 , piece_id=HKKDK2YHRBBULRLBYAVC7QZQR2HGFH4DXZDJAS6EYR654U7SV2RA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.536318192 (75h26m38.516670836s) [noise=true , address=145.79.227.27:4690 , piece_id=S6DIBWGQA3NAM3E3AS5QIB7QJL2SYSDFGANJYOZA3MTT6MQZS7AQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.538479769 (75h26m38.518832418s) [noise=true , address=145.79.227.21:4474 , piece_id=4PXKGNK6IBHWO7TKWG2O3BJI5P4MSYUWZD4NJ7JMOXP2TCY3VVWA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.539308517 (75h26m38.519661165s) [noise=true , address=145.79.227.39:4014 , piece_id=XPW63KAMFGYPJULSRSYPO3V5PIJX5DH7BLGFCDSYRSU46AXCYYRA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.541778178 (75h26m38.522130824s) [noise=true , address=77.90.31.21:4878 , piece_id=7PHBVJ5HENMIOKLSRV34B6LOBJUGG6O3LUY4J3M6EFHV57JPXLSA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.553574259 (75h26m38.533926895s) [noise=true , address=145.79.227.25:4619 , piece_id=IPCP5WU2OE2JZDF47UKPU7GFPEIKLLBETHPV6UOQ7EYZG7QXUZJQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.555434734 (75h26m38.535787374s) [noise=true , address=145.79.227.20:4439 , piece_id=HDPIFW4OJ6CY5LKLJ57IPHRPFDTXHVDTRJ7FKAXDY7PER6QPXQAA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.556116312 (75h26m38.536468949s) [noise=true , address=145.79.227.32:4835 , piece_id=AFJ6KUBQP5LSAJUSUN7DOSNJ5TBWBBQRRYBTARKHKMOCWGDHDRJQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.556487637 (75h26m38.536840274s) [noise=true , address=145.79.227.22:4511 , piece_id=HE3JRGJIDNHPZFAXR2Y3HOXYZSCOKQ4OP6VPCLBLLZUFMBYU3WOA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.714141565 (75h26m38.694494204s) [noise=true , address=145.79.227.33:4871 , piece_id=HCUMHPIMTFBWYNYL5NHFR5AQSYSIDIN2D7EO7ZYHG5GUCAYMLAPQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:55:27.392456241 (75h27m7.372808888s) [noise=true , address=77.90.31.16:4735 , piece_id=P77WZF5A3CNM7ZXREICQQUGBDYJPHYQIQ4K23NKEXXBY4XFZV73Q, seg_pos={0 0}, upload=66264]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: expected piece hash; context deadline exceeded; EOF Dec 8 18:55:27.392541526 (75h27m7.372894167s) [seg_pos={0 0}, upload=66264]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 8 18:55:27.504922383 (75h27m7.485275021s) [upload=66264]: Done uploading segments... Dec 8 18:55:27.504964045 (75h27m7.485316691s) [upload=66264]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.304858731s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.001587281s trace: https://jaeger.dev.storj.io/trace/655d188af1aa270 FAIL (2025-12-08 18:59:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001611818s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 2.349970779s trace: https://jaeger.dev.storj.io/trace/6086529041566513 FAIL (2025-12-08 18:57:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 8 18:54:57.674974846 (75h26m37.655327486s) [upload=66264]: Starting upload Dec 8 18:54:57.675038729 (75h26m37.655391369s) [upload=66264]: Done starting upload Dec 8 18:54:57.675330133 (75h26m37.655682761s) [upload=66264]: Uploading segments... Dec 8 18:54:57.675406307 (75h26m37.655758943s) [upload=66264]: Got next segment. Inline: false Dec 8 18:54:57.675485211 (75h26m37.655837857s) [upload=66264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 8 18:54:57.675493430 (75h26m37.655846073s) [upload=66264]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 8 18:54:57.785146518 (75h26m37.765499144s) [seg_pos={0 0}, upload=66264]: Begin upload segment... Dec 8 18:54:57.785240478 (75h26m37.765593118s) [seg_pos={0 0}, upload=66264]: Done begin upload segment. Dec 8 18:54:57.785485278 (75h26m37.765837924s) [noise=true , address=145.79.227.32:4835 , piece_id=AFJ6KUBQP5LSAJUSUN7DOSNJ5TBWBBQRRYBTARKHKMOCWGDHDRJQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.785679403 (75h26m37.766032052s) [noise=true , address=77.90.31.12:4590 , piece_id=P6H4KDKDRZZWGB3FVK4KB42KORBAHHHJES65VCTFD72D4JH5VJVA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.786641527 (75h26m37.766994177s) [noise=true , address=145.79.227.30:4763 , piece_id=5YDXU37WXFV36D2XH76XQ6BNY45SYZYTWGV3HBEFNVVT5LBW3NNQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787110826 (75h26m37.767463468s) [noise=true , address=77.90.31.16:4735 , piece_id=P77WZF5A3CNM7ZXREICQQUGBDYJPHYQIQ4K23NKEXXBY4XFZV73Q, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787261208 (75h26m37.767613854s) [noise=true , address=145.79.227.20:4439 , piece_id=HDPIFW4OJ6CY5LKLJ57IPHRPFDTXHVDTRJ7FKAXDY7PER6QPXQAA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787489023 (75h26m37.76784167s) [noise=true , address=77.90.31.11:4554 , piece_id=MJEVYVGP5NQZENUOE3NNBBS3BYC52642XB5OGF2K6J7YHNYORNPQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787843253 (75h26m37.768195901s) [noise=true , address=145.79.227.41:4085 , piece_id=HKKDK2YHRBBULRLBYAVC7QZQR2HGFH4DXZDJAS6EYR654U7SV2RA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787852122 (75h26m37.768204761s) [noise=true , address=145.79.227.25:4619 , piece_id=IPCP5WU2OE2JZDF47UKPU7GFPEIKLLBETHPV6UOQ7EYZG7QXUZJQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.788192899 (75h26m37.768545534s) [noise=true , address=145.79.227.21:4474 , piece_id=4PXKGNK6IBHWO7TKWG2O3BJI5P4MSYUWZD4NJ7JMOXP2TCY3VVWA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.788412033 (75h26m37.76876468s) [noise=true , address=145.79.227.27:4690 , piece_id=S6DIBWGQA3NAM3E3AS5QIB7QJL2SYSDFGANJYOZA3MTT6MQZS7AQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.788516029 (75h26m37.768868665s) [noise=true , address=145.79.227.22:4511 , piece_id=HE3JRGJIDNHPZFAXR2Y3HOXYZSCOKQ4OP6VPCLBLLZUFMBYU3WOA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.788933691 (75h26m37.769286334s) [noise=true , address=145.79.227.33:4871 , piece_id=HCUMHPIMTFBWYNYL5NHFR5AQSYSIDIN2D7EO7ZYHG5GUCAYMLAPQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.789013511 (75h26m37.769366149s) [noise=true , address=77.90.31.17:4772 , piece_id=Q4QNOKUPLIVMP7DVRAQSYKVCBIFYVVL7CYTVOKYYLQZSXD2JMRCQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.789551986 (75h26m37.769904633s) [noise=true , address=77.90.31.21:4878 , piece_id=7PHBVJ5HENMIOKLSRV34B6LOBJUGG6O3LUY4J3M6EFHV57JPXLSA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.790354423 (75h26m37.770707062s) [noise=true , address=77.90.31.12:4589 , piece_id=XA2OR2VJMLK6LGEABP7UBIQNZG6BXVMUX5XP7SVMLBEKJE5YDP7Q, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.790934325 (75h26m37.771286971s) [noise=true , address=145.79.227.39:4014 , piece_id=XPW63KAMFGYPJULSRSYPO3V5PIJX5DH7BLGFCDSYRSU46AXCYYRA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:58.172348027 (75h26m38.15270068s) [noise=true , address=77.90.31.12:4590 , piece_id=P6H4KDKDRZZWGB3FVK4KB42KORBAHHHJES65VCTFD72D4JH5VJVA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000 Dec 8 18:54:58.172831033 (75h26m38.153183679s) [noise=true , address=145.79.227.30:4763 , piece_id=5YDXU37WXFV36D2XH76XQ6BNY45SYZYTWGV3HBEFNVVT5LBW3NNQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: ecclient: upload failed (node:12Xy6Y953cfdoyywnf98XTrbd9WuDZNEV8SRhkBKbZRHrU2ymPg, address:145.79.227.30:4763): protocol: storage node overloaded, request limit: 2000 Dec 8 18:54:58.178590990 (75h26m38.158943635s) [noise=true , address=77.90.31.12:4589 , piece_id=XA2OR2VJMLK6LGEABP7UBIQNZG6BXVMUX5XP7SVMLBEKJE5YDP7Q, seg_pos={0 0}, upload=66264]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000 Dec 8 18:54:58.392481216 (75h26m38.372833849s) [upload=66264]: Next returned nil segment Dec 8 18:54:58.392499713 (75h26m38.372852355s) [upload=66264]: Waiting for error group managing segments... Dec 8 18:54:58.531890881 (75h26m38.512243518s) [noise=true , address=77.90.31.11:4554 , piece_id=MJEVYVGP5NQZENUOE3NNBBS3BYC52642XB5OGF2K6J7YHNYORNPQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.532848367 (75h26m38.513201014s) [noise=true , address=77.90.31.17:4772 , piece_id=Q4QNOKUPLIVMP7DVRAQSYKVCBIFYVVL7CYTVOKYYLQZSXD2JMRCQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.532895902 (75h26m38.513248545s) [noise=true , address=145.79.227.41:4085 , piece_id=HKKDK2YHRBBULRLBYAVC7QZQR2HGFH4DXZDJAS6EYR654U7SV2RA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.536318192 (75h26m38.516670836s) [noise=true , address=145.79.227.27:4690 , piece_id=S6DIBWGQA3NAM3E3AS5QIB7QJL2SYSDFGANJYOZA3MTT6MQZS7AQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.538479769 (75h26m38.518832418s) [noise=true , address=145.79.227.21:4474 , piece_id=4PXKGNK6IBHWO7TKWG2O3BJI5P4MSYUWZD4NJ7JMOXP2TCY3VVWA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.539308517 (75h26m38.519661165s) [noise=true , address=145.79.227.39:4014 , piece_id=XPW63KAMFGYPJULSRSYPO3V5PIJX5DH7BLGFCDSYRSU46AXCYYRA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.541778178 (75h26m38.522130824s) [noise=true , address=77.90.31.21:4878 , piece_id=7PHBVJ5HENMIOKLSRV34B6LOBJUGG6O3LUY4J3M6EFHV57JPXLSA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.553574259 (75h26m38.533926895s) [noise=true , address=145.79.227.25:4619 , piece_id=IPCP5WU2OE2JZDF47UKPU7GFPEIKLLBETHPV6UOQ7EYZG7QXUZJQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.555434734 (75h26m38.535787374s) [noise=true , address=145.79.227.20:4439 , piece_id=HDPIFW4OJ6CY5LKLJ57IPHRPFDTXHVDTRJ7FKAXDY7PER6QPXQAA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.556116312 (75h26m38.536468949s) [noise=true , address=145.79.227.32:4835 , piece_id=AFJ6KUBQP5LSAJUSUN7DOSNJ5TBWBBQRRYBTARKHKMOCWGDHDRJQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.556487637 (75h26m38.536840274s) [noise=true , address=145.79.227.22:4511 , piece_id=HE3JRGJIDNHPZFAXR2Y3HOXYZSCOKQ4OP6VPCLBLLZUFMBYU3WOA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.714141565 (75h26m38.694494204s) [noise=true , address=145.79.227.33:4871 , piece_id=HCUMHPIMTFBWYNYL5NHFR5AQSYSIDIN2D7EO7ZYHG5GUCAYMLAPQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:55:27.392456241 (75h27m7.372808888s) [noise=true , address=77.90.31.16:4735 , piece_id=P77WZF5A3CNM7ZXREICQQUGBDYJPHYQIQ4K23NKEXXBY4XFZV73Q, seg_pos={0 0}, upload=66264]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: expected piece hash; context deadline exceeded; EOF Dec 8 18:55:27.392541526 (75h27m7.372894167s) [seg_pos={0 0}, upload=66264]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 8 18:55:27.504922383 (75h27m7.485275021s) [upload=66264]: Done uploading segments... Dec 8 18:55:27.504964045 (75h27m7.485316691s) [upload=66264]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.34999386s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 2.331322193s trace: https://jaeger.dev.storj.io/trace/6086529041566513 FAIL (2025-12-08 18:57:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 8 18:54:57.674974846 (75h26m37.655327486s) [upload=66264]: Starting upload Dec 8 18:54:57.675038729 (75h26m37.655391369s) [upload=66264]: Done starting upload Dec 8 18:54:57.675330133 (75h26m37.655682761s) [upload=66264]: Uploading segments... Dec 8 18:54:57.675406307 (75h26m37.655758943s) [upload=66264]: Got next segment. Inline: false Dec 8 18:54:57.675485211 (75h26m37.655837857s) [upload=66264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 8 18:54:57.675493430 (75h26m37.655846073s) [upload=66264]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 8 18:54:57.785146518 (75h26m37.765499144s) [seg_pos={0 0}, upload=66264]: Begin upload segment... Dec 8 18:54:57.785240478 (75h26m37.765593118s) [seg_pos={0 0}, upload=66264]: Done begin upload segment. Dec 8 18:54:57.785485278 (75h26m37.765837924s) [noise=true , address=145.79.227.32:4835 , piece_id=AFJ6KUBQP5LSAJUSUN7DOSNJ5TBWBBQRRYBTARKHKMOCWGDHDRJQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.785679403 (75h26m37.766032052s) [noise=true , address=77.90.31.12:4590 , piece_id=P6H4KDKDRZZWGB3FVK4KB42KORBAHHHJES65VCTFD72D4JH5VJVA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.786641527 (75h26m37.766994177s) [noise=true , address=145.79.227.30:4763 , piece_id=5YDXU37WXFV36D2XH76XQ6BNY45SYZYTWGV3HBEFNVVT5LBW3NNQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787110826 (75h26m37.767463468s) [noise=true , address=77.90.31.16:4735 , piece_id=P77WZF5A3CNM7ZXREICQQUGBDYJPHYQIQ4K23NKEXXBY4XFZV73Q, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787261208 (75h26m37.767613854s) [noise=true , address=145.79.227.20:4439 , piece_id=HDPIFW4OJ6CY5LKLJ57IPHRPFDTXHVDTRJ7FKAXDY7PER6QPXQAA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787489023 (75h26m37.76784167s) [noise=true , address=77.90.31.11:4554 , piece_id=MJEVYVGP5NQZENUOE3NNBBS3BYC52642XB5OGF2K6J7YHNYORNPQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787843253 (75h26m37.768195901s) [noise=true , address=145.79.227.41:4085 , piece_id=HKKDK2YHRBBULRLBYAVC7QZQR2HGFH4DXZDJAS6EYR654U7SV2RA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787852122 (75h26m37.768204761s) [noise=true , address=145.79.227.25:4619 , piece_id=IPCP5WU2OE2JZDF47UKPU7GFPEIKLLBETHPV6UOQ7EYZG7QXUZJQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.788192899 (75h26m37.768545534s) [noise=true , address=145.79.227.21:4474 , piece_id=4PXKGNK6IBHWO7TKWG2O3BJI5P4MSYUWZD4NJ7JMOXP2TCY3VVWA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.788412033 (75h26m37.76876468s) [noise=true , address=145.79.227.27:4690 , piece_id=S6DIBWGQA3NAM3E3AS5QIB7QJL2SYSDFGANJYOZA3MTT6MQZS7AQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.788516029 (75h26m37.768868665s) [noise=true , address=145.79.227.22:4511 , piece_id=HE3JRGJIDNHPZFAXR2Y3HOXYZSCOKQ4OP6VPCLBLLZUFMBYU3WOA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.788933691 (75h26m37.769286334s) [noise=true , address=145.79.227.33:4871 , piece_id=HCUMHPIMTFBWYNYL5NHFR5AQSYSIDIN2D7EO7ZYHG5GUCAYMLAPQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.789013511 (75h26m37.769366149s) [noise=true , address=77.90.31.17:4772 , piece_id=Q4QNOKUPLIVMP7DVRAQSYKVCBIFYVVL7CYTVOKYYLQZSXD2JMRCQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.789551986 (75h26m37.769904633s) [noise=true , address=77.90.31.21:4878 , piece_id=7PHBVJ5HENMIOKLSRV34B6LOBJUGG6O3LUY4J3M6EFHV57JPXLSA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.790354423 (75h26m37.770707062s) [noise=true , address=77.90.31.12:4589 , piece_id=XA2OR2VJMLK6LGEABP7UBIQNZG6BXVMUX5XP7SVMLBEKJE5YDP7Q, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.790934325 (75h26m37.771286971s) [noise=true , address=145.79.227.39:4014 , piece_id=XPW63KAMFGYPJULSRSYPO3V5PIJX5DH7BLGFCDSYRSU46AXCYYRA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:58.172348027 (75h26m38.15270068s) [noise=true , address=77.90.31.12:4590 , piece_id=P6H4KDKDRZZWGB3FVK4KB42KORBAHHHJES65VCTFD72D4JH5VJVA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000 Dec 8 18:54:58.172831033 (75h26m38.153183679s) [noise=true , address=145.79.227.30:4763 , piece_id=5YDXU37WXFV36D2XH76XQ6BNY45SYZYTWGV3HBEFNVVT5LBW3NNQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: ecclient: upload failed (node:12Xy6Y953cfdoyywnf98XTrbd9WuDZNEV8SRhkBKbZRHrU2ymPg, address:145.79.227.30:4763): protocol: storage node overloaded, request limit: 2000 Dec 8 18:54:58.178590990 (75h26m38.158943635s) [noise=true , address=77.90.31.12:4589 , piece_id=XA2OR2VJMLK6LGEABP7UBIQNZG6BXVMUX5XP7SVMLBEKJE5YDP7Q, seg_pos={0 0}, upload=66264]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000 Dec 8 18:54:58.392481216 (75h26m38.372833849s) [upload=66264]: Next returned nil segment Dec 8 18:54:58.392499713 (75h26m38.372852355s) [upload=66264]: Waiting for error group managing segments... Dec 8 18:54:58.531890881 (75h26m38.512243518s) [noise=true , address=77.90.31.11:4554 , piece_id=MJEVYVGP5NQZENUOE3NNBBS3BYC52642XB5OGF2K6J7YHNYORNPQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.532848367 (75h26m38.513201014s) [noise=true , address=77.90.31.17:4772 , piece_id=Q4QNOKUPLIVMP7DVRAQSYKVCBIFYVVL7CYTVOKYYLQZSXD2JMRCQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.532895902 (75h26m38.513248545s) [noise=true , address=145.79.227.41:4085 , piece_id=HKKDK2YHRBBULRLBYAVC7QZQR2HGFH4DXZDJAS6EYR654U7SV2RA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.536318192 (75h26m38.516670836s) [noise=true , address=145.79.227.27:4690 , piece_id=S6DIBWGQA3NAM3E3AS5QIB7QJL2SYSDFGANJYOZA3MTT6MQZS7AQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.538479769 (75h26m38.518832418s) [noise=true , address=145.79.227.21:4474 , piece_id=4PXKGNK6IBHWO7TKWG2O3BJI5P4MSYUWZD4NJ7JMOXP2TCY3VVWA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.539308517 (75h26m38.519661165s) [noise=true , address=145.79.227.39:4014 , piece_id=XPW63KAMFGYPJULSRSYPO3V5PIJX5DH7BLGFCDSYRSU46AXCYYRA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.541778178 (75h26m38.522130824s) [noise=true , address=77.90.31.21:4878 , piece_id=7PHBVJ5HENMIOKLSRV34B6LOBJUGG6O3LUY4J3M6EFHV57JPXLSA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.553574259 (75h26m38.533926895s) [noise=true , address=145.79.227.25:4619 , piece_id=IPCP5WU2OE2JZDF47UKPU7GFPEIKLLBETHPV6UOQ7EYZG7QXUZJQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.555434734 (75h26m38.535787374s) [noise=true , address=145.79.227.20:4439 , piece_id=HDPIFW4OJ6CY5LKLJ57IPHRPFDTXHVDTRJ7FKAXDY7PER6QPXQAA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.556116312 (75h26m38.536468949s) [noise=true , address=145.79.227.32:4835 , piece_id=AFJ6KUBQP5LSAJUSUN7DOSNJ5TBWBBQRRYBTARKHKMOCWGDHDRJQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.556487637 (75h26m38.536840274s) [noise=true , address=145.79.227.22:4511 , piece_id=HE3JRGJIDNHPZFAXR2Y3HOXYZSCOKQ4OP6VPCLBLLZUFMBYU3WOA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.714141565 (75h26m38.694494204s) [noise=true , address=145.79.227.33:4871 , piece_id=HCUMHPIMTFBWYNYL5NHFR5AQSYSIDIN2D7EO7ZYHG5GUCAYMLAPQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:55:27.392456241 (75h27m7.372808888s) [noise=true , address=77.90.31.16:4735 , piece_id=P77WZF5A3CNM7ZXREICQQUGBDYJPHYQIQ4K23NKEXXBY4XFZV73Q, seg_pos={0 0}, upload=66264]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: expected piece hash; context deadline exceeded; EOF Dec 8 18:55:27.392541526 (75h27m7.372894167s) [seg_pos={0 0}, upload=66264]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 8 18:55:27.504922383 (75h27m7.485275021s) [upload=66264]: Done uploading segments... Dec 8 18:55:27.504964045 (75h27m7.485316691s) [upload=66264]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.331344956s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 2.351403103s trace: https://jaeger.dev.storj.io/trace/6086529041566513 FAIL (2025-12-08 18:56:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 8 18:54:57.674974846 (75h26m37.655327486s) [upload=66264]: Starting upload Dec 8 18:54:57.675038729 (75h26m37.655391369s) [upload=66264]: Done starting upload Dec 8 18:54:57.675330133 (75h26m37.655682761s) [upload=66264]: Uploading segments... Dec 8 18:54:57.675406307 (75h26m37.655758943s) [upload=66264]: Got next segment. Inline: false Dec 8 18:54:57.675485211 (75h26m37.655837857s) [upload=66264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 8 18:54:57.675493430 (75h26m37.655846073s) [upload=66264]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 8 18:54:57.785146518 (75h26m37.765499144s) [seg_pos={0 0}, upload=66264]: Begin upload segment... Dec 8 18:54:57.785240478 (75h26m37.765593118s) [seg_pos={0 0}, upload=66264]: Done begin upload segment. Dec 8 18:54:57.785485278 (75h26m37.765837924s) [noise=true , address=145.79.227.32:4835 , piece_id=AFJ6KUBQP5LSAJUSUN7DOSNJ5TBWBBQRRYBTARKHKMOCWGDHDRJQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.785679403 (75h26m37.766032052s) [noise=true , address=77.90.31.12:4590 , piece_id=P6H4KDKDRZZWGB3FVK4KB42KORBAHHHJES65VCTFD72D4JH5VJVA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.786641527 (75h26m37.766994177s) [noise=true , address=145.79.227.30:4763 , piece_id=5YDXU37WXFV36D2XH76XQ6BNY45SYZYTWGV3HBEFNVVT5LBW3NNQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787110826 (75h26m37.767463468s) [noise=true , address=77.90.31.16:4735 , piece_id=P77WZF5A3CNM7ZXREICQQUGBDYJPHYQIQ4K23NKEXXBY4XFZV73Q, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787261208 (75h26m37.767613854s) [noise=true , address=145.79.227.20:4439 , piece_id=HDPIFW4OJ6CY5LKLJ57IPHRPFDTXHVDTRJ7FKAXDY7PER6QPXQAA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787489023 (75h26m37.76784167s) [noise=true , address=77.90.31.11:4554 , piece_id=MJEVYVGP5NQZENUOE3NNBBS3BYC52642XB5OGF2K6J7YHNYORNPQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787843253 (75h26m37.768195901s) [noise=true , address=145.79.227.41:4085 , piece_id=HKKDK2YHRBBULRLBYAVC7QZQR2HGFH4DXZDJAS6EYR654U7SV2RA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787852122 (75h26m37.768204761s) [noise=true , address=145.79.227.25:4619 , piece_id=IPCP5WU2OE2JZDF47UKPU7GFPEIKLLBETHPV6UOQ7EYZG7QXUZJQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.788192899 (75h26m37.768545534s) [noise=true , address=145.79.227.21:4474 , piece_id=4PXKGNK6IBHWO7TKWG2O3BJI5P4MSYUWZD4NJ7JMOXP2TCY3VVWA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.788412033 (75h26m37.76876468s) [noise=true , address=145.79.227.27:4690 , piece_id=S6DIBWGQA3NAM3E3AS5QIB7QJL2SYSDFGANJYOZA3MTT6MQZS7AQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.788516029 (75h26m37.768868665s) [noise=true , address=145.79.227.22:4511 , piece_id=HE3JRGJIDNHPZFAXR2Y3HOXYZSCOKQ4OP6VPCLBLLZUFMBYU3WOA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.788933691 (75h26m37.769286334s) [noise=true , address=145.79.227.33:4871 , piece_id=HCUMHPIMTFBWYNYL5NHFR5AQSYSIDIN2D7EO7ZYHG5GUCAYMLAPQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.789013511 (75h26m37.769366149s) [noise=true , address=77.90.31.17:4772 , piece_id=Q4QNOKUPLIVMP7DVRAQSYKVCBIFYVVL7CYTVOKYYLQZSXD2JMRCQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.789551986 (75h26m37.769904633s) [noise=true , address=77.90.31.21:4878 , piece_id=7PHBVJ5HENMIOKLSRV34B6LOBJUGG6O3LUY4J3M6EFHV57JPXLSA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.790354423 (75h26m37.770707062s) [noise=true , address=77.90.31.12:4589 , piece_id=XA2OR2VJMLK6LGEABP7UBIQNZG6BXVMUX5XP7SVMLBEKJE5YDP7Q, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.790934325 (75h26m37.771286971s) [noise=true , address=145.79.227.39:4014 , piece_id=XPW63KAMFGYPJULSRSYPO3V5PIJX5DH7BLGFCDSYRSU46AXCYYRA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:58.172348027 (75h26m38.15270068s) [noise=true , address=77.90.31.12:4590 , piece_id=P6H4KDKDRZZWGB3FVK4KB42KORBAHHHJES65VCTFD72D4JH5VJVA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000 Dec 8 18:54:58.172831033 (75h26m38.153183679s) [noise=true , address=145.79.227.30:4763 , piece_id=5YDXU37WXFV36D2XH76XQ6BNY45SYZYTWGV3HBEFNVVT5LBW3NNQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: ecclient: upload failed (node:12Xy6Y953cfdoyywnf98XTrbd9WuDZNEV8SRhkBKbZRHrU2ymPg, address:145.79.227.30:4763): protocol: storage node overloaded, request limit: 2000 Dec 8 18:54:58.178590990 (75h26m38.158943635s) [noise=true , address=77.90.31.12:4589 , piece_id=XA2OR2VJMLK6LGEABP7UBIQNZG6BXVMUX5XP7SVMLBEKJE5YDP7Q, seg_pos={0 0}, upload=66264]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000 Dec 8 18:54:58.392481216 (75h26m38.372833849s) [upload=66264]: Next returned nil segment Dec 8 18:54:58.392499713 (75h26m38.372852355s) [upload=66264]: Waiting for error group managing segments... Dec 8 18:54:58.531890881 (75h26m38.512243518s) [noise=true , address=77.90.31.11:4554 , piece_id=MJEVYVGP5NQZENUOE3NNBBS3BYC52642XB5OGF2K6J7YHNYORNPQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.532848367 (75h26m38.513201014s) [noise=true , address=77.90.31.17:4772 , piece_id=Q4QNOKUPLIVMP7DVRAQSYKVCBIFYVVL7CYTVOKYYLQZSXD2JMRCQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.532895902 (75h26m38.513248545s) [noise=true , address=145.79.227.41:4085 , piece_id=HKKDK2YHRBBULRLBYAVC7QZQR2HGFH4DXZDJAS6EYR654U7SV2RA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.536318192 (75h26m38.516670836s) [noise=true , address=145.79.227.27:4690 , piece_id=S6DIBWGQA3NAM3E3AS5QIB7QJL2SYSDFGANJYOZA3MTT6MQZS7AQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.538479769 (75h26m38.518832418s) [noise=true , address=145.79.227.21:4474 , piece_id=4PXKGNK6IBHWO7TKWG2O3BJI5P4MSYUWZD4NJ7JMOXP2TCY3VVWA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.539308517 (75h26m38.519661165s) [noise=true , address=145.79.227.39:4014 , piece_id=XPW63KAMFGYPJULSRSYPO3V5PIJX5DH7BLGFCDSYRSU46AXCYYRA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.541778178 (75h26m38.522130824s) [noise=true , address=77.90.31.21:4878 , piece_id=7PHBVJ5HENMIOKLSRV34B6LOBJUGG6O3LUY4J3M6EFHV57JPXLSA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.553574259 (75h26m38.533926895s) [noise=true , address=145.79.227.25:4619 , piece_id=IPCP5WU2OE2JZDF47UKPU7GFPEIKLLBETHPV6UOQ7EYZG7QXUZJQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.555434734 (75h26m38.535787374s) [noise=true , address=145.79.227.20:4439 , piece_id=HDPIFW4OJ6CY5LKLJ57IPHRPFDTXHVDTRJ7FKAXDY7PER6QPXQAA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.556116312 (75h26m38.536468949s) [noise=true , address=145.79.227.32:4835 , piece_id=AFJ6KUBQP5LSAJUSUN7DOSNJ5TBWBBQRRYBTARKHKMOCWGDHDRJQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.556487637 (75h26m38.536840274s) [noise=true , address=145.79.227.22:4511 , piece_id=HE3JRGJIDNHPZFAXR2Y3HOXYZSCOKQ4OP6VPCLBLLZUFMBYU3WOA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.714141565 (75h26m38.694494204s) [noise=true , address=145.79.227.33:4871 , piece_id=HCUMHPIMTFBWYNYL5NHFR5AQSYSIDIN2D7EO7ZYHG5GUCAYMLAPQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:55:27.392456241 (75h27m7.372808888s) [noise=true , address=77.90.31.16:4735 , piece_id=P77WZF5A3CNM7ZXREICQQUGBDYJPHYQIQ4K23NKEXXBY4XFZV73Q, seg_pos={0 0}, upload=66264]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: expected piece hash; context deadline exceeded; EOF Dec 8 18:55:27.392541526 (75h27m7.372894167s) [seg_pos={0 0}, upload=66264]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 8 18:55:27.504922383 (75h27m7.485275021s) [upload=66264]: Done uploading segments... Dec 8 18:55:27.504964045 (75h27m7.485316691s) [upload=66264]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.351423796s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 2.397819522s trace: https://jaeger.dev.storj.io/trace/6086529041566513 FAIL (2025-12-08 18:55:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 8 18:54:57.674974846 (75h26m37.655327486s) [upload=66264]: Starting upload Dec 8 18:54:57.675038729 (75h26m37.655391369s) [upload=66264]: Done starting upload Dec 8 18:54:57.675330133 (75h26m37.655682761s) [upload=66264]: Uploading segments... Dec 8 18:54:57.675406307 (75h26m37.655758943s) [upload=66264]: Got next segment. Inline: false Dec 8 18:54:57.675485211 (75h26m37.655837857s) [upload=66264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 8 18:54:57.675493430 (75h26m37.655846073s) [upload=66264]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 8 18:54:57.785146518 (75h26m37.765499144s) [seg_pos={0 0}, upload=66264]: Begin upload segment... Dec 8 18:54:57.785240478 (75h26m37.765593118s) [seg_pos={0 0}, upload=66264]: Done begin upload segment. Dec 8 18:54:57.785485278 (75h26m37.765837924s) [noise=true , address=145.79.227.32:4835 , piece_id=AFJ6KUBQP5LSAJUSUN7DOSNJ5TBWBBQRRYBTARKHKMOCWGDHDRJQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.785679403 (75h26m37.766032052s) [noise=true , address=77.90.31.12:4590 , piece_id=P6H4KDKDRZZWGB3FVK4KB42KORBAHHHJES65VCTFD72D4JH5VJVA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.786641527 (75h26m37.766994177s) [noise=true , address=145.79.227.30:4763 , piece_id=5YDXU37WXFV36D2XH76XQ6BNY45SYZYTWGV3HBEFNVVT5LBW3NNQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787110826 (75h26m37.767463468s) [noise=true , address=77.90.31.16:4735 , piece_id=P77WZF5A3CNM7ZXREICQQUGBDYJPHYQIQ4K23NKEXXBY4XFZV73Q, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787261208 (75h26m37.767613854s) [noise=true , address=145.79.227.20:4439 , piece_id=HDPIFW4OJ6CY5LKLJ57IPHRPFDTXHVDTRJ7FKAXDY7PER6QPXQAA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787489023 (75h26m37.76784167s) [noise=true , address=77.90.31.11:4554 , piece_id=MJEVYVGP5NQZENUOE3NNBBS3BYC52642XB5OGF2K6J7YHNYORNPQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787843253 (75h26m37.768195901s) [noise=true , address=145.79.227.41:4085 , piece_id=HKKDK2YHRBBULRLBYAVC7QZQR2HGFH4DXZDJAS6EYR654U7SV2RA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787852122 (75h26m37.768204761s) [noise=true , address=145.79.227.25:4619 , piece_id=IPCP5WU2OE2JZDF47UKPU7GFPEIKLLBETHPV6UOQ7EYZG7QXUZJQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.788192899 (75h26m37.768545534s) [noise=true , address=145.79.227.21:4474 , piece_id=4PXKGNK6IBHWO7TKWG2O3BJI5P4MSYUWZD4NJ7JMOXP2TCY3VVWA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.788412033 (75h26m37.76876468s) [noise=true , address=145.79.227.27:4690 , piece_id=S6DIBWGQA3NAM3E3AS5QIB7QJL2SYSDFGANJYOZA3MTT6MQZS7AQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.788516029 (75h26m37.768868665s) [noise=true , address=145.79.227.22:4511 , piece_id=HE3JRGJIDNHPZFAXR2Y3HOXYZSCOKQ4OP6VPCLBLLZUFMBYU3WOA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.788933691 (75h26m37.769286334s) [noise=true , address=145.79.227.33:4871 , piece_id=HCUMHPIMTFBWYNYL5NHFR5AQSYSIDIN2D7EO7ZYHG5GUCAYMLAPQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.789013511 (75h26m37.769366149s) [noise=true , address=77.90.31.17:4772 , piece_id=Q4QNOKUPLIVMP7DVRAQSYKVCBIFYVVL7CYTVOKYYLQZSXD2JMRCQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.789551986 (75h26m37.769904633s) [noise=true , address=77.90.31.21:4878 , piece_id=7PHBVJ5HENMIOKLSRV34B6LOBJUGG6O3LUY4J3M6EFHV57JPXLSA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.790354423 (75h26m37.770707062s) [noise=true , address=77.90.31.12:4589 , piece_id=XA2OR2VJMLK6LGEABP7UBIQNZG6BXVMUX5XP7SVMLBEKJE5YDP7Q, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.790934325 (75h26m37.771286971s) [noise=true , address=145.79.227.39:4014 , piece_id=XPW63KAMFGYPJULSRSYPO3V5PIJX5DH7BLGFCDSYRSU46AXCYYRA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:58.172348027 (75h26m38.15270068s) [noise=true , address=77.90.31.12:4590 , piece_id=P6H4KDKDRZZWGB3FVK4KB42KORBAHHHJES65VCTFD72D4JH5VJVA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000 Dec 8 18:54:58.172831033 (75h26m38.153183679s) [noise=true , address=145.79.227.30:4763 , piece_id=5YDXU37WXFV36D2XH76XQ6BNY45SYZYTWGV3HBEFNVVT5LBW3NNQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: ecclient: upload failed (node:12Xy6Y953cfdoyywnf98XTrbd9WuDZNEV8SRhkBKbZRHrU2ymPg, address:145.79.227.30:4763): protocol: storage node overloaded, request limit: 2000 Dec 8 18:54:58.178590990 (75h26m38.158943635s) [noise=true , address=77.90.31.12:4589 , piece_id=XA2OR2VJMLK6LGEABP7UBIQNZG6BXVMUX5XP7SVMLBEKJE5YDP7Q, seg_pos={0 0}, upload=66264]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000 Dec 8 18:54:58.392481216 (75h26m38.372833849s) [upload=66264]: Next returned nil segment Dec 8 18:54:58.392499713 (75h26m38.372852355s) [upload=66264]: Waiting for error group managing segments... Dec 8 18:54:58.531890881 (75h26m38.512243518s) [noise=true , address=77.90.31.11:4554 , piece_id=MJEVYVGP5NQZENUOE3NNBBS3BYC52642XB5OGF2K6J7YHNYORNPQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.532848367 (75h26m38.513201014s) [noise=true , address=77.90.31.17:4772 , piece_id=Q4QNOKUPLIVMP7DVRAQSYKVCBIFYVVL7CYTVOKYYLQZSXD2JMRCQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.532895902 (75h26m38.513248545s) [noise=true , address=145.79.227.41:4085 , piece_id=HKKDK2YHRBBULRLBYAVC7QZQR2HGFH4DXZDJAS6EYR654U7SV2RA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.536318192 (75h26m38.516670836s) [noise=true , address=145.79.227.27:4690 , piece_id=S6DIBWGQA3NAM3E3AS5QIB7QJL2SYSDFGANJYOZA3MTT6MQZS7AQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.538479769 (75h26m38.518832418s) [noise=true , address=145.79.227.21:4474 , piece_id=4PXKGNK6IBHWO7TKWG2O3BJI5P4MSYUWZD4NJ7JMOXP2TCY3VVWA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.539308517 (75h26m38.519661165s) [noise=true , address=145.79.227.39:4014 , piece_id=XPW63KAMFGYPJULSRSYPO3V5PIJX5DH7BLGFCDSYRSU46AXCYYRA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.541778178 (75h26m38.522130824s) [noise=true , address=77.90.31.21:4878 , piece_id=7PHBVJ5HENMIOKLSRV34B6LOBJUGG6O3LUY4J3M6EFHV57JPXLSA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.553574259 (75h26m38.533926895s) [noise=true , address=145.79.227.25:4619 , piece_id=IPCP5WU2OE2JZDF47UKPU7GFPEIKLLBETHPV6UOQ7EYZG7QXUZJQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.555434734 (75h26m38.535787374s) [noise=true , address=145.79.227.20:4439 , piece_id=HDPIFW4OJ6CY5LKLJ57IPHRPFDTXHVDTRJ7FKAXDY7PER6QPXQAA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.556116312 (75h26m38.536468949s) [noise=true , address=145.79.227.32:4835 , piece_id=AFJ6KUBQP5LSAJUSUN7DOSNJ5TBWBBQRRYBTARKHKMOCWGDHDRJQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.556487637 (75h26m38.536840274s) [noise=true , address=145.79.227.22:4511 , piece_id=HE3JRGJIDNHPZFAXR2Y3HOXYZSCOKQ4OP6VPCLBLLZUFMBYU3WOA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.714141565 (75h26m38.694494204s) [noise=true , address=145.79.227.33:4871 , piece_id=HCUMHPIMTFBWYNYL5NHFR5AQSYSIDIN2D7EO7ZYHG5GUCAYMLAPQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:55:27.392456241 (75h27m7.372808888s) [noise=true , address=77.90.31.16:4735 , piece_id=P77WZF5A3CNM7ZXREICQQUGBDYJPHYQIQ4K23NKEXXBY4XFZV73Q, seg_pos={0 0}, upload=66264]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: expected piece hash; context deadline exceeded; EOF Dec 8 18:55:27.392541526 (75h27m7.372894167s) [seg_pos={0 0}, upload=66264]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 8 18:55:27.504922383 (75h27m7.485275021s) [upload=66264]: Done uploading segments... Dec 8 18:55:27.504964045 (75h27m7.485316691s) [upload=66264]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.397849469s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.114388537s trace: https://jaeger.dev.storj.io/trace/6086529041566513 FAIL (2025-12-08 18:55:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 8 18:54:57.674974846 (75h26m37.655327486s) [upload=66264]: Starting upload Dec 8 18:54:57.675038729 (75h26m37.655391369s) [upload=66264]: Done starting upload Dec 8 18:54:57.675330133 (75h26m37.655682761s) [upload=66264]: Uploading segments... Dec 8 18:54:57.675406307 (75h26m37.655758943s) [upload=66264]: Got next segment. Inline: false Dec 8 18:54:57.675485211 (75h26m37.655837857s) [upload=66264]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 8 18:54:57.675493430 (75h26m37.655846073s) [upload=66264]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 8 18:54:57.785146518 (75h26m37.765499144s) [seg_pos={0 0}, upload=66264]: Begin upload segment... Dec 8 18:54:57.785240478 (75h26m37.765593118s) [seg_pos={0 0}, upload=66264]: Done begin upload segment. Dec 8 18:54:57.785485278 (75h26m37.765837924s) [noise=true , address=145.79.227.32:4835 , piece_id=AFJ6KUBQP5LSAJUSUN7DOSNJ5TBWBBQRRYBTARKHKMOCWGDHDRJQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.785679403 (75h26m37.766032052s) [noise=true , address=77.90.31.12:4590 , piece_id=P6H4KDKDRZZWGB3FVK4KB42KORBAHHHJES65VCTFD72D4JH5VJVA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.786641527 (75h26m37.766994177s) [noise=true , address=145.79.227.30:4763 , piece_id=5YDXU37WXFV36D2XH76XQ6BNY45SYZYTWGV3HBEFNVVT5LBW3NNQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787110826 (75h26m37.767463468s) [noise=true , address=77.90.31.16:4735 , piece_id=P77WZF5A3CNM7ZXREICQQUGBDYJPHYQIQ4K23NKEXXBY4XFZV73Q, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787261208 (75h26m37.767613854s) [noise=true , address=145.79.227.20:4439 , piece_id=HDPIFW4OJ6CY5LKLJ57IPHRPFDTXHVDTRJ7FKAXDY7PER6QPXQAA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787489023 (75h26m37.76784167s) [noise=true , address=77.90.31.11:4554 , piece_id=MJEVYVGP5NQZENUOE3NNBBS3BYC52642XB5OGF2K6J7YHNYORNPQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787843253 (75h26m37.768195901s) [noise=true , address=145.79.227.41:4085 , piece_id=HKKDK2YHRBBULRLBYAVC7QZQR2HGFH4DXZDJAS6EYR654U7SV2RA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.787852122 (75h26m37.768204761s) [noise=true , address=145.79.227.25:4619 , piece_id=IPCP5WU2OE2JZDF47UKPU7GFPEIKLLBETHPV6UOQ7EYZG7QXUZJQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.788192899 (75h26m37.768545534s) [noise=true , address=145.79.227.21:4474 , piece_id=4PXKGNK6IBHWO7TKWG2O3BJI5P4MSYUWZD4NJ7JMOXP2TCY3VVWA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.788412033 (75h26m37.76876468s) [noise=true , address=145.79.227.27:4690 , piece_id=S6DIBWGQA3NAM3E3AS5QIB7QJL2SYSDFGANJYOZA3MTT6MQZS7AQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.788516029 (75h26m37.768868665s) [noise=true , address=145.79.227.22:4511 , piece_id=HE3JRGJIDNHPZFAXR2Y3HOXYZSCOKQ4OP6VPCLBLLZUFMBYU3WOA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.788933691 (75h26m37.769286334s) [noise=true , address=145.79.227.33:4871 , piece_id=HCUMHPIMTFBWYNYL5NHFR5AQSYSIDIN2D7EO7ZYHG5GUCAYMLAPQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.789013511 (75h26m37.769366149s) [noise=true , address=77.90.31.17:4772 , piece_id=Q4QNOKUPLIVMP7DVRAQSYKVCBIFYVVL7CYTVOKYYLQZSXD2JMRCQ, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.789551986 (75h26m37.769904633s) [noise=true , address=77.90.31.21:4878 , piece_id=7PHBVJ5HENMIOKLSRV34B6LOBJUGG6O3LUY4J3M6EFHV57JPXLSA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.790354423 (75h26m37.770707062s) [noise=true , address=77.90.31.12:4589 , piece_id=XA2OR2VJMLK6LGEABP7UBIQNZG6BXVMUX5XP7SVMLBEKJE5YDP7Q, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:57.790934325 (75h26m37.771286971s) [noise=true , address=145.79.227.39:4014 , piece_id=XPW63KAMFGYPJULSRSYPO3V5PIJX5DH7BLGFCDSYRSU46AXCYYRA, seg_pos={0 0}, upload=66264]: Uploading piece... Dec 8 18:54:58.172348027 (75h26m38.15270068s) [noise=true , address=77.90.31.12:4590 , piece_id=P6H4KDKDRZZWGB3FVK4KB42KORBAHHHJES65VCTFD72D4JH5VJVA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000 Dec 8 18:54:58.172831033 (75h26m38.153183679s) [noise=true , address=145.79.227.30:4763 , piece_id=5YDXU37WXFV36D2XH76XQ6BNY45SYZYTWGV3HBEFNVVT5LBW3NNQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: ecclient: upload failed (node:12Xy6Y953cfdoyywnf98XTrbd9WuDZNEV8SRhkBKbZRHrU2ymPg, address:145.79.227.30:4763): protocol: storage node overloaded, request limit: 2000 Dec 8 18:54:58.178590990 (75h26m38.158943635s) [noise=true , address=77.90.31.12:4589 , piece_id=XA2OR2VJMLK6LGEABP7UBIQNZG6BXVMUX5XP7SVMLBEKJE5YDP7Q, seg_pos={0 0}, upload=66264]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000 Dec 8 18:54:58.392481216 (75h26m38.372833849s) [upload=66264]: Next returned nil segment Dec 8 18:54:58.392499713 (75h26m38.372852355s) [upload=66264]: Waiting for error group managing segments... Dec 8 18:54:58.531890881 (75h26m38.512243518s) [noise=true , address=77.90.31.11:4554 , piece_id=MJEVYVGP5NQZENUOE3NNBBS3BYC52642XB5OGF2K6J7YHNYORNPQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.532848367 (75h26m38.513201014s) [noise=true , address=77.90.31.17:4772 , piece_id=Q4QNOKUPLIVMP7DVRAQSYKVCBIFYVVL7CYTVOKYYLQZSXD2JMRCQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.532895902 (75h26m38.513248545s) [noise=true , address=145.79.227.41:4085 , piece_id=HKKDK2YHRBBULRLBYAVC7QZQR2HGFH4DXZDJAS6EYR654U7SV2RA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.536318192 (75h26m38.516670836s) [noise=true , address=145.79.227.27:4690 , piece_id=S6DIBWGQA3NAM3E3AS5QIB7QJL2SYSDFGANJYOZA3MTT6MQZS7AQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.538479769 (75h26m38.518832418s) [noise=true , address=145.79.227.21:4474 , piece_id=4PXKGNK6IBHWO7TKWG2O3BJI5P4MSYUWZD4NJ7JMOXP2TCY3VVWA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.539308517 (75h26m38.519661165s) [noise=true , address=145.79.227.39:4014 , piece_id=XPW63KAMFGYPJULSRSYPO3V5PIJX5DH7BLGFCDSYRSU46AXCYYRA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.541778178 (75h26m38.522130824s) [noise=true , address=77.90.31.21:4878 , piece_id=7PHBVJ5HENMIOKLSRV34B6LOBJUGG6O3LUY4J3M6EFHV57JPXLSA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.553574259 (75h26m38.533926895s) [noise=true , address=145.79.227.25:4619 , piece_id=IPCP5WU2OE2JZDF47UKPU7GFPEIKLLBETHPV6UOQ7EYZG7QXUZJQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.555434734 (75h26m38.535787374s) [noise=true , address=145.79.227.20:4439 , piece_id=HDPIFW4OJ6CY5LKLJ57IPHRPFDTXHVDTRJ7FKAXDY7PER6QPXQAA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.556116312 (75h26m38.536468949s) [noise=true , address=145.79.227.32:4835 , piece_id=AFJ6KUBQP5LSAJUSUN7DOSNJ5TBWBBQRRYBTARKHKMOCWGDHDRJQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.556487637 (75h26m38.536840274s) [noise=true , address=145.79.227.22:4511 , piece_id=HE3JRGJIDNHPZFAXR2Y3HOXYZSCOKQ4OP6VPCLBLLZUFMBYU3WOA, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:54:58.714141565 (75h26m38.694494204s) [noise=true , address=145.79.227.33:4871 , piece_id=HCUMHPIMTFBWYNYL5NHFR5AQSYSIDIN2D7EO7ZYHG5GUCAYMLAPQ, seg_pos={0 0}, upload=66264]: Done uploading piece. err: <nil> Dec 8 18:55:27.392456241 (75h27m7.372808888s) [noise=true , address=77.90.31.16:4735 , piece_id=P77WZF5A3CNM7ZXREICQQUGBDYJPHYQIQ4K23NKEXXBY4XFZV73Q, seg_pos={0 0}, upload=66264]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: expected piece hash; context deadline exceeded; EOF Dec 8 18:55:27.392541526 (75h27m7.372894167s) [seg_pos={0 0}, upload=66264]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 8 18:55:27.504922383 (75h27m7.485275021s) [upload=66264]: Done uploading segments... Dec 8 18:55:27.504964045 (75h27m7.485316691s) [upload=66264]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.114411228s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.498048439s trace: https://jaeger.dev.storj.io/trace/17a981e9c178eaf2 FAIL (2025-12-08 13:12:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.498095033s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 2.917855892s trace: https://jaeger.dev.storj.io/trace/17a981e9c178eaf2 FAIL (2025-12-08 13:11:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.917891091s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 2.639618068s trace: https://jaeger.dev.storj.io/trace/17a981e9c178eaf2 FAIL (2025-12-08 13:10:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.639646029s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 2.866549008s trace: https://jaeger.dev.storj.io/trace/17a981e9c178eaf2 FAIL (2025-12-08 13:10:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.866583299s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 2.955759366s trace: https://jaeger.dev.storj.io/trace/17a981e9c178eaf2 FAIL (2025-12-08 13:09:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.955782911s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 2.558166528s trace: https://jaeger.dev.storj.io/trace/17a981e9c178eaf2 FAIL (2025-12-08 13:08:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.558188912s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 2.681873286s trace: https://jaeger.dev.storj.io/trace/17a981e9c178eaf2 FAIL (2025-12-08 13:08:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.681898301s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 2.592751974s trace: https://jaeger.dev.storj.io/trace/17a981e9c178eaf2 FAIL (2025-12-08 13:07:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.592779406s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 2.72600525s trace: https://jaeger.dev.storj.io/trace/17a981e9c178eaf2 FAIL (2025-12-08 13:07:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.726036346s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 2.740883392s trace: https://jaeger.dev.storj.io/trace/17a981e9c178eaf2 FAIL (2025-12-08 13:06:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.74090494s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 2.640801487s trace: https://jaeger.dev.storj.io/trace/17a981e9c178eaf2 FAIL (2025-12-08 13:06:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.640841245s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 2.738664889s trace: https://jaeger.dev.storj.io/trace/17a981e9c178eaf2 FAIL (2025-12-08 13:04:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.738879307s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 2.568900329s trace: https://jaeger.dev.storj.io/trace/17a981e9c178eaf2 FAIL (2025-12-08 13:03:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.568927002s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 2.729463548s trace: https://jaeger.dev.storj.io/trace/17a981e9c178eaf2 FAIL (2025-12-08 13:03:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.72948457s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 2.625872217s trace: https://jaeger.dev.storj.io/trace/17a981e9c178eaf2 FAIL (2025-12-08 13:02:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.625897201s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 2.689249475s trace: https://jaeger.dev.storj.io/trace/17a981e9c178eaf2 FAIL (2025-12-08 13:01:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.689271637s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 2.950627853s trace: https://jaeger.dev.storj.io/trace/17a981e9c178eaf2 FAIL (2025-12-08 13:01:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.95066728s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 2.714762554s trace: https://jaeger.dev.storj.io/trace/17a981e9c178eaf2 FAIL (2025-12-08 12:59:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.714838826s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 2.864280259s trace: https://jaeger.dev.storj.io/trace/17a981e9c178eaf2 FAIL (2025-12-08 12:59:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.864309592s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 2.662063002s trace: https://jaeger.dev.storj.io/trace/17a981e9c178eaf2 FAIL (2025-12-08 12:58:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.662092097s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 2.857587891s trace: https://jaeger.dev.storj.io/trace/17a981e9c178eaf2 FAIL (2025-12-08 12:57:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.857691148s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 2.830769519s trace: https://jaeger.dev.storj.io/trace/17a981e9c178eaf2 FAIL (2025-12-08 12:56:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.830811444s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 2.768754092s trace: https://jaeger.dev.storj.io/trace/17a981e9c178eaf2 FAIL (2025-12-08 12:55:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.768777392s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 2.705684819s trace: https://jaeger.dev.storj.io/trace/17a981e9c178eaf2 FAIL (2025-12-08 12:54:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.705710779s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 2.658602211s trace: https://jaeger.dev.storj.io/trace/17a981e9c178eaf2 FAIL (2025-12-08 12:53:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.658631554s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 2.640625569s trace: https://jaeger.dev.storj.io/trace/17a981e9c178eaf2 FAIL (2025-12-08 12:53:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.640653778s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 2.883433318s trace: https://jaeger.dev.storj.io/trace/17a981e9c178eaf2 FAIL (2025-12-08 12:51:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.883457827s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 2.665054111s trace: https://jaeger.dev.storj.io/trace/17a981e9c178eaf2 FAIL (2025-12-08 12:50:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.665097082s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 2.946618925s trace: https://jaeger.dev.storj.io/trace/17a981e9c178eaf2 FAIL (2025-12-08 12:50:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.946719119s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 2.798257147s trace: https://jaeger.dev.storj.io/trace/17a981e9c178eaf2 FAIL (2025-12-08 12:49:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.798284429s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 2.552608386s trace: https://jaeger.dev.storj.io/trace/17a981e9c178eaf2 FAIL (2025-12-08 12:49:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.552634753s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 2.604752896s trace: https://jaeger.dev.storj.io/trace/17a981e9c178eaf2 FAIL (2025-12-08 12:48:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.604793743s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 2.636520924s trace: https://jaeger.dev.storj.io/trace/17a981e9c178eaf2 FAIL (2025-12-08 12:47:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.636633635s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 2.766272663s trace: https://jaeger.dev.storj.io/trace/17a981e9c178eaf2 FAIL (2025-12-08 12:46:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.766297981s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 2.667796236s trace: https://jaeger.dev.storj.io/trace/17a981e9c178eaf2 FAIL (2025-12-08 12:45:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.66781765s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)