Orbiter History

Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure

Orbiter Check History

Showing 50 of 354467 reports (Page 577 of 7090)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [.........................................X..X...X..X..X.....] duration: 17.263642951s trace: https://jaeger.dev.storj.io/trace/220541cf08cfd830 FAIL (2026-03-18 19:48:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== .BatchRequestItem_ObjectBegin Mar 18 19:45:25.326452964 (195h34m28.342122568s) [upload=187860]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 19:45:25.456683819 (195h34m28.472353428s) [seg_pos={0 0}, upload=187860]: Begin upload segment... Mar 18 19:45:25.456770594 (195h34m28.472440184s) [seg_pos={0 0}, upload=187860]: Done begin upload segment. Mar 18 19:45:25.456821582 (195h34m28.472491172s) [noise=true , address=145.79.227.36:4525 , piece_id=7FBEMPOKGXT2YB5Q6EEYJBFBX35RRCOOTQ2UETDSIVFW34CQZDVA, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.456867918 (195h34m28.472537526s) [noise=true , address=45.91.116.5:4324 , piece_id=ZFBWMYRD65NBX2FUNBIJF57VRTZOUJEPXOEWHWU63QWBATPWJCDQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.457527075 (195h34m28.47319668s) [noise=true , address=79.127.219.38:4827 , piece_id=2WC4XR64CXG7I7EPU6VFNYUAC76BJ3EKEPKSPRT4I55IYRFQVXLQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.458341322 (195h34m28.474010934s) [noise=true , address=79.127.205.233:4144 , piece_id=C3QUG3TRTWKVN3ZOCEWVNP74WE4QMYA65AHCAKEXOPBPKY4AUZDA, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.458481997 (195h34m28.4741516s) [noise=true , address=145.79.227.17:4993 , piece_id=AN3MJQVU5MVMQLL623PWSLWGH3N242LVPKFOG6AZAUJA6WGMHMFA, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.459053712 (195h34m28.474723318s) [noise=true , address=145.79.227.5:4238 , piece_id=PNXIWK2VZZSCR53VQOBOMP5Z4F5YYRIWWX4CNPOXNRJAB4RWKJLQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.459541420 (195h34m28.475211026s) [noise=true , address=79.127.205.235:4260 , piece_id=VRXKZO7BI77HQO7F33I3EHJZQOS3VCNGH4XZGZEEOHYTM4FOIIDQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.460127026 (195h34m28.475796632s) [noise=true , address=145.79.227.33:4887 , piece_id=GPFUU4XAWYSCN4AG3VV6AI6BWEGLSHJFN5VGTVFGB3PFHPXOGU5Q, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.460537480 (195h34m28.476207086s) [noise=true , address=145.79.227.20:4472 , piece_id=6PS3R5ANRNMQFWPH3F2MWMIVTP53URHAQ7E3NLX72P6U6PGCLQTQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.461390949 (195h34m28.477060561s) [noise=true , address=193.149.164.68:4039 , piece_id=FLJZMLJXBWFG7FOEHWTVDJU2NTW2GZPHNK5S2MIT5K2EBEMACKSQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.461470344 (195h34m28.477139959s) [noise=true , address=79.127.205.234:4206 , piece_id=FCSOPKSPRRYZ5E6RDK72WZVVAWTOD53MYSUST63NR52RY6QDHMKQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.462470296 (195h34m28.478139902s) [noise=true , address=145.79.227.11:4747 , piece_id=JTWQJUGUKHAVWGG6T2ZVJHZ4R5FZTTYFYKXXRD3MH4ZBLEOD4MRQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.462553305 (195h34m28.478222915s) [noise=true , address=77.90.31.17:4782 , piece_id=7ZJZOZOLIXGWYAGAV65WIEY76DOCM6CSE2HAUWXRPKHCLVOQQFEQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.463505826 (195h34m28.479175427s) [noise=true , address=77.90.31.10:4537 , piece_id=N4UDTJUBLM6NLN2NNE3YYKAYHMCNGQWLX4R6TZ35UPDONZ7NMIOA, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.464069344 (195h34m28.479738951s) [noise=true , address=167.17.72.69:4716 , piece_id=3ETXEX7UWPM4DU3DL2EGA5Z4QLKUKHAT23KG3PD5YL6XEZCJTGRA, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.465380624 (195h34m28.481050227s) [noise=true , address=145.79.227.2:4135 , piece_id=DHYVS2YDQW7FQANTEAPJKWBCNLUWPZNANCHFNDWLGFHPVOYPVTJQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.465888656 (195h34m28.481558265s) [noise=true , address=145.79.227.44:4206 , piece_id=L42NBORHYROKARBF6Z7CUJQ4SO5IA5SOBRA3TJWOYZDZKTSNXE4Q, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.466308049 (195h34m28.481977655s) [noise=true , address=79.127.219.46:4920 , piece_id=76VAJBNCJC4WAYXSEEQGZW7BEGGULKJFLV75UOBDBAVII7AKR62A, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:26.019613415 (195h34m29.035283015s) [upload=187860]: Next returned nil segment Mar 18 19:45:26.019627850 (195h34m29.035297451s) [upload=187860]: Waiting for error group managing segments... Mar 18 19:45:26.193532210 (195h34m29.209201815s) [noise=true , address=45.91.116.5:4324 , piece_id=ZFBWMYRD65NBX2FUNBIJF57VRTZOUJEPXOEWHWU63QWBATPWJCDQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.218509951 (195h34m29.234179558s) [noise=true , address=79.127.219.38:4827 , piece_id=2WC4XR64CXG7I7EPU6VFNYUAC76BJ3EKEPKSPRT4I55IYRFQVXLQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.219073762 (195h34m29.234743372s) [noise=true , address=77.90.31.10:4537 , piece_id=N4UDTJUBLM6NLN2NNE3YYKAYHMCNGQWLX4R6TZ35UPDONZ7NMIOA, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.271321232 (195h34m29.286990847s) [noise=true , address=79.127.219.46:4920 , piece_id=76VAJBNCJC4WAYXSEEQGZW7BEGGULKJFLV75UOBDBAVII7AKR62A, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.293176147 (195h34m29.308845754s) [noise=true , address=77.90.31.17:4782 , piece_id=7ZJZOZOLIXGWYAGAV65WIEY76DOCM6CSE2HAUWXRPKHCLVOQQFEQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.662025397 (195h34m29.677694999s) [noise=true , address=167.17.72.69:4716 , piece_id=3ETXEX7UWPM4DU3DL2EGA5Z4QLKUKHAT23KG3PD5YL6XEZCJTGRA, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.879968454 (195h34m29.895638055s) [noise=true , address=79.127.205.233:4144 , piece_id=C3QUG3TRTWKVN3ZOCEWVNP74WE4QMYA65AHCAKEXOPBPKY4AUZDA, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.898670905 (195h34m29.914340506s) [noise=true , address=79.127.205.235:4260 , piece_id=VRXKZO7BI77HQO7F33I3EHJZQOS3VCNGH4XZGZEEOHYTM4FOIIDQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.903771534 (195h34m29.919441134s) [noise=true , address=193.149.164.68:4039 , piece_id=FLJZMLJXBWFG7FOEHWTVDJU2NTW2GZPHNK5S2MIT5K2EBEMACKSQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.993657693 (195h34m30.009327292s) [noise=true , address=79.127.205.234:4206 , piece_id=FCSOPKSPRRYZ5E6RDK72WZVVAWTOD53MYSUST63NR52RY6QDHMKQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:46:25.039034405 (195h35m28.054704015s) [noise=true , address=145.79.227.2:4135 , piece_id=DHYVS2YDQW7FQANTEAPJKWBCNLUWPZNANCHFNDWLGFHPVOYPVTJQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:1uEBWRaMH4mLApUcgHyHQmGxxbyXnYUScg5Z2on9LHESYkqj3S, address:145.79.227.2:4135): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.039411284 (195h35m28.055080905s) [noise=true , address=145.79.227.44:4206 , piece_id=L42NBORHYROKARBF6Z7CUJQ4SO5IA5SOBRA3TJWOYZDZKTSNXE4Q, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:12Mp4ZzYgrP3ZSXDeHbSTXaAJMPgGKUqVjdNtyfeKE5qBMAzBke, address:145.79.227.44:4206): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.039509444 (195h35m28.055179057s) [noise=true , address=145.79.227.5:4238 , piece_id=PNXIWK2VZZSCR53VQOBOMP5Z4F5YYRIWWX4CNPOXNRJAB4RWKJLQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:122AVjaN3YaiABnUjuZwSb6apPDeszZAWvmzwtP47k1fxsKLD5V, address:145.79.227.5:4238): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.039966227 (195h35m28.055635833s) [noise=true , address=145.79.227.33:4887 , piece_id=GPFUU4XAWYSCN4AG3VV6AI6BWEGLSHJFN5VGTVFGB3PFHPXOGU5Q, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:1orvhuWfkUGCykWsS2E5sHtMcEW3fFcCNzTQ4xSxqdhUrBza8H, address:145.79.227.33:4887): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.040027384 (195h35m28.05569699s) [noise=true , address=145.79.227.17:4993 , piece_id=AN3MJQVU5MVMQLL623PWSLWGH3N242LVPKFOG6AZAUJA6WGMHMFA, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD, address:145.79.227.17:4993): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.040345443 (195h35m28.056015047s) [noise=true , address=145.79.227.20:4472 , piece_id=6PS3R5ANRNMQFWPH3F2MWMIVTP53URHAQ7E3NLX72P6U6PGCLQTQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:12GACnovbrme3fQknSDpZWVLq838VKop9mPjxnVXGdJ8KaqyaTi, address:145.79.227.20:4472): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.040426406 (195h35m28.056095999s) [noise=true , address=145.79.227.11:4747 , piece_id=JTWQJUGUKHAVWGG6T2ZVJHZ4R5FZTTYFYKXXRD3MH4ZBLEOD4MRQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:12mkHFDS6YCheghJdaJFybhcv9yhV4Acfs1RhnCW8yzA23neFRC, address:145.79.227.11:4747): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.040650249 (195h35m28.056319864s) [noise=true , address=145.79.227.36:4525 , piece_id=7FBEMPOKGXT2YB5Q6EEYJBFBX35RRCOOTQ2UETDSIVFW34CQZDVA, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX, address:145.79.227.36:4525): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.040699435 (195h35m28.056369021s) [seg_pos={0 0}, upload=187860]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Mar 18 19:46:25.162188571 (195h35m28.177858163s) [upload=187860]: Done uploading segments... Mar 18 19:46:25.162240944 (195h35m28.177910547s) [upload=187860]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration:* 17.263672487s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 19 out of 60 is failed (threshold=3) [.........................................XXXXXXXXXXXXXXXXXXX] duration: 30.107451654s trace: https://jaeger.dev.storj.io/trace/73e35414189be1d1 FAIL (2026-03-18 19:48:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== 18.935042183 (195h37m21.950711785s) [upload=187912]: Got next segment. Inline: false Mar 18 19:48:18.935110522 (195h37m21.950780124s) [upload=187912]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 19:48:18.935117422 (195h37m21.950787028s) [upload=187912]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 19:48:21.072012170 (195h37m24.087681772s) [seg_pos={0 0}, upload=187912]: Begin upload segment... Mar 18 19:48:21.072073682 (195h37m24.087743273s) [seg_pos={0 0}, upload=187912]: Done begin upload segment. Mar 18 19:48:21.072108678 (195h37m24.08777829s) [noise=true , address=145.79.227.12:4768 , piece_id=EF63QLD73PAILBK3ACHGZ3QBGSRFP4S3T2CIQUVUNHKS2BJX2R7Q, seg_pos={0 0}, upload=187912]: Uploading piece... Mar 18 19:48:21.072188274 (195h37m24.087857886s) [noise=true , address=145.79.227.38:4982 , piece_id=G3RQ3I7GPZ2RKEROW64BJO4GQEP5DE3CRN4EC4N4ZLIL2CZA5YWQ, seg_pos={0 0}, upload=187912]: Uploading piece... Mar 18 19:48:21.072727482 (195h37m24.088397091s) [noise=true , address=45.91.116.10:4083 , piece_id=KAVGSF45B2IAEPUOFQ76KZ7ZZOYBH7AI267FFY7OFR64CSV53YHQ, seg_pos={0 0}, upload=187912]: Uploading piece... Mar 18 19:48:21.073211763 (195h37m24.088881376s) [noise=true , address=145.79.227.11:4735 , piece_id=DW7D465QIBPBLBLNALZOE3NSFUYX65WU6WSCJNQZWD7MSM7XYPNA, seg_pos={0 0}, upload=187912]: Uploading piece... Mar 18 19:48:21.073857209 (195h37m24.089526819s) [noise=true , address=145.79.227.9:4369 , piece_id=ASIH3SYUHID6UEKAMHFM2OIQNYK5JC52FO2UKS6BCF7AOITUZEFA, seg_pos={0 0}, upload=187912]: Uploading piece... Mar 18 19:48:21.074443453 (195h37m24.090113062s) [noise=true , address=145.79.227.47:4265 , piece_id=4G7I5DNTFXHVFDJJIP5YAHNPDCTBUBAWPJ23HKCCVMPOCNVXFEXQ, seg_pos={0 0}, upload=187912]: Uploading piece... Mar 18 19:48:21.074934277 (195h37m24.090603886s) [noise=true , address=77.90.31.10:4519 , piece_id=ISFMUDRKU7QSPV3HMIYJNKJFNICOQFARHYHZTD2VSGEUXMAKITAA, seg_pos={0 0}, upload=187912]: Uploading piece... Mar 18 19:48:21.075463753 (195h37m24.091133368s) [noise=true , address=145.79.227.19:4082 , piece_id=EYZGBFP6D3S3HUHK37NCEZALJPMNVJV4FOTNPFXLCEUXC5CGOFOA, seg_pos={0 0}, upload=187912]: Uploading piece... Mar 18 19:48:21.076113206 (195h37m24.091782811s) [noise=true , address=145.79.227.40:4052 , piece_id=LR24R4XIZ7K7QPVSUK656OJW25GKBQSZTE4QANAEWNCWX76HBSGQ, seg_pos={0 0}, upload=187912]: Uploading piece... Mar 18 19:48:21.077990320 (195h37m24.093659928s) [noise=true , address=145.79.227.51:4409 , piece_id=B6AM5FGE3MYH57XQNMHOKAADWJ34U2RF5GQKJZHFDMS2N7STYOQA, seg_pos={0 0}, upload=187912]: Uploading piece... Mar 18 19:48:21.078256882 (195h37m24.093926491s) [noise=true , address=45.91.116.9:4264 , piece_id=F6MGTLZWURUMEVYJVPZZEVVNCRKQEPWNRCVQG34OLGM2TZAU7T7A, seg_pos={0 0}, upload=187912]: Uploading piece... Mar 18 19:48:21.080548426 (195h37m24.096218028s) [noise=true , address=45.91.116.13:4156 , piece_id=JNNYEBHOYF6FVP6QCCZ6QT7VZXP7QVPAWKIAZT5BF7IEF2BWJ3TA, seg_pos={0 0}, upload=187912]: Uploading piece... Mar 18 19:48:21.082521381 (195h37m24.098190989s) [noise=true , address=77.90.31.17:4770 , piece_id=FXL3QM4UXEZX2DCIZONRJW2NKTMZ5IKN3LKUG4G3BWGRYIOGAIPQ, seg_pos={0 0}, upload=187912]: Uploading piece... Mar 18 19:48:21.084314561 (195h37m24.099984175s) [noise=true , address=145.79.227.34:4908 , piece_id=C2O6M4QJ6MFRXRY2E327GJ5PTTIH2JOWYMHJN55QSWSXRSUP7HQA, seg_pos={0 0}, upload=187912]: Uploading piece... Mar 18 19:48:21.086259267 (195h37m24.101928875s) [noise=true , address=79.127.219.45:4858 , piece_id=MO326AJAN3UGJP7ILGUQLG6JOLULEPTPIS3ETXL3T3BG4ZZ4T4XA, seg_pos={0 0}, upload=187912]: Uploading piece... Mar 18 19:48:21.088117513 (195h37m24.103787124s) [noise=true , address=45.91.116.5:4298 , piece_id=GQFBQRT4FUEJBXRMX5URV7E2MYKWMZKXXCNDY534YHF2BNUVOPEA, seg_pos={0 0}, upload=187912]: Uploading piece... Mar 18 19:48:21.687895812 (195h37m24.703565399s) [upload=187912]: Next returned nil segment Mar 18 19:48:21.688000060 (195h37m24.703669661s) [upload=187912]: Waiting for error group managing segments... Mar 18 19:48:21.822569996 (195h37m24.838239599s) [noise=true , address=77.90.31.10:4519 , piece_id=ISFMUDRKU7QSPV3HMIYJNKJFNICOQFARHYHZTD2VSGEUXMAKITAA, seg_pos={0 0}, upload=187912]: Done uploading piece. err: <nil> Mar 18 19:48:21.826294318 (195h37m24.841963931s) [noise=true , address=45.91.116.13:4156 , piece_id=JNNYEBHOYF6FVP6QCCZ6QT7VZXP7QVPAWKIAZT5BF7IEF2BWJ3TA, seg_pos={0 0}, upload=187912]: Done uploading piece. err: <nil> Mar 18 19:48:21.828336670 (195h37m24.844006269s) [noise=true , address=45.91.116.9:4264 , piece_id=F6MGTLZWURUMEVYJVPZZEVVNCRKQEPWNRCVQG34OLGM2TZAU7T7A, seg_pos={0 0}, upload=187912]: Done uploading piece. err: <nil> Mar 18 19:48:21.835131342 (195h37m24.850800946s) [noise=true , address=45.91.116.5:4298 , piece_id=GQFBQRT4FUEJBXRMX5URV7E2MYKWMZKXXCNDY534YHF2BNUVOPEA, seg_pos={0 0}, upload=187912]: Done uploading piece. err: <nil> Mar 18 19:48:21.835219634 (195h37m24.850889245s) [noise=true , address=45.91.116.10:4083 , piece_id=KAVGSF45B2IAEPUOFQ76KZ7ZZOYBH7AI267FFY7OFR64CSV53YHQ, seg_pos={0 0}, upload=187912]: Done uploading piece. err: <nil> Mar 18 19:48:21.850875841 (195h37m24.866545449s) [noise=true , address=77.90.31.17:4770 , piece_id=FXL3QM4UXEZX2DCIZONRJW2NKTMZ5IKN3LKUG4G3BWGRYIOGAIPQ, seg_pos={0 0}, upload=187912]: Done uploading piece. err: <nil> Mar 18 19:48:21.878248250 (195h37m24.893917856s) [noise=true , address=79.127.219.45:4858 , piece_id=MO326AJAN3UGJP7ILGUQLG6JOLULEPTPIS3ETXL3T3BG4ZZ4T4XA, seg_pos={0 0}, upload=187912]: Done uploading piece. err: <nil> Mar 18 19:48:48.652377354 (195h37m51.668046961s) [noise=true , address=145.79.227.9:4369 , piece_id=ASIH3SYUHID6UEKAMHFM2OIQNYK5JC52FO2UKS6BCF7AOITUZEFA, seg_pos={0 0}, upload=187912]: Done uploading piece. err: ecclient: upload failed (node:1nwygPHyRJEZkjCoSZJTVhXDKLHc7AonVMzKMht6ppwZwhcGCT, address:145.79.227.9:4369): protocol: rpc: dial tcp 145.79.227.9:4369: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:48:48.652774297 (195h37m51.668443904s) [noise=true , address=145.79.227.12:4768 , piece_id=EF63QLD73PAILBK3ACHGZ3QBGSRFP4S3T2CIQUVUNHKS2BJX2R7Q, seg_pos={0 0}, upload=187912]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: rpc: dial tcp 145.79.227.12:4768: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:48:48.652921767 (195h37m51.668591376s) [noise=true , address=145.79.227.47:4265 , piece_id=4G7I5DNTFXHVFDJJIP5YAHNPDCTBUBAWPJ23HKCCVMPOCNVXFEXQ, seg_pos={0 0}, upload=187912]: Done uploading piece. err: ecclient: upload failed (node:12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ, address:145.79.227.47:4265): protocol: rpc: dial tcp 145.79.227.47:4265: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:48:48.653117397 (195h37m51.668787007s) [noise=true , address=145.79.227.19:4082 , piece_id=EYZGBFP6D3S3HUHK37NCEZALJPMNVJV4FOTNPFXLCEUXC5CGOFOA, seg_pos={0 0}, upload=187912]: Done uploading piece. err: ecclient: upload failed (node:12DKsNCtqxLvpJwLszdLs74V3eb8NH2VAV9V3bngzUczawmNoLK, address:145.79.227.19:4082): protocol: rpc: dial tcp 145.79.227.19:4082: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:48:48.653392347 (195h37m51.669061957s) [noise=true , address=145.79.227.38:4982 , piece_id=G3RQ3I7GPZ2RKEROW64BJO4GQEP5DE3CRN4EC4N4ZLIL2CZA5YWQ, seg_pos={0 0}, upload=187912]: Done uploading piece. err: ecclient: upload failed (node:1NxoYJXCXmTvQncbkXt6yH7mhEBBrzVdzhgL5J6W5TD4N3pkaG, address:145.79.227.38:4982): protocol: rpc: dial tcp 145.79.227.38:4982: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:48:48.653555352 (195h37m51.669224961s) [noise=true , address=145.79.227.51:4409 , piece_id=B6AM5FGE3MYH57XQNMHOKAADWJ34U2RF5GQKJZHFDMS2N7STYOQA, seg_pos={0 0}, upload=187912]: Done uploading piece. err: ecclient: upload failed (node:12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj, address:145.79.227.51:4409): protocol: rpc: dial tcp 145.79.227.51:4409: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:48:48.654266985 (195h37m51.669936591s) [noise=true , address=145.79.227.11:4735 , piece_id=DW7D465QIBPBLBLNALZOE3NSFUYX65WU6WSCJNQZWD7MSM7XYPNA, seg_pos={0 0}, upload=187912]: Done uploading piece. err: ecclient: upload failed (node:125ShGXckiJ3N5UWVmsMdhj98oFaRN7sg1KRucyUzqENW4Jp4kS, address:145.79.227.11:4735): protocol: rpc: dial tcp 145.79.227.11:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:48:48.654586887 (195h37m51.670256521s) [noise=true , address=145.79.227.40:4052 , piece_id=LR24R4XIZ7K7QPVSUK656OJW25GKBQSZTE4QANAEWNCWX76HBSGQ, seg_pos={0 0}, upload=187912]: Done uploading piece. err: ecclient: upload failed (node:12PViJBYAanwXRWhMq3WHR6k9Stxy8hjuY5FV9J7ErxCy4N1q3V, address:145.79.227.40:4052): protocol: rpc: dial tcp 145.79.227.40:4052: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:48:48.654841673 (195h37m51.670511289s) [noise=true , address=145.79.227.34:4908 , piece_id=C2O6M4QJ6MFRXRY2E327GJ5PTTIH2JOWYMHJN55QSWSXRSUP7HQA, seg_pos={0 0}, upload=187912]: Done uploading piece. err: ecclient: upload failed (node:1Qm73bcHkYSxHw4GGgnB8u1MPpCiQTVm9D35DFYrhrQf49M2y2, address:145.79.227.34:4908): protocol: rpc: dial tcp 145.79.227.34:4908: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:48:48.654883612 (195h37m51.670553218s) [seg_pos={0 0}, upload=187912]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Mar 18 19:48:48.758251263 (195h37m51.773920864s) [upload=187912]: Done uploading segments... Mar 18 19:48:48.758307932 (195h37m51.773977531s) [upload=187912]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 30.107470607s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 15 out of 60 is failed (threshold=6) [........................................XXXXXXXX....XX.XXXXX] duration: 5.001939816s trace: https://jaeger.dev.storj.io/trace/696bce57c561d71b FAIL (2026-03-18 19:48:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001961517s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [...........................................XXX.XXX..........] duration: 23.270572996s trace: https://jaeger.dev.storj.io/trace/4d6a8ab787f335c0 FAIL (2026-03-18 19:48:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 23.270592927s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [..........................................X..X...X..X..X....] duration: 21.508622285s trace: https://jaeger.dev.storj.io/trace/220541cf08cfd830 FAIL (2026-03-18 19:48:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== .BatchRequestItem_ObjectBegin Mar 18 19:45:25.326452964 (195h34m28.342122568s) [upload=187860]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 19:45:25.456683819 (195h34m28.472353428s) [seg_pos={0 0}, upload=187860]: Begin upload segment... Mar 18 19:45:25.456770594 (195h34m28.472440184s) [seg_pos={0 0}, upload=187860]: Done begin upload segment. Mar 18 19:45:25.456821582 (195h34m28.472491172s) [noise=true , address=145.79.227.36:4525 , piece_id=7FBEMPOKGXT2YB5Q6EEYJBFBX35RRCOOTQ2UETDSIVFW34CQZDVA, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.456867918 (195h34m28.472537526s) [noise=true , address=45.91.116.5:4324 , piece_id=ZFBWMYRD65NBX2FUNBIJF57VRTZOUJEPXOEWHWU63QWBATPWJCDQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.457527075 (195h34m28.47319668s) [noise=true , address=79.127.219.38:4827 , piece_id=2WC4XR64CXG7I7EPU6VFNYUAC76BJ3EKEPKSPRT4I55IYRFQVXLQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.458341322 (195h34m28.474010934s) [noise=true , address=79.127.205.233:4144 , piece_id=C3QUG3TRTWKVN3ZOCEWVNP74WE4QMYA65AHCAKEXOPBPKY4AUZDA, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.458481997 (195h34m28.4741516s) [noise=true , address=145.79.227.17:4993 , piece_id=AN3MJQVU5MVMQLL623PWSLWGH3N242LVPKFOG6AZAUJA6WGMHMFA, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.459053712 (195h34m28.474723318s) [noise=true , address=145.79.227.5:4238 , piece_id=PNXIWK2VZZSCR53VQOBOMP5Z4F5YYRIWWX4CNPOXNRJAB4RWKJLQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.459541420 (195h34m28.475211026s) [noise=true , address=79.127.205.235:4260 , piece_id=VRXKZO7BI77HQO7F33I3EHJZQOS3VCNGH4XZGZEEOHYTM4FOIIDQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.460127026 (195h34m28.475796632s) [noise=true , address=145.79.227.33:4887 , piece_id=GPFUU4XAWYSCN4AG3VV6AI6BWEGLSHJFN5VGTVFGB3PFHPXOGU5Q, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.460537480 (195h34m28.476207086s) [noise=true , address=145.79.227.20:4472 , piece_id=6PS3R5ANRNMQFWPH3F2MWMIVTP53URHAQ7E3NLX72P6U6PGCLQTQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.461390949 (195h34m28.477060561s) [noise=true , address=193.149.164.68:4039 , piece_id=FLJZMLJXBWFG7FOEHWTVDJU2NTW2GZPHNK5S2MIT5K2EBEMACKSQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.461470344 (195h34m28.477139959s) [noise=true , address=79.127.205.234:4206 , piece_id=FCSOPKSPRRYZ5E6RDK72WZVVAWTOD53MYSUST63NR52RY6QDHMKQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.462470296 (195h34m28.478139902s) [noise=true , address=145.79.227.11:4747 , piece_id=JTWQJUGUKHAVWGG6T2ZVJHZ4R5FZTTYFYKXXRD3MH4ZBLEOD4MRQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.462553305 (195h34m28.478222915s) [noise=true , address=77.90.31.17:4782 , piece_id=7ZJZOZOLIXGWYAGAV65WIEY76DOCM6CSE2HAUWXRPKHCLVOQQFEQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.463505826 (195h34m28.479175427s) [noise=true , address=77.90.31.10:4537 , piece_id=N4UDTJUBLM6NLN2NNE3YYKAYHMCNGQWLX4R6TZ35UPDONZ7NMIOA, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.464069344 (195h34m28.479738951s) [noise=true , address=167.17.72.69:4716 , piece_id=3ETXEX7UWPM4DU3DL2EGA5Z4QLKUKHAT23KG3PD5YL6XEZCJTGRA, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.465380624 (195h34m28.481050227s) [noise=true , address=145.79.227.2:4135 , piece_id=DHYVS2YDQW7FQANTEAPJKWBCNLUWPZNANCHFNDWLGFHPVOYPVTJQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.465888656 (195h34m28.481558265s) [noise=true , address=145.79.227.44:4206 , piece_id=L42NBORHYROKARBF6Z7CUJQ4SO5IA5SOBRA3TJWOYZDZKTSNXE4Q, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.466308049 (195h34m28.481977655s) [noise=true , address=79.127.219.46:4920 , piece_id=76VAJBNCJC4WAYXSEEQGZW7BEGGULKJFLV75UOBDBAVII7AKR62A, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:26.019613415 (195h34m29.035283015s) [upload=187860]: Next returned nil segment Mar 18 19:45:26.019627850 (195h34m29.035297451s) [upload=187860]: Waiting for error group managing segments... Mar 18 19:45:26.193532210 (195h34m29.209201815s) [noise=true , address=45.91.116.5:4324 , piece_id=ZFBWMYRD65NBX2FUNBIJF57VRTZOUJEPXOEWHWU63QWBATPWJCDQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.218509951 (195h34m29.234179558s) [noise=true , address=79.127.219.38:4827 , piece_id=2WC4XR64CXG7I7EPU6VFNYUAC76BJ3EKEPKSPRT4I55IYRFQVXLQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.219073762 (195h34m29.234743372s) [noise=true , address=77.90.31.10:4537 , piece_id=N4UDTJUBLM6NLN2NNE3YYKAYHMCNGQWLX4R6TZ35UPDONZ7NMIOA, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.271321232 (195h34m29.286990847s) [noise=true , address=79.127.219.46:4920 , piece_id=76VAJBNCJC4WAYXSEEQGZW7BEGGULKJFLV75UOBDBAVII7AKR62A, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.293176147 (195h34m29.308845754s) [noise=true , address=77.90.31.17:4782 , piece_id=7ZJZOZOLIXGWYAGAV65WIEY76DOCM6CSE2HAUWXRPKHCLVOQQFEQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.662025397 (195h34m29.677694999s) [noise=true , address=167.17.72.69:4716 , piece_id=3ETXEX7UWPM4DU3DL2EGA5Z4QLKUKHAT23KG3PD5YL6XEZCJTGRA, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.879968454 (195h34m29.895638055s) [noise=true , address=79.127.205.233:4144 , piece_id=C3QUG3TRTWKVN3ZOCEWVNP74WE4QMYA65AHCAKEXOPBPKY4AUZDA, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.898670905 (195h34m29.914340506s) [noise=true , address=79.127.205.235:4260 , piece_id=VRXKZO7BI77HQO7F33I3EHJZQOS3VCNGH4XZGZEEOHYTM4FOIIDQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.903771534 (195h34m29.919441134s) [noise=true , address=193.149.164.68:4039 , piece_id=FLJZMLJXBWFG7FOEHWTVDJU2NTW2GZPHNK5S2MIT5K2EBEMACKSQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.993657693 (195h34m30.009327292s) [noise=true , address=79.127.205.234:4206 , piece_id=FCSOPKSPRRYZ5E6RDK72WZVVAWTOD53MYSUST63NR52RY6QDHMKQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:46:25.039034405 (195h35m28.054704015s) [noise=true , address=145.79.227.2:4135 , piece_id=DHYVS2YDQW7FQANTEAPJKWBCNLUWPZNANCHFNDWLGFHPVOYPVTJQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:1uEBWRaMH4mLApUcgHyHQmGxxbyXnYUScg5Z2on9LHESYkqj3S, address:145.79.227.2:4135): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.039411284 (195h35m28.055080905s) [noise=true , address=145.79.227.44:4206 , piece_id=L42NBORHYROKARBF6Z7CUJQ4SO5IA5SOBRA3TJWOYZDZKTSNXE4Q, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:12Mp4ZzYgrP3ZSXDeHbSTXaAJMPgGKUqVjdNtyfeKE5qBMAzBke, address:145.79.227.44:4206): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.039509444 (195h35m28.055179057s) [noise=true , address=145.79.227.5:4238 , piece_id=PNXIWK2VZZSCR53VQOBOMP5Z4F5YYRIWWX4CNPOXNRJAB4RWKJLQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:122AVjaN3YaiABnUjuZwSb6apPDeszZAWvmzwtP47k1fxsKLD5V, address:145.79.227.5:4238): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.039966227 (195h35m28.055635833s) [noise=true , address=145.79.227.33:4887 , piece_id=GPFUU4XAWYSCN4AG3VV6AI6BWEGLSHJFN5VGTVFGB3PFHPXOGU5Q, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:1orvhuWfkUGCykWsS2E5sHtMcEW3fFcCNzTQ4xSxqdhUrBza8H, address:145.79.227.33:4887): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.040027384 (195h35m28.05569699s) [noise=true , address=145.79.227.17:4993 , piece_id=AN3MJQVU5MVMQLL623PWSLWGH3N242LVPKFOG6AZAUJA6WGMHMFA, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD, address:145.79.227.17:4993): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.040345443 (195h35m28.056015047s) [noise=true , address=145.79.227.20:4472 , piece_id=6PS3R5ANRNMQFWPH3F2MWMIVTP53URHAQ7E3NLX72P6U6PGCLQTQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:12GACnovbrme3fQknSDpZWVLq838VKop9mPjxnVXGdJ8KaqyaTi, address:145.79.227.20:4472): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.040426406 (195h35m28.056095999s) [noise=true , address=145.79.227.11:4747 , piece_id=JTWQJUGUKHAVWGG6T2ZVJHZ4R5FZTTYFYKXXRD3MH4ZBLEOD4MRQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:12mkHFDS6YCheghJdaJFybhcv9yhV4Acfs1RhnCW8yzA23neFRC, address:145.79.227.11:4747): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.040650249 (195h35m28.056319864s) [noise=true , address=145.79.227.36:4525 , piece_id=7FBEMPOKGXT2YB5Q6EEYJBFBX35RRCOOTQ2UETDSIVFW34CQZDVA, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX, address:145.79.227.36:4525): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.040699435 (195h35m28.056369021s) [seg_pos={0 0}, upload=187860]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Mar 18 19:46:25.162188571 (195h35m28.177858163s) [upload=187860]: Done uploading segments... Mar 18 19:46:25.162240944 (195h35m28.177910547s) [upload=187860]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration:* 21.508653207s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 18 out of 60 is failed (threshold=3) [..........................................XXXXXXXXXXXXXXXXXX] duration: 30.116033175s trace: https://jaeger.dev.storj.io/trace/9f9497bc930c8d7 FAIL (2026-03-18 19:48:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 1 (195h36m52.01699424s) [seg_pos={0 0}, upload=187902]: Done begin upload segment. Mar 18 19:47:49.001378985 (195h36m52.017048617s) [noise=true , address=145.79.227.50:4375 , piece_id=BIEY2OTEYATNIXO6DBVKDA437WIQRIQB4GUMZEQAY7PG3NRM4TFA, seg_pos={0 0}, upload=187902]: Uploading piece... Mar 18 19:47:49.001957623 (195h36m52.017627234s) [noise=true , address=145.79.227.22:4514 , piece_id=LBZYK3KSQ2BFK5D36AV7B4O7IIULDYVYAN6WOBXKRTM2HWEW3SAA, seg_pos={0 0}, upload=187902]: Uploading piece... Mar 18 19:47:49.003013817 (195h36m52.018683434s) [noise=true , address=145.79.227.21:4479 , piece_id=LJ75GSDWPQLPEAA2HGC3346633HSANU4KNBWSB2QKCR5TSQUZVQQ, seg_pos={0 0}, upload=187902]: Uploading piece... Mar 18 19:47:49.003757978 (195h36m52.019427602s) [noise=true , address=145.79.227.25:4622 , piece_id=GRHIE5XOJW4WUUDGMNL6YBO4AALHOXCXIYQ2MTBLOQUAPL2CFEZA, seg_pos={0 0}, upload=187902]: Uploading piece... Mar 18 19:47:49.004005677 (195h36m52.019675287s) [noise=true , address=79.127.219.38:4822 , piece_id=OU6GHLQWYQBAQTG3XB735CHS3BCOQTWRN36V3HC56A4A4QBVKTJA, seg_pos={0 0}, upload=187902]: Uploading piece... Mar 18 19:47:49.004617335 (195h36m52.020286948s) [noise=true , address=145.79.227.41:4089 , piece_id=6647NG2NCL6I2MYJQUEXHOZBHVNH3P32HXMETB4EQZW4WJ7FWGAQ, seg_pos={0 0}, upload=187902]: Uploading piece... Mar 18 19:47:49.005022393 (195h36m52.020691993s) [noise=true , address=77.90.31.18:4810 , piece_id=2INYBJDL2WUINKQV3DIRDVM2MXIE7BGT6YC3N7FPL57YVOCFCFIA, seg_pos={0 0}, upload=187902]: Uploading piece... Mar 18 19:47:49.005215037 (195h36m52.020884648s) [noise=true , address=77.90.31.11:4553 , piece_id=EAGRDIICW5N4MJGJWTEISNZQTEBEDI2XOWGQLQYNOW7S45CCHXNA, seg_pos={0 0}, upload=187902]: Uploading piece... Mar 18 19:47:49.005902587 (195h36m52.02157219s) [noise=true , address=145.79.227.46:4601 , piece_id=6AEFZZPMIO5XV7XVU6STNRYDDU7HM4P2YCY4RXG55SYTVXHRBTUA, seg_pos={0 0}, upload=187902]: Uploading piece... Mar 18 19:47:49.005954326 (195h36m52.021623939s) [noise=true , address=145.79.227.32:4837 , piece_id=J44JUEWDVY3Q6AGDU33NZJU2CS2YQHHGCFNEKJRAFKMRONECZXJA, seg_pos={0 0}, upload=187902]: Uploading piece... Mar 18 19:47:49.006576072 (195h36m52.022245686s) [noise=true , address=145.79.227.47:4265 , piece_id=NHD3LQXWP2OZZTA4FEATHUHO5KPFUWJE5C7GTH4FZBCMLYFMHAQA, seg_pos={0 0}, upload=187902]: Uploading piece... Mar 18 19:47:49.006586242 (195h36m52.022255849s) [noise=true , address=145.79.227.38:4979 , piece_id=DFA4FTYL3MTLNG5PUISFOKSPNNZOTUW6WFJDP3TJHSNRC5A3DXDA, seg_pos={0 0}, upload=187902]: Uploading piece... Mar 18 19:47:49.007203045 (195h36m52.022872655s) [noise=true , address=145.79.227.52:4448 , piece_id=V3SQ7O4DIUY3WLEBDB6SXRYKSLCJXU7LSIRIEK53IO7FDTPU5OFQ, seg_pos={0 0}, upload=187902]: Uploading piece... Mar 18 19:47:49.007237999 (195h36m52.022907602s) [noise=true , address=45.91.116.14:4193 , piece_id=6SJSRXDNXXDI2JGNQXULQA2LKJR3SRVKQPBZ7DERKWYDPUYWAWQQ, seg_pos={0 0}, upload=187902]: Uploading piece... Mar 18 19:47:49.007844701 (195h36m52.02351431s) [noise=true , address=145.79.227.44:4197 , piece_id=EZRJ7Y5DGBZZVZICZEVA2UW2IQ7GR36KXWQFU52H3ISR2RGCWLZA, seg_pos={0 0}, upload=187902]: Uploading piece... Mar 18 19:47:49.008479691 (195h36m52.024149303s) [noise=true , address=45.91.116.10:4081 , piece_id=TZHE5X2AV47BW32MXT33ON5K5Y7GE5ALDB3IUM3OX2SEXDOTQQHQ, seg_pos={0 0}, upload=187902]: Uploading piece... Mar 18 19:47:49.603060887 (195h36m52.61873049s) [upload=187902]: Next returned nil segment Mar 18 19:47:49.603077397 (195h36m52.618747006s) [upload=187902]: Waiting for error group managing segments... Mar 18 19:47:49.737366591 (195h36m52.753036191s) [noise=true , address=77.90.31.18:4810 , piece_id=2INYBJDL2WUINKQV3DIRDVM2MXIE7BGT6YC3N7FPL57YVOCFCFIA, seg_pos={0 0}, upload=187902]: Done uploading piece. err: <nil> Mar 18 19:47:49.759392242 (195h36m52.775061833s) [noise=true , address=45.91.116.10:4081 , piece_id=TZHE5X2AV47BW32MXT33ON5K5Y7GE5ALDB3IUM3OX2SEXDOTQQHQ, seg_pos={0 0}, upload=187902]: Done uploading piece. err: <nil> Mar 18 19:47:49.764454106 (195h36m52.780123706s) [noise=true , address=77.90.31.11:4553 , piece_id=EAGRDIICW5N4MJGJWTEISNZQTEBEDI2XOWGQLQYNOW7S45CCHXNA, seg_pos={0 0}, upload=187902]: Done uploading piece. err: <nil> Mar 18 19:47:49.766896106 (195h36m52.782565707s) [noise=true , address=45.91.116.14:4193 , piece_id=6SJSRXDNXXDI2JGNQXULQA2LKJR3SRVKQPBZ7DERKWYDPUYWAWQQ, seg_pos={0 0}, upload=187902]: Done uploading piece. err: <nil> Mar 18 19:47:49.815311372 (195h36m52.830980961s) [noise=true , address=79.127.219.38:4822 , piece_id=OU6GHLQWYQBAQTG3XB735CHS3BCOQTWRN36V3HC56A4A4QBVKTJA, seg_pos={0 0}, upload=187902]: Done uploading piece. err: <nil> Mar 18 19:48:18.383959405 (195h37m21.399629007s) [noise=true , address=145.79.227.25:4622 , piece_id=GRHIE5XOJW4WUUDGMNL6YBO4AALHOXCXIYQ2MTBLOQUAPL2CFEZA, seg_pos={0 0}, upload=187902]: Done uploading piece. err: ecclient: upload failed (node:1xEK5XL3sYcip7PsQcZBQYihgnEZykdMmsKkCr2TUmbTFYncfs, address:145.79.227.25:4622): protocol: rpc: dial tcp 145.79.227.25:4622: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:48:18.384225274 (195h37m21.39989488s) [noise=true , address=145.79.227.38:4979 , piece_id=DFA4FTYL3MTLNG5PUISFOKSPNNZOTUW6WFJDP3TJHSNRC5A3DXDA, seg_pos={0 0}, upload=187902]: Done uploading piece. err: ecclient: upload failed (node:12B518TvbbRhj6LiXxV4GwUgAoUYmWMWeTpyYerKnKuZYBjXoma, address:145.79.227.38:4979): protocol: rpc: dial tcp 145.79.227.38:4979: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:48:18.384472630 (195h37m21.400142241s) [noise=true , address=145.79.227.46:4601 , piece_id=6AEFZZPMIO5XV7XVU6STNRYDDU7HM4P2YCY4RXG55SYTVXHRBTUA, seg_pos={0 0}, upload=187902]: Done uploading piece. err: ecclient: upload failed (node:125GxLsL5UtXkuJ1LFvF42N9iZkKtRK3rq63xdR25JQbtwoT1L5, address:145.79.227.46:4601): protocol: rpc: dial tcp 145.79.227.46:4601: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:48:18.384720547 (195h37m21.400390168s) [noise=true , address=145.79.227.22:4514 , piece_id=LBZYK3KSQ2BFK5D36AV7B4O7IIULDYVYAN6WOBXKRTM2HWEW3SAA, seg_pos={0 0}, upload=187902]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: rpc: dial tcp 145.79.227.22:4514: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:48:18.384947198 (195h37m21.400616811s) [noise=true , address=145.79.227.21:4479 , piece_id=LJ75GSDWPQLPEAA2HGC3346633HSANU4KNBWSB2QKCR5TSQUZVQQ, seg_pos={0 0}, upload=187902]: Done uploading piece. err: ecclient: upload failed (node:12VCopFEoHk99GsQNQPuWvTQZRD1bpXi8HHLKXaXRdvFThTxmAk, address:145.79.227.21:4479): protocol: rpc: dial tcp 145.79.227.21:4479: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:48:18.386508651 (195h37m21.402178262s) [noise=true , address=145.79.227.41:4089 , piece_id=6647NG2NCL6I2MYJQUEXHOZBHVNH3P32HXMETB4EQZW4WJ7FWGAQ, seg_pos={0 0}, upload=187902]: Done uploading piece. err: ecclient: upload failed (node:12L2DTyJJ3v7KRjA424PVwQn9mpeq5Pq4ZTCerhrStgqgc3cVFk, address:145.79.227.41:4089): protocol: rpc: dial tcp 145.79.227.41:4089: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:48:18.386800636 (195h37m21.402470248s) [noise=true , address=145.79.227.52:4448 , piece_id=V3SQ7O4DIUY3WLEBDB6SXRYKSLCJXU7LSIRIEK53IO7FDTPU5OFQ, seg_pos={0 0}, upload=187902]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: rpc: dial tcp 145.79.227.52:4448: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:48:18.387036951 (195h37m21.40270654s) [noise=true , address=145.79.227.32:4837 , piece_id=J44JUEWDVY3Q6AGDU33NZJU2CS2YQHHGCFNEKJRAFKMRONECZXJA, seg_pos={0 0}, upload=187902]: Done uploading piece. err: ecclient: upload failed (node:1jmioLYNGZjDcByx8B1oHbqfid2DRqRfgoHhz85VcoTqbWz6Ls, address:145.79.227.32:4837): protocol: rpc: dial tcp 145.79.227.32:4837: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:48:18.387085247 (195h37m21.40275485s) [noise=true , address=145.79.227.50:4375 , piece_id=BIEY2OTEYATNIXO6DBVKDA437WIQRIQB4GUMZEQAY7PG3NRM4TFA, seg_pos={0 0}, upload=187902]: Done uploading piece. err: ecclient: upload failed (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5, address:145.79.227.50:4375): protocol: rpc: dial tcp 145.79.227.50:4375: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:48:18.387498598 (195h37m21.403168213s) [noise=true , address=145.79.227.47:4265 , piece_id=NHD3LQXWP2OZZTA4FEATHUHO5KPFUWJE5C7GTH4FZBCMLYFMHAQA, seg_pos={0 0}, upload=187902]: Done uploading piece. err: ecclient: upload failed (node:12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ, address:145.79.227.47:4265): protocol: rpc: dial tcp 145.79.227.47:4265: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:48:18.387546335 (195h37m21.403215945s) [noise=true , address=145.79.227.44:4197 , piece_id=EZRJ7Y5DGBZZVZICZEVA2UW2IQ7GR36KXWQFU52H3ISR2RGCWLZA, seg_pos={0 0}, upload=187902]: Done uploading piece. err: ecclient: upload failed (node:12niHLkT5pbmYSbarHZGeeCRtwjZRXtvEtHMGAb7ja4a7Xuuvfi, address:145.79.227.44:4197): protocol: rpc: dial tcp 145.79.227.44:4197: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:48:18.387588924 (195h37m21.403258511s) [seg_pos={0 0}, upload=187902]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Mar 18 19:48:18.498211220 (195h37m21.513880819s) [upload=187902]: Done uploading segments... Mar 18 19:48:18.498249951 (195h37m21.513919551s) [upload=187902]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 30.116056892s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [...........................................XXXXXXXXXXXXXXXXX] duration: 30.206558044s trace: https://jaeger.dev.storj.io/trace/42f56577b302d7c7 FAIL (2026-03-18 19:47:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Mar 18 19:47:18.076167307 (195h36m21.091836921s) [upload=187893]: Starting upload Mar 18 19:47:18.076266830 (195h36m21.091936442s) [upload=187893]: Done starting upload Mar 18 19:47:18.076319602 (195h36m21.091989206s) [upload=187893]: Uploading segments... Mar 18 19:47:18.076730235 (195h36m21.092399838s) [upload=187893]: Got next segment. Inline: false Mar 18 19:47:18.076813904 (195h36m21.092483509s) [upload=187893]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 19:47:18.076820681 (195h36m21.092490288s) [upload=187893]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 19:47:18.186239631 (195h36m21.201909233s) [seg_pos={0 0}, upload=187893]: Begin upload segment... Mar 18 19:47:18.186306305 (195h36m21.201975897s) [seg_pos={0 0}, upload=187893]: Done begin upload segment. Mar 18 19:47:18.186338657 (195h36m21.202008257s) [noise=true , address=45.91.116.8:4070 , piece_id=GKGNZK2SVCXDW7KXNF3GAGRSHJQFDIG4S32PP3BLXTORUF5O3MJA, seg_pos={0 0}, upload=187893]: Uploading piece... Mar 18 19:47:18.186460472 (195h36m21.202130076s) [noise=true , address=145.79.227.33:4873 , piece_id=3KNKPDTQQB57PFD2RSLZW4UUZDFY3QCHCBHNU43MSVQWI7BBYCZQ, seg_pos={0 0}, upload=187893]: Uploading piece... Mar 18 19:47:18.187200316 (195h36m21.202869923s) [noise=true , address=77.90.31.11:4555 , piece_id=EU372FZXOXNPADKBAJ5YEO7BBJ7U453ZFATCH77AKKVVLAQZBYFA, seg_pos={0 0}, upload=187893]: Uploading piece... Mar 18 19:47:18.188325817 (195h36m21.203995439s) [noise=true , address=45.91.116.14:4224 , piece_id=S2FGVIZNTI6EPMBLITVGG2OI4TFYA2ANIMAUA7U52E4PVFKGXUCQ, seg_pos={0 0}, upload=187893]: Uploading piece... Mar 18 19:47:18.188623611 (195h36m21.204293217s) [noise=true , address=145.79.227.2:4119 , piece_id=MA6YWCKWF7742D3LBOWXMDF3EW6EJIMH3V6N77WUUNROBVH4QT6A, seg_pos={0 0}, upload=187893]: Uploading piece... Mar 18 19:47:18.188811761 (195h36m21.204481368s) [noise=true , address=45.91.116.6:4976 , piece_id=ZI422NPEJSBPJKHSYRKER5R6YRWM4XKH4KVCQNIPYL7V4PBZICTQ, seg_pos={0 0}, upload=187893]: Uploading piece... Mar 18 19:47:18.189015808 (195h36m21.204685411s) [noise=true , address=145.79.227.14:4843 , piece_id=XKMKP63LH55GKKA3VEEOW4TM34XRJAUZXECYUWPOUMCHLD74RGCA, seg_pos={0 0}, upload=187893]: Uploading piece... Mar 18 19:47:18.189176982 (195h36m21.204846582s) [noise=true , address=145.79.227.5:4226 , piece_id=CL2GAZ3ZYKEH3FCOV3IUX576RPZSSSUMZDNFIEAPIUAQ57NRGEMQ, seg_pos={0 0}, upload=187893]: Uploading piece... Mar 18 19:47:18.189348652 (195h36m21.205018255s) [noise=true , address=145.79.227.22:4515 , piece_id=OQLP56N5QGMKJ2G57LTYJVWLX2WWQEZVZXNINNERXPIQIRYDC2TQ, seg_pos={0 0}, upload=187893]: Uploading piece... Mar 18 19:47:18.189571052 (195h36m21.205240652s) [noise=true , address=79.127.219.38:4821 , piece_id=TI6V7SE37USQL2PKTO3V5HGGLKGVSAH2RZOCTCMYDCELGFUMRK4Q, seg_pos={0 0}, upload=187893]: Uploading piece... Mar 18 19:47:18.189848518 (195h36m21.205518119s) [noise=true , address=45.91.116.5:4298 , piece_id=OXCA6SZ4QZ5LT7GSRPQIKTPUHEAUBVWFVGQL6YLCRLNR4TQ5WZCQ, seg_pos={0 0}, upload=187893]: Uploading piece... Mar 18 19:47:18.190126391 (195h36m21.205795995s) [noise=true , address=79.127.219.46:4892 , piece_id=XVPZ6U6ZNSLKZK626A62WQVLVXSXZKDG6NKQERNJZWMHN3W4KTFQ, seg_pos={0 0}, upload=187893]: Uploading piece... Mar 18 19:47:18.190285236 (195h36m21.205954836s) [noise=true , address=77.90.31.14:4663 , piece_id=OP5DEHULJXZSZHPYTBSPSXEAFES7J7GLNQX3E32CMI6R3LPYF7HA, seg_pos={0 0}, upload=187893]: Uploading piece... Mar 18 19:47:18.190721140 (195h36m21.206390741s) [noise=true , address=145.79.227.31:4802 , piece_id=4ABTXNYMEHQUAN5EKPWCVVUJ5DS5ALJGCUBUAPAOFNA3AUUVPLDQ, seg_pos={0 0}, upload=187893]: Uploading piece... Mar 18 19:47:18.194630711 (195h36m21.210300323s) [noise=true , address=145.79.227.20:4441 , piece_id=IPFJLPMA2RQX3S2MOH5HOWYKQIHT7PD2U4DXGBUFOQXZQERJZ3DQ, seg_pos={0 0}, upload=187893]: Uploading piece... Mar 18 19:47:18.195176047 (195h36m21.210845647s) [noise=true , address=77.90.31.18:4805 , piece_id=FDCR6TKNGB6YKSDQHEVHLNJISPCA2NXKJONNSKW5EDGYLDYQJIXQ, seg_pos={0 0}, upload=187893]: Uploading piece... Mar 18 19:47:18.831946219 (195h36m21.847615824s) [upload=187893]: Next returned nil segment Mar 18 19:47:18.831962582 (195h36m21.84763219s) [upload=187893]: Waiting for error group managing segments... Mar 18 19:47:18.931952032 (195h36m21.947621635s) [noise=true , address=77.90.31.11:4555 , piece_id=EU372FZXOXNPADKBAJ5YEO7BBJ7U453ZFATCH77AKKVVLAQZBYFA, seg_pos={0 0}, upload=187893]: Done uploading piece. err: <nil> Mar 18 19:47:18.980325702 (195h36m21.995995303s) [noise=true , address=79.127.219.46:4892 , piece_id=XVPZ6U6ZNSLKZK626A62WQVLVXSXZKDG6NKQERNJZWMHN3W4KTFQ, seg_pos={0 0}, upload=187893]: Done uploading piece. err: <nil> Mar 18 19:47:18.985964844 (195h36m22.001634444s) [noise=true , address=45.91.116.14:4224 , piece_id=S2FGVIZNTI6EPMBLITVGG2OI4TFYA2ANIMAUA7U52E4PVFKGXUCQ, seg_pos={0 0}, upload=187893]: Done uploading piece. err: <nil> Mar 18 19:47:18.986419629 (195h36m22.002089239s) [noise=true , address=45.91.116.6:4976 , piece_id=ZI422NPEJSBPJKHSYRKER5R6YRWM4XKH4KVCQNIPYL7V4PBZICTQ, seg_pos={0 0}, upload=187893]: Done uploading piece. err: <nil> Mar 18 19:47:18.996637190 (195h36m22.012306794s) [noise=true , address=45.91.116.8:4070 , piece_id=GKGNZK2SVCXDW7KXNF3GAGRSHJQFDIG4S32PP3BLXTORUF5O3MJA, seg_pos={0 0}, upload=187893]: Done uploading piece. err: <nil> Mar 18 19:47:18.997187065 (195h36m22.012856676s) [noise=true , address=45.91.116.5:4298 , piece_id=OXCA6SZ4QZ5LT7GSRPQIKTPUHEAUBVWFVGQL6YLCRLNR4TQ5WZCQ, seg_pos={0 0}, upload=187893]: Done uploading piece. err: <nil> Mar 18 19:47:19.002145224 (195h36m22.017814845s) [noise=true , address=77.90.31.18:4805 , piece_id=FDCR6TKNGB6YKSDQHEVHLNJISPCA2NXKJONNSKW5EDGYLDYQJIXQ, seg_pos={0 0}, upload=187893]: Done uploading piece. err: <nil> Mar 18 19:47:19.029478151 (195h36m22.045147751s) [noise=true , address=77.90.31.14:4663 , piece_id=OP5DEHULJXZSZHPYTBSPSXEAFES7J7GLNQX3E32CMI6R3LPYF7HA, seg_pos={0 0}, upload=187893]: Done uploading piece. err: <nil> Mar 18 19:47:19.040966239 (195h36m22.05663584s) [noise=true , address=79.127.219.38:4821 , piece_id=TI6V7SE37USQL2PKTO3V5HGGLKGVSAH2RZOCTCMYDCELGFUMRK4Q, seg_pos={0 0}, upload=187893]: Done uploading piece. err: <nil> Mar 18 19:47:47.793993003 (195h36m50.809662612s) [noise=true , address=145.79.227.20:4441 , piece_id=IPFJLPMA2RQX3S2MOH5HOWYKQIHT7PD2U4DXGBUFOQXZQERJZ3DQ, seg_pos={0 0}, upload=187893]: Done uploading piece. err: ecclient: upload failed (node:12LjGecRNJE1kq67iMtxZFhWeaUZpUogsGxbz4VXwnHdQtTnbBr, address:145.79.227.20:4441): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:47:47.794327050 (195h36m50.809996658s) [noise=true , address=145.79.227.2:4119 , piece_id=MA6YWCKWF7742D3LBOWXMDF3EW6EJIMH3V6N77WUUNROBVH4QT6A, seg_pos={0 0}, upload=187893]: Done uploading piece. err: ecclient: upload failed (node:121xNAnkm9HZXUszp26UmWTsuhp3igyeDVea6gSCdtrQUQDVHpW, address:145.79.227.2:4119): protocol: rpc: dial tcp 145.79.227.2:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:47:47.794353810 (195h36m50.810023416s) [noise=true , address=145.79.227.14:4843 , piece_id=XKMKP63LH55GKKA3VEEOW4TM34XRJAUZXECYUWPOUMCHLD74RGCA, seg_pos={0 0}, upload=187893]: Done uploading piece. err: ecclient: upload failed (node:12HvL2JDP4GCsu3tziVEKRMh14nEzyRMRuWNSmVvs27LzHTbNpS, address:145.79.227.14:4843): protocol: rpc: dial tcp 145.79.227.14:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:47:47.794677965 (195h36m50.810347574s) [noise=true , address=145.79.227.5:4226 , piece_id=CL2GAZ3ZYKEH3FCOV3IUX576RPZSSSUMZDNFIEAPIUAQ57NRGEMQ, seg_pos={0 0}, upload=187893]: Done uploading piece. err: ecclient: upload failed (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt, address:145.79.227.5:4226): protocol: rpc: dial tcp 145.79.227.5:4226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:47:47.795004070 (195h36m50.81067368s) [noise=true , address=145.79.227.33:4873 , piece_id=3KNKPDTQQB57PFD2RSLZW4UUZDFY3QCHCBHNU43MSVQWI7BBYCZQ, seg_pos={0 0}, upload=187893]: Done uploading piece. err: ecclient: upload failed (node:12b5GdYtmYz9D1QW2z78W3Z2fWvMJGtrvrMJ1njbcqbtCDUdwHo, address:145.79.227.33:4873): protocol: rpc: dial tcp 145.79.227.33:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:47:47.795019704 (195h36m50.810689299s) [noise=true , address=145.79.227.31:4802 , piece_id=4ABTXNYMEHQUAN5EKPWCVVUJ5DS5ALJGCUBUAPAOFNA3AUUVPLDQ, seg_pos={0 0}, upload=187893]: Done uploading piece. err: ecclient: upload failed (node:1iMEuZv8uHpYjTqP9JVpznKPPdEyyy4xMTsgVUGMuCjZkV4cri, address:145.79.227.31:4802): protocol: rpc: dial tcp 145.79.227.31:4802: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:47:47.795352737 (195h36m50.811022347s) [noise=true , address=145.79.227.22:4515 , piece_id=OQLP56N5QGMKJ2G57LTYJVWLX2WWQEZVZXNINNERXPIQIRYDC2TQ, seg_pos={0 0}, upload=187893]: Done uploading piece. err: ecclient: upload failed (node:1ZZmn2sRUkYuNmHY3RY7skQwAYoVYth7rJ8DhmjEpzgE4Rtmxb, address:145.79.227.22:4515): protocol: rpc: dial tcp 145.79.227.22:4515: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:47:47.795403902 (195h36m50.811073502s) [seg_pos={0 0}, upload=187893]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Mar 18 19:47:47.998456985 (195h36m51.014126593s) [upload=187893]: Done uploading segments... Mar 18 19:47:47.998507884 (195h36m51.014177495s) [upload=187893]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 30.206587504s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [...........................................X..X...X..X..X...] duration: 3.492030959s trace: https://jaeger.dev.storj.io/trace/220541cf08cfd830 FAIL (2026-03-18 19:47:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== .BatchRequestItem_ObjectBegin Mar 18 19:45:25.326452964 (195h34m28.342122568s) [upload=187860]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 19:45:25.456683819 (195h34m28.472353428s) [seg_pos={0 0}, upload=187860]: Begin upload segment... Mar 18 19:45:25.456770594 (195h34m28.472440184s) [seg_pos={0 0}, upload=187860]: Done begin upload segment. Mar 18 19:45:25.456821582 (195h34m28.472491172s) [noise=true , address=145.79.227.36:4525 , piece_id=7FBEMPOKGXT2YB5Q6EEYJBFBX35RRCOOTQ2UETDSIVFW34CQZDVA, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.456867918 (195h34m28.472537526s) [noise=true , address=45.91.116.5:4324 , piece_id=ZFBWMYRD65NBX2FUNBIJF57VRTZOUJEPXOEWHWU63QWBATPWJCDQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.457527075 (195h34m28.47319668s) [noise=true , address=79.127.219.38:4827 , piece_id=2WC4XR64CXG7I7EPU6VFNYUAC76BJ3EKEPKSPRT4I55IYRFQVXLQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.458341322 (195h34m28.474010934s) [noise=true , address=79.127.205.233:4144 , piece_id=C3QUG3TRTWKVN3ZOCEWVNP74WE4QMYA65AHCAKEXOPBPKY4AUZDA, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.458481997 (195h34m28.4741516s) [noise=true , address=145.79.227.17:4993 , piece_id=AN3MJQVU5MVMQLL623PWSLWGH3N242LVPKFOG6AZAUJA6WGMHMFA, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.459053712 (195h34m28.474723318s) [noise=true , address=145.79.227.5:4238 , piece_id=PNXIWK2VZZSCR53VQOBOMP5Z4F5YYRIWWX4CNPOXNRJAB4RWKJLQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.459541420 (195h34m28.475211026s) [noise=true , address=79.127.205.235:4260 , piece_id=VRXKZO7BI77HQO7F33I3EHJZQOS3VCNGH4XZGZEEOHYTM4FOIIDQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.460127026 (195h34m28.475796632s) [noise=true , address=145.79.227.33:4887 , piece_id=GPFUU4XAWYSCN4AG3VV6AI6BWEGLSHJFN5VGTVFGB3PFHPXOGU5Q, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.460537480 (195h34m28.476207086s) [noise=true , address=145.79.227.20:4472 , piece_id=6PS3R5ANRNMQFWPH3F2MWMIVTP53URHAQ7E3NLX72P6U6PGCLQTQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.461390949 (195h34m28.477060561s) [noise=true , address=193.149.164.68:4039 , piece_id=FLJZMLJXBWFG7FOEHWTVDJU2NTW2GZPHNK5S2MIT5K2EBEMACKSQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.461470344 (195h34m28.477139959s) [noise=true , address=79.127.205.234:4206 , piece_id=FCSOPKSPRRYZ5E6RDK72WZVVAWTOD53MYSUST63NR52RY6QDHMKQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.462470296 (195h34m28.478139902s) [noise=true , address=145.79.227.11:4747 , piece_id=JTWQJUGUKHAVWGG6T2ZVJHZ4R5FZTTYFYKXXRD3MH4ZBLEOD4MRQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.462553305 (195h34m28.478222915s) [noise=true , address=77.90.31.17:4782 , piece_id=7ZJZOZOLIXGWYAGAV65WIEY76DOCM6CSE2HAUWXRPKHCLVOQQFEQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.463505826 (195h34m28.479175427s) [noise=true , address=77.90.31.10:4537 , piece_id=N4UDTJUBLM6NLN2NNE3YYKAYHMCNGQWLX4R6TZ35UPDONZ7NMIOA, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.464069344 (195h34m28.479738951s) [noise=true , address=167.17.72.69:4716 , piece_id=3ETXEX7UWPM4DU3DL2EGA5Z4QLKUKHAT23KG3PD5YL6XEZCJTGRA, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.465380624 (195h34m28.481050227s) [noise=true , address=145.79.227.2:4135 , piece_id=DHYVS2YDQW7FQANTEAPJKWBCNLUWPZNANCHFNDWLGFHPVOYPVTJQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.465888656 (195h34m28.481558265s) [noise=true , address=145.79.227.44:4206 , piece_id=L42NBORHYROKARBF6Z7CUJQ4SO5IA5SOBRA3TJWOYZDZKTSNXE4Q, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.466308049 (195h34m28.481977655s) [noise=true , address=79.127.219.46:4920 , piece_id=76VAJBNCJC4WAYXSEEQGZW7BEGGULKJFLV75UOBDBAVII7AKR62A, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:26.019613415 (195h34m29.035283015s) [upload=187860]: Next returned nil segment Mar 18 19:45:26.019627850 (195h34m29.035297451s) [upload=187860]: Waiting for error group managing segments... Mar 18 19:45:26.193532210 (195h34m29.209201815s) [noise=true , address=45.91.116.5:4324 , piece_id=ZFBWMYRD65NBX2FUNBIJF57VRTZOUJEPXOEWHWU63QWBATPWJCDQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.218509951 (195h34m29.234179558s) [noise=true , address=79.127.219.38:4827 , piece_id=2WC4XR64CXG7I7EPU6VFNYUAC76BJ3EKEPKSPRT4I55IYRFQVXLQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.219073762 (195h34m29.234743372s) [noise=true , address=77.90.31.10:4537 , piece_id=N4UDTJUBLM6NLN2NNE3YYKAYHMCNGQWLX4R6TZ35UPDONZ7NMIOA, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.271321232 (195h34m29.286990847s) [noise=true , address=79.127.219.46:4920 , piece_id=76VAJBNCJC4WAYXSEEQGZW7BEGGULKJFLV75UOBDBAVII7AKR62A, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.293176147 (195h34m29.308845754s) [noise=true , address=77.90.31.17:4782 , piece_id=7ZJZOZOLIXGWYAGAV65WIEY76DOCM6CSE2HAUWXRPKHCLVOQQFEQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.662025397 (195h34m29.677694999s) [noise=true , address=167.17.72.69:4716 , piece_id=3ETXEX7UWPM4DU3DL2EGA5Z4QLKUKHAT23KG3PD5YL6XEZCJTGRA, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.879968454 (195h34m29.895638055s) [noise=true , address=79.127.205.233:4144 , piece_id=C3QUG3TRTWKVN3ZOCEWVNP74WE4QMYA65AHCAKEXOPBPKY4AUZDA, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.898670905 (195h34m29.914340506s) [noise=true , address=79.127.205.235:4260 , piece_id=VRXKZO7BI77HQO7F33I3EHJZQOS3VCNGH4XZGZEEOHYTM4FOIIDQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.903771534 (195h34m29.919441134s) [noise=true , address=193.149.164.68:4039 , piece_id=FLJZMLJXBWFG7FOEHWTVDJU2NTW2GZPHNK5S2MIT5K2EBEMACKSQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.993657693 (195h34m30.009327292s) [noise=true , address=79.127.205.234:4206 , piece_id=FCSOPKSPRRYZ5E6RDK72WZVVAWTOD53MYSUST63NR52RY6QDHMKQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:46:25.039034405 (195h35m28.054704015s) [noise=true , address=145.79.227.2:4135 , piece_id=DHYVS2YDQW7FQANTEAPJKWBCNLUWPZNANCHFNDWLGFHPVOYPVTJQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:1uEBWRaMH4mLApUcgHyHQmGxxbyXnYUScg5Z2on9LHESYkqj3S, address:145.79.227.2:4135): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.039411284 (195h35m28.055080905s) [noise=true , address=145.79.227.44:4206 , piece_id=L42NBORHYROKARBF6Z7CUJQ4SO5IA5SOBRA3TJWOYZDZKTSNXE4Q, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:12Mp4ZzYgrP3ZSXDeHbSTXaAJMPgGKUqVjdNtyfeKE5qBMAzBke, address:145.79.227.44:4206): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.039509444 (195h35m28.055179057s) [noise=true , address=145.79.227.5:4238 , piece_id=PNXIWK2VZZSCR53VQOBOMP5Z4F5YYRIWWX4CNPOXNRJAB4RWKJLQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:122AVjaN3YaiABnUjuZwSb6apPDeszZAWvmzwtP47k1fxsKLD5V, address:145.79.227.5:4238): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.039966227 (195h35m28.055635833s) [noise=true , address=145.79.227.33:4887 , piece_id=GPFUU4XAWYSCN4AG3VV6AI6BWEGLSHJFN5VGTVFGB3PFHPXOGU5Q, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:1orvhuWfkUGCykWsS2E5sHtMcEW3fFcCNzTQ4xSxqdhUrBza8H, address:145.79.227.33:4887): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.040027384 (195h35m28.05569699s) [noise=true , address=145.79.227.17:4993 , piece_id=AN3MJQVU5MVMQLL623PWSLWGH3N242LVPKFOG6AZAUJA6WGMHMFA, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD, address:145.79.227.17:4993): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.040345443 (195h35m28.056015047s) [noise=true , address=145.79.227.20:4472 , piece_id=6PS3R5ANRNMQFWPH3F2MWMIVTP53URHAQ7E3NLX72P6U6PGCLQTQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:12GACnovbrme3fQknSDpZWVLq838VKop9mPjxnVXGdJ8KaqyaTi, address:145.79.227.20:4472): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.040426406 (195h35m28.056095999s) [noise=true , address=145.79.227.11:4747 , piece_id=JTWQJUGUKHAVWGG6T2ZVJHZ4R5FZTTYFYKXXRD3MH4ZBLEOD4MRQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:12mkHFDS6YCheghJdaJFybhcv9yhV4Acfs1RhnCW8yzA23neFRC, address:145.79.227.11:4747): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.040650249 (195h35m28.056319864s) [noise=true , address=145.79.227.36:4525 , piece_id=7FBEMPOKGXT2YB5Q6EEYJBFBX35RRCOOTQ2UETDSIVFW34CQZDVA, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX, address:145.79.227.36:4525): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.040699435 (195h35m28.056369021s) [seg_pos={0 0}, upload=187860]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Mar 18 19:46:25.162188571 (195h35m28.177858163s) [upload=187860]: Done uploading segments... Mar 18 19:46:25.162240944 (195h35m28.177910547s) [upload=187860]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration:* 3.492061919s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 14 out of 60 is failed (threshold=6) [.........................................XXXXXXXX....XX.XXXX] duration: 5.001215548s trace: https://jaeger.dev.storj.io/trace/63b8ee23a113b1f0 FAIL (2026-03-18 19:47:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.00129551s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [............................................XXX.XXX.........] duration: 3.221251079s trace: https://jaeger.dev.storj.io/trace/4d6a8ab787f335c0 FAIL (2026-03-18 19:47:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.22128575s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [............................................XXXXXXXXXXXXXXXX] duration: 30.110178916s trace: https://jaeger.dev.storj.io/trace/3c3625f5695d350 FAIL (2026-03-18 19:47:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Mar 18 19:46:47.821250921 (195h35m50.836920534s) [upload=187883]: Starting upload Mar 18 19:46:47.821336476 (195h35m50.837006068s) [upload=187883]: Done starting upload Mar 18 19:46:47.821549542 (195h35m50.837219132s) [upload=187883]: Uploading segments... Mar 18 19:46:47.821888940 (195h35m50.83755854s) [upload=187883]: Got next segment. Inline: false Mar 18 19:46:47.822036594 (195h35m50.837706195s) [upload=187883]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 19:46:47.822045100 (195h35m50.837714702s) [upload=187883]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 19:46:47.933877930 (195h35m50.949547532s) [seg_pos={0 0}, upload=187883]: Begin upload segment... Mar 18 19:46:47.933963245 (195h35m50.949632848s) [seg_pos={0 0}, upload=187883]: Done begin upload segment. Mar 18 19:46:47.934022950 (195h35m50.94969255s) [noise=true , address=45.91.116.12:4244 , piece_id=7QLVUKBHAAVNRPK2P5S5X6YBCKNQ4MMN7EX3GXKCZMSSATW5FXHA, seg_pos={0 0}, upload=187883]: Uploading piece... Mar 18 19:46:47.934039059 (195h35m50.949708664s) [noise=true , address=45.91.116.14:4192 , piece_id=OF7BCBKQBEXKAQ43VO3LY2Z3F5NE72XJGEMGB4F636WKGNTSUG6Q, seg_pos={0 0}, upload=187883]: Uploading piece... Mar 18 19:46:47.934387933 (195h35m50.950057536s) [noise=true , address=77.90.31.19:4846 , piece_id=HN5EFXPYPUSVY6UZRNDB7X5AK3N32DYWTK7YI52UN5HNIIXCO2PQ, seg_pos={0 0}, upload=187883]: Uploading piece... Mar 18 19:46:47.934798112 (195h35m50.950467715s) [noise=true , address=79.127.219.41:4930 , piece_id=PU667OWQJKGMUFQJF32QH5DH3YIJSPQPFF4OMMTBKDWDY5BUKTEA, seg_pos={0 0}, upload=187883]: Uploading piece... Mar 18 19:46:47.935283075 (195h35m50.950952678s) [noise=true , address=145.79.227.2:4113 , piece_id=FZE2GIGLMREMJI52JJ5PF5OZLZXDPX47IW3355FA6SJIDSRZXISA, seg_pos={0 0}, upload=187883]: Uploading piece... Mar 18 19:46:47.935468700 (195h35m50.95113831s) [noise=true , address=79.127.219.46:4892 , piece_id=KTCLIQLKZURSWVAXWTGE5G553OC4WPJCLMVRHHIHUDEIYQ64QZTQ, seg_pos={0 0}, upload=187883]: Uploading piece... Mar 18 19:46:47.935695834 (195h35m50.951365433s) [noise=true , address=77.90.31.21:4881 , piece_id=AGVPGETO65EMDTCHSIVZDT7BCWO234X337B2MLP4MJFW5HTJMLCQ, seg_pos={0 0}, upload=187883]: Uploading piece... Mar 18 19:46:47.935939789 (195h35m50.95160939s) [noise=true , address=145.79.227.9:4371 , piece_id=D2JEGDZJ7AIUGIJP3652EUMP7CZU4F7ZF44KSEXZ6GXGMKZOOUTA, seg_pos={0 0}, upload=187883]: Uploading piece... Mar 18 19:46:47.936114219 (195h35m50.95178382s) [noise=true , address=145.79.227.40:4051 , piece_id=J7FQF2PBTPVDKRHDIHD5NWOCP42B2MIDFY4W34SVQQ5PXYZYIRCQ, seg_pos={0 0}, upload=187883]: Uploading piece... Mar 18 19:46:47.936324592 (195h35m50.951994192s) [noise=true , address=77.90.31.11:4558 , piece_id=NB6PZ5FHKZNSYH5E3F4AEBVBVQFNTWGJJOTVR3KLNYI2RPP337RQ, seg_pos={0 0}, upload=187883]: Uploading piece... Mar 18 19:46:47.936564670 (195h35m50.952234271s) [noise=true , address=45.91.116.6:4976 , piece_id=5RTXEDEZR5PQKXOAAU4WODBR4HNEUDDDHPPTHNHIOXOSKRJH53VQ, seg_pos={0 0}, upload=187883]: Uploading piece... Mar 18 19:46:47.936713212 (195h35m50.952382815s) [noise=true , address=145.79.227.33:4870 , piece_id=4JEXCKKULHZFQA3CVJFJVGKJG3HJUWCIUQZ2NLOR4ZEJJB3Q3YBQ, seg_pos={0 0}, upload=187883]: Uploading piece... Mar 18 19:46:47.937097574 (195h35m50.952767174s) [noise=true , address=145.79.227.49:4337 , piece_id=WBDAYLMKP2GNO54CTPFNXPA5CB2WTAL2JHHIWKMUTJRQFE6TCELQ, seg_pos={0 0}, upload=187883]: Uploading piece... Mar 18 19:46:47.937615161 (195h35m50.953284765s) [noise=true , address=77.90.31.18:4810 , piece_id=M3E3Z3P4ZCKNKWMGJOXOL6JJ3H3VM5DFCPS2DTCLJEEH6QENKK3Q, seg_pos={0 0}, upload=187883]: Uploading piece... Mar 18 19:46:47.938111440 (195h35m50.953781043s) [noise=true , address=145.79.227.30:4764 , piece_id=PCH4OWO36PPUR2X2L3PNHOYXJMPMSYLT3YYZVRUP4ISW7J3AUMWA, seg_pos={0 0}, upload=187883]: Uploading piece... Mar 18 19:46:47.939144741 (195h35m50.954814341s) [noise=true , address=77.90.31.16:4737 , piece_id=V54PIPFFTEWVRCKK52N74V5AYIVRTIZSQ6QSGTA56XWY44YFU5BQ, seg_pos={0 0}, upload=187883]: Uploading piece... Mar 18 19:46:48.575787372 (195h35m51.591456976s) [upload=187883]: Next returned nil segment Mar 18 19:46:48.575807088 (195h35m51.591476696s) [upload=187883]: Waiting for error group managing segments... Mar 18 19:46:48.702237464 (195h35m51.717907074s) [noise=true , address=45.91.116.6:4976 , piece_id=5RTXEDEZR5PQKXOAAU4WODBR4HNEUDDDHPPTHNHIOXOSKRJH53VQ, seg_pos={0 0}, upload=187883]: Done uploading piece. err: <nil> Mar 18 19:46:48.704153655 (195h35m51.719823257s) [noise=true , address=45.91.116.12:4244 , piece_id=7QLVUKBHAAVNRPK2P5S5X6YBCKNQ4MMN7EX3GXKCZMSSATW5FXHA, seg_pos={0 0}, upload=187883]: Done uploading piece. err: <nil> Mar 18 19:46:48.711656205 (195h35m51.727325811s) [noise=true , address=45.91.116.14:4192 , piece_id=OF7BCBKQBEXKAQ43VO3LY2Z3F5NE72XJGEMGB4F636WKGNTSUG6Q, seg_pos={0 0}, upload=187883]: Done uploading piece. err: <nil> Mar 18 19:46:48.736170978 (195h35m51.751840578s) [noise=true , address=77.90.31.11:4558 , piece_id=NB6PZ5FHKZNSYH5E3F4AEBVBVQFNTWGJJOTVR3KLNYI2RPP337RQ, seg_pos={0 0}, upload=187883]: Done uploading piece. err: <nil> Mar 18 19:46:48.739301764 (195h35m51.754971365s) [noise=true , address=79.127.219.46:4892 , piece_id=KTCLIQLKZURSWVAXWTGE5G553OC4WPJCLMVRHHIHUDEIYQ64QZTQ, seg_pos={0 0}, upload=187883]: Done uploading piece. err: <nil> Mar 18 19:46:48.755360539 (195h35m51.771030138s) [noise=true , address=79.127.219.41:4930 , piece_id=PU667OWQJKGMUFQJF32QH5DH3YIJSPQPFF4OMMTBKDWDY5BUKTEA, seg_pos={0 0}, upload=187883]: Done uploading piece. err: <nil> Mar 18 19:46:48.768162098 (195h35m51.783831706s) [noise=true , address=77.90.31.18:4810 , piece_id=M3E3Z3P4ZCKNKWMGJOXOL6JJ3H3VM5DFCPS2DTCLJEEH6QENKK3Q, seg_pos={0 0}, upload=187883]: Done uploading piece. err: <nil> Mar 18 19:46:48.775652550 (195h35m51.791322156s) [noise=true , address=77.90.31.21:4881 , piece_id=AGVPGETO65EMDTCHSIVZDT7BCWO234X337B2MLP4MJFW5HTJMLCQ, seg_pos={0 0}, upload=187883]: Done uploading piece. err: <nil> Mar 18 19:46:48.782392338 (195h35m51.798061955s) [noise=true , address=77.90.31.19:4846 , piece_id=HN5EFXPYPUSVY6UZRNDB7X5AK3N32DYWTK7YI52UN5HNIIXCO2PQ, seg_pos={0 0}, upload=187883]: Done uploading piece. err: <nil> Mar 18 19:46:48.790269643 (195h35m51.805939248s) [noise=true , address=77.90.31.16:4737 , piece_id=V54PIPFFTEWVRCKK52N74V5AYIVRTIZSQ6QSGTA56XWY44YFU5BQ, seg_pos={0 0}, upload=187883]: Done uploading piece. err: <nil> Mar 18 19:47:17.541478418 (195h36m20.557148026s) [noise=true , address=145.79.227.2:4113 , piece_id=FZE2GIGLMREMJI52JJ5PF5OZLZXDPX47IW3355FA6SJIDSRZXISA, seg_pos={0 0}, upload=187883]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: rpc: dial tcp 145.79.227.2:4113: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:47:17.542371246 (195h36m20.558040858s) [noise=true , address=145.79.227.9:4371 , piece_id=D2JEGDZJ7AIUGIJP3652EUMP7CZU4F7ZF44KSEXZ6GXGMKZOOUTA, seg_pos={0 0}, upload=187883]: Done uploading piece. err: ecclient: upload failed (node:12cSqcGZqvZqs1QKBi6WK4fUDKgvQcmTupR1C9ZkWz879XeWPEL, address:145.79.227.9:4371): protocol: rpc: dial tcp 145.79.227.9:4371: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:47:17.542426890 (195h36m20.558096502s) [noise=true , address=145.79.227.40:4051 , piece_id=J7FQF2PBTPVDKRHDIHD5NWOCP42B2MIDFY4W34SVQQ5PXYZYIRCQ, seg_pos={0 0}, upload=187883]: Done uploading piece. err: ecclient: upload failed (node:12rPbTARUrq7YsD8X75ocnyEvC517itk79s3Dp1wUg72fVuBS8n, address:145.79.227.40:4051): protocol: rpc: dial tcp 145.79.227.40:4051: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:47:17.542721522 (195h36m20.55839113s) [noise=true , address=145.79.227.30:4764 , piece_id=PCH4OWO36PPUR2X2L3PNHOYXJMPMSYLT3YYZVRUP4ISW7J3AUMWA, seg_pos={0 0}, upload=187883]: Done uploading piece. err: ecclient: upload failed (node:127hafVDbun7FoG4JdcogBmRNT6rhL2QQbLDLstFnx7HCWyiC9N, address:145.79.227.30:4764): protocol: rpc: dial tcp 145.79.227.30:4764: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:47:17.542909525 (195h36m20.558579135s) [noise=true , address=145.79.227.49:4337 , piece_id=WBDAYLMKP2GNO54CTPFNXPA5CB2WTAL2JHHIWKMUTJRQFE6TCELQ, seg_pos={0 0}, upload=187883]: Done uploading piece. err: ecclient: upload failed (node:1nTZFfzNCcVAjut7NXf9seifysJDfvXkS8aZSnXKfw6gh1TrjE, address:145.79.227.49:4337): protocol: rpc: dial tcp 145.79.227.49:4337: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:47:17.543266631 (195h36m20.55893624s) [noise=true , address=145.79.227.33:4870 , piece_id=4JEXCKKULHZFQA3CVJFJVGKJG3HJUWCIUQZ2NLOR4ZEJJB3Q3YBQ, seg_pos={0 0}, upload=187883]: Done uploading piece. err: ecclient: upload failed (node:12sDH7ccMQhjePqpbTkkxMAtumNLxpMHiT56cnfaYrUTFYxuPVt, address:145.79.227.33:4870): protocol: rpc: dial tcp 145.79.227.33:4870: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:47:17.543295303 (195h36m20.558964903s) [seg_pos={0 0}, upload=187883]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Mar 18 19:47:17.648973560 (195h36m20.664643177s) [upload=187883]: Done uploading segments... Mar 18 19:47:17.649049774 (195h36m20.664719379s) [upload=187883]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 30.110255628s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [............................................X..X...X..X..X..] duration: 3.414293538s trace: https://jaeger.dev.storj.io/trace/220541cf08cfd830 FAIL (2026-03-18 19:47:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== .BatchRequestItem_ObjectBegin Mar 18 19:45:25.326452964 (195h34m28.342122568s) [upload=187860]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 19:45:25.456683819 (195h34m28.472353428s) [seg_pos={0 0}, upload=187860]: Begin upload segment... Mar 18 19:45:25.456770594 (195h34m28.472440184s) [seg_pos={0 0}, upload=187860]: Done begin upload segment. Mar 18 19:45:25.456821582 (195h34m28.472491172s) [noise=true , address=145.79.227.36:4525 , piece_id=7FBEMPOKGXT2YB5Q6EEYJBFBX35RRCOOTQ2UETDSIVFW34CQZDVA, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.456867918 (195h34m28.472537526s) [noise=true , address=45.91.116.5:4324 , piece_id=ZFBWMYRD65NBX2FUNBIJF57VRTZOUJEPXOEWHWU63QWBATPWJCDQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.457527075 (195h34m28.47319668s) [noise=true , address=79.127.219.38:4827 , piece_id=2WC4XR64CXG7I7EPU6VFNYUAC76BJ3EKEPKSPRT4I55IYRFQVXLQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.458341322 (195h34m28.474010934s) [noise=true , address=79.127.205.233:4144 , piece_id=C3QUG3TRTWKVN3ZOCEWVNP74WE4QMYA65AHCAKEXOPBPKY4AUZDA, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.458481997 (195h34m28.4741516s) [noise=true , address=145.79.227.17:4993 , piece_id=AN3MJQVU5MVMQLL623PWSLWGH3N242LVPKFOG6AZAUJA6WGMHMFA, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.459053712 (195h34m28.474723318s) [noise=true , address=145.79.227.5:4238 , piece_id=PNXIWK2VZZSCR53VQOBOMP5Z4F5YYRIWWX4CNPOXNRJAB4RWKJLQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.459541420 (195h34m28.475211026s) [noise=true , address=79.127.205.235:4260 , piece_id=VRXKZO7BI77HQO7F33I3EHJZQOS3VCNGH4XZGZEEOHYTM4FOIIDQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.460127026 (195h34m28.475796632s) [noise=true , address=145.79.227.33:4887 , piece_id=GPFUU4XAWYSCN4AG3VV6AI6BWEGLSHJFN5VGTVFGB3PFHPXOGU5Q, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.460537480 (195h34m28.476207086s) [noise=true , address=145.79.227.20:4472 , piece_id=6PS3R5ANRNMQFWPH3F2MWMIVTP53URHAQ7E3NLX72P6U6PGCLQTQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.461390949 (195h34m28.477060561s) [noise=true , address=193.149.164.68:4039 , piece_id=FLJZMLJXBWFG7FOEHWTVDJU2NTW2GZPHNK5S2MIT5K2EBEMACKSQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.461470344 (195h34m28.477139959s) [noise=true , address=79.127.205.234:4206 , piece_id=FCSOPKSPRRYZ5E6RDK72WZVVAWTOD53MYSUST63NR52RY6QDHMKQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.462470296 (195h34m28.478139902s) [noise=true , address=145.79.227.11:4747 , piece_id=JTWQJUGUKHAVWGG6T2ZVJHZ4R5FZTTYFYKXXRD3MH4ZBLEOD4MRQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.462553305 (195h34m28.478222915s) [noise=true , address=77.90.31.17:4782 , piece_id=7ZJZOZOLIXGWYAGAV65WIEY76DOCM6CSE2HAUWXRPKHCLVOQQFEQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.463505826 (195h34m28.479175427s) [noise=true , address=77.90.31.10:4537 , piece_id=N4UDTJUBLM6NLN2NNE3YYKAYHMCNGQWLX4R6TZ35UPDONZ7NMIOA, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.464069344 (195h34m28.479738951s) [noise=true , address=167.17.72.69:4716 , piece_id=3ETXEX7UWPM4DU3DL2EGA5Z4QLKUKHAT23KG3PD5YL6XEZCJTGRA, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.465380624 (195h34m28.481050227s) [noise=true , address=145.79.227.2:4135 , piece_id=DHYVS2YDQW7FQANTEAPJKWBCNLUWPZNANCHFNDWLGFHPVOYPVTJQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.465888656 (195h34m28.481558265s) [noise=true , address=145.79.227.44:4206 , piece_id=L42NBORHYROKARBF6Z7CUJQ4SO5IA5SOBRA3TJWOYZDZKTSNXE4Q, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.466308049 (195h34m28.481977655s) [noise=true , address=79.127.219.46:4920 , piece_id=76VAJBNCJC4WAYXSEEQGZW7BEGGULKJFLV75UOBDBAVII7AKR62A, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:26.019613415 (195h34m29.035283015s) [upload=187860]: Next returned nil segment Mar 18 19:45:26.019627850 (195h34m29.035297451s) [upload=187860]: Waiting for error group managing segments... Mar 18 19:45:26.193532210 (195h34m29.209201815s) [noise=true , address=45.91.116.5:4324 , piece_id=ZFBWMYRD65NBX2FUNBIJF57VRTZOUJEPXOEWHWU63QWBATPWJCDQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.218509951 (195h34m29.234179558s) [noise=true , address=79.127.219.38:4827 , piece_id=2WC4XR64CXG7I7EPU6VFNYUAC76BJ3EKEPKSPRT4I55IYRFQVXLQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.219073762 (195h34m29.234743372s) [noise=true , address=77.90.31.10:4537 , piece_id=N4UDTJUBLM6NLN2NNE3YYKAYHMCNGQWLX4R6TZ35UPDONZ7NMIOA, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.271321232 (195h34m29.286990847s) [noise=true , address=79.127.219.46:4920 , piece_id=76VAJBNCJC4WAYXSEEQGZW7BEGGULKJFLV75UOBDBAVII7AKR62A, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.293176147 (195h34m29.308845754s) [noise=true , address=77.90.31.17:4782 , piece_id=7ZJZOZOLIXGWYAGAV65WIEY76DOCM6CSE2HAUWXRPKHCLVOQQFEQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.662025397 (195h34m29.677694999s) [noise=true , address=167.17.72.69:4716 , piece_id=3ETXEX7UWPM4DU3DL2EGA5Z4QLKUKHAT23KG3PD5YL6XEZCJTGRA, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.879968454 (195h34m29.895638055s) [noise=true , address=79.127.205.233:4144 , piece_id=C3QUG3TRTWKVN3ZOCEWVNP74WE4QMYA65AHCAKEXOPBPKY4AUZDA, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.898670905 (195h34m29.914340506s) [noise=true , address=79.127.205.235:4260 , piece_id=VRXKZO7BI77HQO7F33I3EHJZQOS3VCNGH4XZGZEEOHYTM4FOIIDQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.903771534 (195h34m29.919441134s) [noise=true , address=193.149.164.68:4039 , piece_id=FLJZMLJXBWFG7FOEHWTVDJU2NTW2GZPHNK5S2MIT5K2EBEMACKSQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.993657693 (195h34m30.009327292s) [noise=true , address=79.127.205.234:4206 , piece_id=FCSOPKSPRRYZ5E6RDK72WZVVAWTOD53MYSUST63NR52RY6QDHMKQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:46:25.039034405 (195h35m28.054704015s) [noise=true , address=145.79.227.2:4135 , piece_id=DHYVS2YDQW7FQANTEAPJKWBCNLUWPZNANCHFNDWLGFHPVOYPVTJQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:1uEBWRaMH4mLApUcgHyHQmGxxbyXnYUScg5Z2on9LHESYkqj3S, address:145.79.227.2:4135): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.039411284 (195h35m28.055080905s) [noise=true , address=145.79.227.44:4206 , piece_id=L42NBORHYROKARBF6Z7CUJQ4SO5IA5SOBRA3TJWOYZDZKTSNXE4Q, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:12Mp4ZzYgrP3ZSXDeHbSTXaAJMPgGKUqVjdNtyfeKE5qBMAzBke, address:145.79.227.44:4206): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.039509444 (195h35m28.055179057s) [noise=true , address=145.79.227.5:4238 , piece_id=PNXIWK2VZZSCR53VQOBOMP5Z4F5YYRIWWX4CNPOXNRJAB4RWKJLQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:122AVjaN3YaiABnUjuZwSb6apPDeszZAWvmzwtP47k1fxsKLD5V, address:145.79.227.5:4238): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.039966227 (195h35m28.055635833s) [noise=true , address=145.79.227.33:4887 , piece_id=GPFUU4XAWYSCN4AG3VV6AI6BWEGLSHJFN5VGTVFGB3PFHPXOGU5Q, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:1orvhuWfkUGCykWsS2E5sHtMcEW3fFcCNzTQ4xSxqdhUrBza8H, address:145.79.227.33:4887): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.040027384 (195h35m28.05569699s) [noise=true , address=145.79.227.17:4993 , piece_id=AN3MJQVU5MVMQLL623PWSLWGH3N242LVPKFOG6AZAUJA6WGMHMFA, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD, address:145.79.227.17:4993): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.040345443 (195h35m28.056015047s) [noise=true , address=145.79.227.20:4472 , piece_id=6PS3R5ANRNMQFWPH3F2MWMIVTP53URHAQ7E3NLX72P6U6PGCLQTQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:12GACnovbrme3fQknSDpZWVLq838VKop9mPjxnVXGdJ8KaqyaTi, address:145.79.227.20:4472): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.040426406 (195h35m28.056095999s) [noise=true , address=145.79.227.11:4747 , piece_id=JTWQJUGUKHAVWGG6T2ZVJHZ4R5FZTTYFYKXXRD3MH4ZBLEOD4MRQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:12mkHFDS6YCheghJdaJFybhcv9yhV4Acfs1RhnCW8yzA23neFRC, address:145.79.227.11:4747): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.040650249 (195h35m28.056319864s) [noise=true , address=145.79.227.36:4525 , piece_id=7FBEMPOKGXT2YB5Q6EEYJBFBX35RRCOOTQ2UETDSIVFW34CQZDVA, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX, address:145.79.227.36:4525): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.040699435 (195h35m28.056369021s) [seg_pos={0 0}, upload=187860]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Mar 18 19:46:25.162188571 (195h35m28.177858163s) [upload=187860]: Done uploading segments... Mar 18 19:46:25.162240944 (195h35m28.177910547s) [upload=187860]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration:* 3.414323873s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [.............................................XXX.XXX........] duration: 3.517439935s trace: https://jaeger.dev.storj.io/trace/4d6a8ab787f335c0 FAIL (2026-03-18 19:46:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.517501891s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 13 out of 60 is failed (threshold=6) [..........................................XXXXXXXX....XX.XXX] duration: 5.001849219s trace: https://jaeger.dev.storj.io/trace/7206752ac6076f5c FAIL (2026-03-18 19:46:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001878151s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [.............................................X..X...X..X..X.] duration: 3.345409036s trace: https://jaeger.dev.storj.io/trace/220541cf08cfd830 FAIL (2026-03-18 19:46:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== .BatchRequestItem_ObjectBegin Mar 18 19:45:25.326452964 (195h34m28.342122568s) [upload=187860]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 19:45:25.456683819 (195h34m28.472353428s) [seg_pos={0 0}, upload=187860]: Begin upload segment... Mar 18 19:45:25.456770594 (195h34m28.472440184s) [seg_pos={0 0}, upload=187860]: Done begin upload segment. Mar 18 19:45:25.456821582 (195h34m28.472491172s) [noise=true , address=145.79.227.36:4525 , piece_id=7FBEMPOKGXT2YB5Q6EEYJBFBX35RRCOOTQ2UETDSIVFW34CQZDVA, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.456867918 (195h34m28.472537526s) [noise=true , address=45.91.116.5:4324 , piece_id=ZFBWMYRD65NBX2FUNBIJF57VRTZOUJEPXOEWHWU63QWBATPWJCDQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.457527075 (195h34m28.47319668s) [noise=true , address=79.127.219.38:4827 , piece_id=2WC4XR64CXG7I7EPU6VFNYUAC76BJ3EKEPKSPRT4I55IYRFQVXLQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.458341322 (195h34m28.474010934s) [noise=true , address=79.127.205.233:4144 , piece_id=C3QUG3TRTWKVN3ZOCEWVNP74WE4QMYA65AHCAKEXOPBPKY4AUZDA, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.458481997 (195h34m28.4741516s) [noise=true , address=145.79.227.17:4993 , piece_id=AN3MJQVU5MVMQLL623PWSLWGH3N242LVPKFOG6AZAUJA6WGMHMFA, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.459053712 (195h34m28.474723318s) [noise=true , address=145.79.227.5:4238 , piece_id=PNXIWK2VZZSCR53VQOBOMP5Z4F5YYRIWWX4CNPOXNRJAB4RWKJLQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.459541420 (195h34m28.475211026s) [noise=true , address=79.127.205.235:4260 , piece_id=VRXKZO7BI77HQO7F33I3EHJZQOS3VCNGH4XZGZEEOHYTM4FOIIDQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.460127026 (195h34m28.475796632s) [noise=true , address=145.79.227.33:4887 , piece_id=GPFUU4XAWYSCN4AG3VV6AI6BWEGLSHJFN5VGTVFGB3PFHPXOGU5Q, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.460537480 (195h34m28.476207086s) [noise=true , address=145.79.227.20:4472 , piece_id=6PS3R5ANRNMQFWPH3F2MWMIVTP53URHAQ7E3NLX72P6U6PGCLQTQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.461390949 (195h34m28.477060561s) [noise=true , address=193.149.164.68:4039 , piece_id=FLJZMLJXBWFG7FOEHWTVDJU2NTW2GZPHNK5S2MIT5K2EBEMACKSQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.461470344 (195h34m28.477139959s) [noise=true , address=79.127.205.234:4206 , piece_id=FCSOPKSPRRYZ5E6RDK72WZVVAWTOD53MYSUST63NR52RY6QDHMKQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.462470296 (195h34m28.478139902s) [noise=true , address=145.79.227.11:4747 , piece_id=JTWQJUGUKHAVWGG6T2ZVJHZ4R5FZTTYFYKXXRD3MH4ZBLEOD4MRQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.462553305 (195h34m28.478222915s) [noise=true , address=77.90.31.17:4782 , piece_id=7ZJZOZOLIXGWYAGAV65WIEY76DOCM6CSE2HAUWXRPKHCLVOQQFEQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.463505826 (195h34m28.479175427s) [noise=true , address=77.90.31.10:4537 , piece_id=N4UDTJUBLM6NLN2NNE3YYKAYHMCNGQWLX4R6TZ35UPDONZ7NMIOA, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.464069344 (195h34m28.479738951s) [noise=true , address=167.17.72.69:4716 , piece_id=3ETXEX7UWPM4DU3DL2EGA5Z4QLKUKHAT23KG3PD5YL6XEZCJTGRA, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.465380624 (195h34m28.481050227s) [noise=true , address=145.79.227.2:4135 , piece_id=DHYVS2YDQW7FQANTEAPJKWBCNLUWPZNANCHFNDWLGFHPVOYPVTJQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.465888656 (195h34m28.481558265s) [noise=true , address=145.79.227.44:4206 , piece_id=L42NBORHYROKARBF6Z7CUJQ4SO5IA5SOBRA3TJWOYZDZKTSNXE4Q, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.466308049 (195h34m28.481977655s) [noise=true , address=79.127.219.46:4920 , piece_id=76VAJBNCJC4WAYXSEEQGZW7BEGGULKJFLV75UOBDBAVII7AKR62A, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:26.019613415 (195h34m29.035283015s) [upload=187860]: Next returned nil segment Mar 18 19:45:26.019627850 (195h34m29.035297451s) [upload=187860]: Waiting for error group managing segments... Mar 18 19:45:26.193532210 (195h34m29.209201815s) [noise=true , address=45.91.116.5:4324 , piece_id=ZFBWMYRD65NBX2FUNBIJF57VRTZOUJEPXOEWHWU63QWBATPWJCDQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.218509951 (195h34m29.234179558s) [noise=true , address=79.127.219.38:4827 , piece_id=2WC4XR64CXG7I7EPU6VFNYUAC76BJ3EKEPKSPRT4I55IYRFQVXLQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.219073762 (195h34m29.234743372s) [noise=true , address=77.90.31.10:4537 , piece_id=N4UDTJUBLM6NLN2NNE3YYKAYHMCNGQWLX4R6TZ35UPDONZ7NMIOA, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.271321232 (195h34m29.286990847s) [noise=true , address=79.127.219.46:4920 , piece_id=76VAJBNCJC4WAYXSEEQGZW7BEGGULKJFLV75UOBDBAVII7AKR62A, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.293176147 (195h34m29.308845754s) [noise=true , address=77.90.31.17:4782 , piece_id=7ZJZOZOLIXGWYAGAV65WIEY76DOCM6CSE2HAUWXRPKHCLVOQQFEQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.662025397 (195h34m29.677694999s) [noise=true , address=167.17.72.69:4716 , piece_id=3ETXEX7UWPM4DU3DL2EGA5Z4QLKUKHAT23KG3PD5YL6XEZCJTGRA, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.879968454 (195h34m29.895638055s) [noise=true , address=79.127.205.233:4144 , piece_id=C3QUG3TRTWKVN3ZOCEWVNP74WE4QMYA65AHCAKEXOPBPKY4AUZDA, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.898670905 (195h34m29.914340506s) [noise=true , address=79.127.205.235:4260 , piece_id=VRXKZO7BI77HQO7F33I3EHJZQOS3VCNGH4XZGZEEOHYTM4FOIIDQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.903771534 (195h34m29.919441134s) [noise=true , address=193.149.164.68:4039 , piece_id=FLJZMLJXBWFG7FOEHWTVDJU2NTW2GZPHNK5S2MIT5K2EBEMACKSQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.993657693 (195h34m30.009327292s) [noise=true , address=79.127.205.234:4206 , piece_id=FCSOPKSPRRYZ5E6RDK72WZVVAWTOD53MYSUST63NR52RY6QDHMKQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:46:25.039034405 (195h35m28.054704015s) [noise=true , address=145.79.227.2:4135 , piece_id=DHYVS2YDQW7FQANTEAPJKWBCNLUWPZNANCHFNDWLGFHPVOYPVTJQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:1uEBWRaMH4mLApUcgHyHQmGxxbyXnYUScg5Z2on9LHESYkqj3S, address:145.79.227.2:4135): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.039411284 (195h35m28.055080905s) [noise=true , address=145.79.227.44:4206 , piece_id=L42NBORHYROKARBF6Z7CUJQ4SO5IA5SOBRA3TJWOYZDZKTSNXE4Q, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:12Mp4ZzYgrP3ZSXDeHbSTXaAJMPgGKUqVjdNtyfeKE5qBMAzBke, address:145.79.227.44:4206): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.039509444 (195h35m28.055179057s) [noise=true , address=145.79.227.5:4238 , piece_id=PNXIWK2VZZSCR53VQOBOMP5Z4F5YYRIWWX4CNPOXNRJAB4RWKJLQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:122AVjaN3YaiABnUjuZwSb6apPDeszZAWvmzwtP47k1fxsKLD5V, address:145.79.227.5:4238): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.039966227 (195h35m28.055635833s) [noise=true , address=145.79.227.33:4887 , piece_id=GPFUU4XAWYSCN4AG3VV6AI6BWEGLSHJFN5VGTVFGB3PFHPXOGU5Q, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:1orvhuWfkUGCykWsS2E5sHtMcEW3fFcCNzTQ4xSxqdhUrBza8H, address:145.79.227.33:4887): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.040027384 (195h35m28.05569699s) [noise=true , address=145.79.227.17:4993 , piece_id=AN3MJQVU5MVMQLL623PWSLWGH3N242LVPKFOG6AZAUJA6WGMHMFA, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD, address:145.79.227.17:4993): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.040345443 (195h35m28.056015047s) [noise=true , address=145.79.227.20:4472 , piece_id=6PS3R5ANRNMQFWPH3F2MWMIVTP53URHAQ7E3NLX72P6U6PGCLQTQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:12GACnovbrme3fQknSDpZWVLq838VKop9mPjxnVXGdJ8KaqyaTi, address:145.79.227.20:4472): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.040426406 (195h35m28.056095999s) [noise=true , address=145.79.227.11:4747 , piece_id=JTWQJUGUKHAVWGG6T2ZVJHZ4R5FZTTYFYKXXRD3MH4ZBLEOD4MRQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:12mkHFDS6YCheghJdaJFybhcv9yhV4Acfs1RhnCW8yzA23neFRC, address:145.79.227.11:4747): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.040650249 (195h35m28.056319864s) [noise=true , address=145.79.227.36:4525 , piece_id=7FBEMPOKGXT2YB5Q6EEYJBFBX35RRCOOTQ2UETDSIVFW34CQZDVA, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX, address:145.79.227.36:4525): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.040699435 (195h35m28.056369021s) [seg_pos={0 0}, upload=187860]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Mar 18 19:46:25.162188571 (195h35m28.177858163s) [upload=187860]: Done uploading segments... Mar 18 19:46:25.162240944 (195h35m28.177910547s) [upload=187860]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration:* 3.345431587s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [..............................................X..X...X..X..X] duration: 1m0.125416446s trace: https://jaeger.dev.storj.io/trace/220541cf08cfd830 FAIL (2026-03-18 19:46:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== .BatchRequestItem_ObjectBegin Mar 18 19:45:25.326452964 (195h34m28.342122568s) [upload=187860]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 19:45:25.456683819 (195h34m28.472353428s) [seg_pos={0 0}, upload=187860]: Begin upload segment... Mar 18 19:45:25.456770594 (195h34m28.472440184s) [seg_pos={0 0}, upload=187860]: Done begin upload segment. Mar 18 19:45:25.456821582 (195h34m28.472491172s) [noise=true , address=145.79.227.36:4525 , piece_id=7FBEMPOKGXT2YB5Q6EEYJBFBX35RRCOOTQ2UETDSIVFW34CQZDVA, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.456867918 (195h34m28.472537526s) [noise=true , address=45.91.116.5:4324 , piece_id=ZFBWMYRD65NBX2FUNBIJF57VRTZOUJEPXOEWHWU63QWBATPWJCDQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.457527075 (195h34m28.47319668s) [noise=true , address=79.127.219.38:4827 , piece_id=2WC4XR64CXG7I7EPU6VFNYUAC76BJ3EKEPKSPRT4I55IYRFQVXLQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.458341322 (195h34m28.474010934s) [noise=true , address=79.127.205.233:4144 , piece_id=C3QUG3TRTWKVN3ZOCEWVNP74WE4QMYA65AHCAKEXOPBPKY4AUZDA, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.458481997 (195h34m28.4741516s) [noise=true , address=145.79.227.17:4993 , piece_id=AN3MJQVU5MVMQLL623PWSLWGH3N242LVPKFOG6AZAUJA6WGMHMFA, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.459053712 (195h34m28.474723318s) [noise=true , address=145.79.227.5:4238 , piece_id=PNXIWK2VZZSCR53VQOBOMP5Z4F5YYRIWWX4CNPOXNRJAB4RWKJLQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.459541420 (195h34m28.475211026s) [noise=true , address=79.127.205.235:4260 , piece_id=VRXKZO7BI77HQO7F33I3EHJZQOS3VCNGH4XZGZEEOHYTM4FOIIDQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.460127026 (195h34m28.475796632s) [noise=true , address=145.79.227.33:4887 , piece_id=GPFUU4XAWYSCN4AG3VV6AI6BWEGLSHJFN5VGTVFGB3PFHPXOGU5Q, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.460537480 (195h34m28.476207086s) [noise=true , address=145.79.227.20:4472 , piece_id=6PS3R5ANRNMQFWPH3F2MWMIVTP53URHAQ7E3NLX72P6U6PGCLQTQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.461390949 (195h34m28.477060561s) [noise=true , address=193.149.164.68:4039 , piece_id=FLJZMLJXBWFG7FOEHWTVDJU2NTW2GZPHNK5S2MIT5K2EBEMACKSQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.461470344 (195h34m28.477139959s) [noise=true , address=79.127.205.234:4206 , piece_id=FCSOPKSPRRYZ5E6RDK72WZVVAWTOD53MYSUST63NR52RY6QDHMKQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.462470296 (195h34m28.478139902s) [noise=true , address=145.79.227.11:4747 , piece_id=JTWQJUGUKHAVWGG6T2ZVJHZ4R5FZTTYFYKXXRD3MH4ZBLEOD4MRQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.462553305 (195h34m28.478222915s) [noise=true , address=77.90.31.17:4782 , piece_id=7ZJZOZOLIXGWYAGAV65WIEY76DOCM6CSE2HAUWXRPKHCLVOQQFEQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.463505826 (195h34m28.479175427s) [noise=true , address=77.90.31.10:4537 , piece_id=N4UDTJUBLM6NLN2NNE3YYKAYHMCNGQWLX4R6TZ35UPDONZ7NMIOA, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.464069344 (195h34m28.479738951s) [noise=true , address=167.17.72.69:4716 , piece_id=3ETXEX7UWPM4DU3DL2EGA5Z4QLKUKHAT23KG3PD5YL6XEZCJTGRA, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.465380624 (195h34m28.481050227s) [noise=true , address=145.79.227.2:4135 , piece_id=DHYVS2YDQW7FQANTEAPJKWBCNLUWPZNANCHFNDWLGFHPVOYPVTJQ, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.465888656 (195h34m28.481558265s) [noise=true , address=145.79.227.44:4206 , piece_id=L42NBORHYROKARBF6Z7CUJQ4SO5IA5SOBRA3TJWOYZDZKTSNXE4Q, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:25.466308049 (195h34m28.481977655s) [noise=true , address=79.127.219.46:4920 , piece_id=76VAJBNCJC4WAYXSEEQGZW7BEGGULKJFLV75UOBDBAVII7AKR62A, seg_pos={0 0}, upload=187860]: Uploading piece... Mar 18 19:45:26.019613415 (195h34m29.035283015s) [upload=187860]: Next returned nil segment Mar 18 19:45:26.019627850 (195h34m29.035297451s) [upload=187860]: Waiting for error group managing segments... Mar 18 19:45:26.193532210 (195h34m29.209201815s) [noise=true , address=45.91.116.5:4324 , piece_id=ZFBWMYRD65NBX2FUNBIJF57VRTZOUJEPXOEWHWU63QWBATPWJCDQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.218509951 (195h34m29.234179558s) [noise=true , address=79.127.219.38:4827 , piece_id=2WC4XR64CXG7I7EPU6VFNYUAC76BJ3EKEPKSPRT4I55IYRFQVXLQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.219073762 (195h34m29.234743372s) [noise=true , address=77.90.31.10:4537 , piece_id=N4UDTJUBLM6NLN2NNE3YYKAYHMCNGQWLX4R6TZ35UPDONZ7NMIOA, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.271321232 (195h34m29.286990847s) [noise=true , address=79.127.219.46:4920 , piece_id=76VAJBNCJC4WAYXSEEQGZW7BEGGULKJFLV75UOBDBAVII7AKR62A, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.293176147 (195h34m29.308845754s) [noise=true , address=77.90.31.17:4782 , piece_id=7ZJZOZOLIXGWYAGAV65WIEY76DOCM6CSE2HAUWXRPKHCLVOQQFEQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.662025397 (195h34m29.677694999s) [noise=true , address=167.17.72.69:4716 , piece_id=3ETXEX7UWPM4DU3DL2EGA5Z4QLKUKHAT23KG3PD5YL6XEZCJTGRA, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.879968454 (195h34m29.895638055s) [noise=true , address=79.127.205.233:4144 , piece_id=C3QUG3TRTWKVN3ZOCEWVNP74WE4QMYA65AHCAKEXOPBPKY4AUZDA, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.898670905 (195h34m29.914340506s) [noise=true , address=79.127.205.235:4260 , piece_id=VRXKZO7BI77HQO7F33I3EHJZQOS3VCNGH4XZGZEEOHYTM4FOIIDQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.903771534 (195h34m29.919441134s) [noise=true , address=193.149.164.68:4039 , piece_id=FLJZMLJXBWFG7FOEHWTVDJU2NTW2GZPHNK5S2MIT5K2EBEMACKSQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:45:26.993657693 (195h34m30.009327292s) [noise=true , address=79.127.205.234:4206 , piece_id=FCSOPKSPRRYZ5E6RDK72WZVVAWTOD53MYSUST63NR52RY6QDHMKQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: <nil> Mar 18 19:46:25.039034405 (195h35m28.054704015s) [noise=true , address=145.79.227.2:4135 , piece_id=DHYVS2YDQW7FQANTEAPJKWBCNLUWPZNANCHFNDWLGFHPVOYPVTJQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:1uEBWRaMH4mLApUcgHyHQmGxxbyXnYUScg5Z2on9LHESYkqj3S, address:145.79.227.2:4135): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.039411284 (195h35m28.055080905s) [noise=true , address=145.79.227.44:4206 , piece_id=L42NBORHYROKARBF6Z7CUJQ4SO5IA5SOBRA3TJWOYZDZKTSNXE4Q, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:12Mp4ZzYgrP3ZSXDeHbSTXaAJMPgGKUqVjdNtyfeKE5qBMAzBke, address:145.79.227.44:4206): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.039509444 (195h35m28.055179057s) [noise=true , address=145.79.227.5:4238 , piece_id=PNXIWK2VZZSCR53VQOBOMP5Z4F5YYRIWWX4CNPOXNRJAB4RWKJLQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:122AVjaN3YaiABnUjuZwSb6apPDeszZAWvmzwtP47k1fxsKLD5V, address:145.79.227.5:4238): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.039966227 (195h35m28.055635833s) [noise=true , address=145.79.227.33:4887 , piece_id=GPFUU4XAWYSCN4AG3VV6AI6BWEGLSHJFN5VGTVFGB3PFHPXOGU5Q, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:1orvhuWfkUGCykWsS2E5sHtMcEW3fFcCNzTQ4xSxqdhUrBza8H, address:145.79.227.33:4887): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.040027384 (195h35m28.05569699s) [noise=true , address=145.79.227.17:4993 , piece_id=AN3MJQVU5MVMQLL623PWSLWGH3N242LVPKFOG6AZAUJA6WGMHMFA, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD, address:145.79.227.17:4993): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.040345443 (195h35m28.056015047s) [noise=true , address=145.79.227.20:4472 , piece_id=6PS3R5ANRNMQFWPH3F2MWMIVTP53URHAQ7E3NLX72P6U6PGCLQTQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:12GACnovbrme3fQknSDpZWVLq838VKop9mPjxnVXGdJ8KaqyaTi, address:145.79.227.20:4472): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.040426406 (195h35m28.056095999s) [noise=true , address=145.79.227.11:4747 , piece_id=JTWQJUGUKHAVWGG6T2ZVJHZ4R5FZTTYFYKXXRD3MH4ZBLEOD4MRQ, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:12mkHFDS6YCheghJdaJFybhcv9yhV4Acfs1RhnCW8yzA23neFRC, address:145.79.227.11:4747): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.040650249 (195h35m28.056319864s) [noise=true , address=145.79.227.36:4525 , piece_id=7FBEMPOKGXT2YB5Q6EEYJBFBX35RRCOOTQ2UETDSIVFW34CQZDVA, seg_pos={0 0}, upload=187860]: Done uploading piece. err: ecclient: upload failed (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX, address:145.79.227.36:4525): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:46:25.040699435 (195h35m28.056369021s) [seg_pos={0 0}, upload=187860]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Mar 18 19:46:25.162188571 (195h35m28.177858163s) [upload=187860]: Done uploading segments... Mar 18 19:46:25.162240944 (195h35m28.177910547s) [upload=187860]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration:* 1m0.125527886s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [.............................................XXXXXXXXXXXXXXX] duration: 30.110084174s trace: https://jaeger.dev.storj.io/trace/1ed3238af2ea26b9 FAIL (2026-03-18 19:46:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 445 (195h34m56.896797031s) [seg_pos={0 0}, upload=187868]: Done begin upload segment. Mar 18 19:45:53.881153808 (195h34m56.896823409s) [noise=true , address=145.79.227.44:4195 , piece_id=2POMXM2VVUC5U7MQKU7HLHOO7FN4AWP7WNBAT5X7WR6DYHOJFA5Q, seg_pos={0 0}, upload=187868]: Uploading piece... Mar 18 19:45:53.881212205 (195h34m56.896881811s) [noise=true , address=77.90.31.10:4521 , piece_id=L7YSFY6UQNM47XDUPGB25655BCNEXGUSIIAMTFQGFFJYJFZFA76Q, seg_pos={0 0}, upload=187868]: Uploading piece... Mar 18 19:45:53.881875449 (195h34m56.897545112s) [noise=true , address=145.79.227.42:4123 , piece_id=KGHLO5YC3T6M4YS2MHLUM2CNWYSHLCFJRR7F26QSSQ3CFQO2ZJ2Q, seg_pos={0 0}, upload=187868]: Uploading piece... Mar 18 19:45:53.882273460 (195h34m56.897943061s) [noise=true , address=145.79.227.11:4733 , piece_id=7YJ27DFIBMOT7GG4EDQOA7GVSQABWQHQQVW4PCZQNGOH2TYIUNZA, seg_pos={0 0}, upload=187868]: Uploading piece... Mar 18 19:45:53.882951396 (195h34m56.898620999s) [noise=true , address=77.90.31.14:4663 , piece_id=GPRPLD4HIYLLNTHS4ULQTB5NC57I4YYIHCLBLRNNZZRU36WJA3NA, seg_pos={0 0}, upload=187868]: Uploading piece... Mar 18 19:45:53.883316758 (195h34m56.898986357s) [noise=true , address=145.79.227.37:4566 , piece_id=7OISTCQNMLG6UPGKB7BWMNZ4LHI5KWLXL6WKXU4X3LI7H3ANATLA, seg_pos={0 0}, upload=187868]: Uploading piece... Mar 18 19:45:53.883877818 (195h34m56.89954742s) [noise=true , address=145.79.227.47:4270 , piece_id=ZWEWBFAG5WA6LFIQIFUF4O34MLIEEFF62SPQ3Z75PCAKL2OJ2HQA, seg_pos={0 0}, upload=187868]: Uploading piece... Mar 18 19:45:53.884159518 (195h34m56.899829118s) [noise=true , address=145.79.227.5:4221 , piece_id=KTLFVD57ES6BTDFDLWLBF2DA2RX376OZUFDL4G36QMF6TILQSZ5A, seg_pos={0 0}, upload=187868]: Uploading piece... Mar 18 19:45:53.884822235 (195h34m56.900491835s) [noise=true , address=145.79.227.36:4517 , piece_id=3LIOPYQ572M2J532BUA7HTZUTI6KAVPDCZJYKDOFCDWWHNZJFRVA, seg_pos={0 0}, upload=187868]: Uploading piece... Mar 18 19:45:53.885044277 (195h34m56.900713886s) [noise=true , address=145.79.227.6:4262 , piece_id=JR7HW2JWYDTP5F4N7HUWY4YYEVXPSZM4JJCWRBWXUMC7ZW2VWJVA, seg_pos={0 0}, upload=187868]: Uploading piece... Mar 18 19:45:53.885751686 (195h34m56.901421295s) [noise=true , address=145.79.227.38:4980 , piece_id=LLJKMB5UMJTZXMA4NJPVSK4SSTXXHDXLHZFFWUZQUSKPJ3NPTGYQ, seg_pos={0 0}, upload=187868]: Uploading piece... Mar 18 19:45:53.885863428 (195h34m56.901533035s) [noise=true , address=77.90.31.13:4627 , piece_id=OK6LGVNYZ7KAA2RPFTG5IRSV2JRF63OCQ2U4ULIVD5YXHB6ZJOSQ, seg_pos={0 0}, upload=187868]: Uploading piece... Mar 18 19:45:53.886555777 (195h34m56.902225385s) [noise=true , address=145.79.227.20:4443 , piece_id=LS3IAM4XH5PBHLSH63P44RA5VN5N3PYJMZG2LZY7J2UHIEIXHD4Q, seg_pos={0 0}, upload=187868]: Uploading piece... Mar 18 19:45:53.886841253 (195h34m56.902510864s) [noise=true , address=77.90.31.19:4845 , piece_id=O36T5EF5KQJTSKLN5EEO2SAPNEOGQG4OSDKC2H4ANYAIS4L5S7JA, seg_pos={0 0}, upload=187868]: Uploading piece... Mar 18 19:45:53.887233316 (195h34m56.902902922s) [noise=true , address=145.79.227.43:4157 , piece_id=NHAUOS4YX4OJEHX7RCYQ4JPHQWPPMZQJ5EBQE2Z54KJYKUVTX4NA, seg_pos={0 0}, upload=187868]: Uploading piece... Mar 18 19:45:53.887629422 (195h34m56.903299029s) [noise=true , address=45.91.116.9:4295 , piece_id=OFO3KBKGDMADESL6Y2L37BKDCEKTG4WBIF54DK4HSWTY7HDOAKZQ, seg_pos={0 0}, upload=187868]: Uploading piece... Mar 18 19:45:54.476239008 (195h34m57.491908608s) [upload=187868]: Next returned nil segment Mar 18 19:45:54.476260235 (195h34m57.491929838s) [upload=187868]: Waiting for error group managing segments... Mar 18 19:45:54.629492998 (195h34m57.645162606s) [noise=true , address=77.90.31.10:4521 , piece_id=L7YSFY6UQNM47XDUPGB25655BCNEXGUSIIAMTFQGFFJYJFZFA76Q, seg_pos={0 0}, upload=187868]: Done uploading piece. err: <nil> Mar 18 19:45:54.643417060 (195h34m57.659086663s) [noise=true , address=45.91.116.9:4295 , piece_id=OFO3KBKGDMADESL6Y2L37BKDCEKTG4WBIF54DK4HSWTY7HDOAKZQ, seg_pos={0 0}, upload=187868]: Done uploading piece. err: <nil> Mar 18 19:45:54.717901997 (195h34m57.733571606s) [noise=true , address=77.90.31.19:4845 , piece_id=O36T5EF5KQJTSKLN5EEO2SAPNEOGQG4OSDKC2H4ANYAIS4L5S7JA, seg_pos={0 0}, upload=187868]: Done uploading piece. err: <nil> Mar 18 19:45:54.727767963 (195h34m57.743437562s) [noise=true , address=77.90.31.13:4627 , piece_id=OK6LGVNYZ7KAA2RPFTG5IRSV2JRF63OCQ2U4ULIVD5YXHB6ZJOSQ, seg_pos={0 0}, upload=187868]: Done uploading piece. err: <nil> Mar 18 19:45:54.754680328 (195h34m57.770349923s) [noise=true , address=77.90.31.14:4663 , piece_id=GPRPLD4HIYLLNTHS4ULQTB5NC57I4YYIHCLBLRNNZZRU36WJA3NA, seg_pos={0 0}, upload=187868]: Done uploading piece. err: <nil> Mar 18 19:46:23.479280568 (195h35m26.494950172s) [noise=true , address=145.79.227.37:4566 , piece_id=7OISTCQNMLG6UPGKB7BWMNZ4LHI5KWLXL6WKXU4X3LI7H3ANATLA, seg_pos={0 0}, upload=187868]: Done uploading piece. err: ecclient: upload failed (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F, address:145.79.227.37:4566): protocol: rpc: dial tcp 145.79.227.37:4566: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:46:23.479812889 (195h35m26.495482496s) [noise=true , address=145.79.227.5:4221 , piece_id=KTLFVD57ES6BTDFDLWLBF2DA2RX376OZUFDL4G36QMF6TILQSZ5A, seg_pos={0 0}, upload=187868]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: rpc: dial tcp 145.79.227.5:4221: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:46:23.480098543 (195h35m26.495768153s) [noise=true , address=145.79.227.47:4270 , piece_id=ZWEWBFAG5WA6LFIQIFUF4O34MLIEEFF62SPQ3Z75PCAKL2OJ2HQA, seg_pos={0 0}, upload=187868]: Done uploading piece. err: ecclient: upload failed (node:12sJfMr1nMTpizd5PZ2bZ6M8tZWhFL2ukdJ3cNSRQtdUHNJhg81, address:145.79.227.47:4270): protocol: rpc: dial tcp 145.79.227.47:4270: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:46:23.480182081 (195h35m26.495851669s) [noise=true , address=145.79.227.36:4517 , piece_id=3LIOPYQ572M2J532BUA7HTZUTI6KAVPDCZJYKDOFCDWWHNZJFRVA, seg_pos={0 0}, upload=187868]: Done uploading piece. err: ecclient: upload failed (node:12HZjKTnjpeNU5B6muVHGrmWQS8GsyuYTW6cLxzNk5CGt3b8PJS, address:145.79.227.36:4517): protocol: rpc: dial tcp 145.79.227.36:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:46:23.480403491 (195h35m26.496073092s) [noise=true , address=145.79.227.38:4980 , piece_id=LLJKMB5UMJTZXMA4NJPVSK4SSTXXHDXLHZFFWUZQUSKPJ3NPTGYQ, seg_pos={0 0}, upload=187868]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: rpc: dial tcp 145.79.227.38:4980: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:46:23.480478411 (195h35m26.496148012s) [noise=true , address=145.79.227.20:4443 , piece_id=LS3IAM4XH5PBHLSH63P44RA5VN5N3PYJMZG2LZY7J2UHIEIXHD4Q, seg_pos={0 0}, upload=187868]: Done uploading piece. err: ecclient: upload failed (node:1rTVUAPUj4PUy8qHuutnB2WYCVpJdiVwfutA6UK4TZoPzvQEUv, address:145.79.227.20:4443): protocol: rpc: dial tcp 145.79.227.20:4443: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:46:23.480493478 (195h35m26.496163106s) [noise=true , address=145.79.227.43:4157 , piece_id=NHAUOS4YX4OJEHX7RCYQ4JPHQWPPMZQJ5EBQE2Z54KJYKUVTX4NA, seg_pos={0 0}, upload=187868]: Done uploading piece. err: ecclient: upload failed (node:1ZrzADk2mKG4R9tjxUKjQeJvb4WmfjLksGFyvstHRfYbBC7XNz, address:145.79.227.43:4157): protocol: rpc: dial tcp 145.79.227.43:4157: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:46:23.480506495 (195h35m26.496176096s) [noise=true , address=145.79.227.6:4262 , piece_id=JR7HW2JWYDTP5F4N7HUWY4YYEVXPSZM4JJCWRBWXUMC7ZW2VWJVA, seg_pos={0 0}, upload=187868]: Done uploading piece. err: ecclient: upload failed (node:12nPw17QDTyrgDBBNKuunFWuDqPV5Rd2GpM7Loc6Ws9vAAZz22s, address:145.79.227.6:4262): protocol: rpc: dial tcp 145.79.227.6:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:46:23.480905098 (195h35m26.496574732s) [noise=true , address=145.79.227.42:4123 , piece_id=KGHLO5YC3T6M4YS2MHLUM2CNWYSHLCFJRR7F26QSSQ3CFQO2ZJ2Q, seg_pos={0 0}, upload=187868]: Done uploading piece. err: ecclient: upload failed (node:12ttjngUh34tGJpCBgVCLfqrWwsdmd88XKWPrmaJrgkMu8htkKM, address:145.79.227.42:4123): protocol: rpc: dial tcp 145.79.227.42:4123: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:46:23.481225575 (195h35m26.49689519s) [noise=true , address=145.79.227.11:4733 , piece_id=7YJ27DFIBMOT7GG4EDQOA7GVSQABWQHQQVW4PCZQNGOH2TYIUNZA, seg_pos={0 0}, upload=187868]: Done uploading piece. err: ecclient: upload failed (node:12T7L64rXL1enKykmSocFM5ZfVeVkKJWgrHVXWbkxayuuW2FXce, address:145.79.227.11:4733): protocol: rpc: dial tcp 145.79.227.11:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:46:23.481507499 (195h35m26.49717711s) [noise=true , address=145.79.227.44:4195 , piece_id=2POMXM2VVUC5U7MQKU7HLHOO7FN4AWP7WNBAT5X7WR6DYHOJFA5Q, seg_pos={0 0}, upload=187868]: Done uploading piece. err: ecclient: upload failed (node:12cizR87omcPERJsRZt64xHfeQyw6BUpBsJPB2txS4LjZR9StMa, address:145.79.227.44:4195): protocol: rpc: dial tcp 145.79.227.44:4195: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:46:23.481541805 (195h35m26.497211393s) [seg_pos={0 0}, upload=187868]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Mar 18 19:46:23.587648173 (195h35m26.603317779s) [upload=187868]: Done uploading segments... Mar 18 19:46:23.587708418 (195h35m26.603378023s) [upload=187868]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 30.110114265s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 12 out of 60 is failed (threshold=6) [...........................................XXXXXXXX....XX.XX] duration: 5.001255516s trace: https://jaeger.dev.storj.io/trace/45066f27372df44f FAIL (2026-03-18 19:46:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.00131928s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [..............................................XXX.XXX.......] duration: 3.049204777s trace: https://jaeger.dev.storj.io/trace/4d6a8ab787f335c0 FAIL (2026-03-18 19:46:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.049265573s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [...............................................XXX.XXX......] duration: 3.622888497s trace: https://jaeger.dev.storj.io/trace/4d6a8ab787f335c0 FAIL (2026-03-18 19:45:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.622915002s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 11 out of 60 is failed (threshold=6) [............................................XXXXXXXX....XX.X] duration: 5.001706252s trace: https://jaeger.dev.storj.io/trace/19e7e7e32fcad7 FAIL (2026-03-18 19:45:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001754818s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [..............................................XXXXXXXXXXXXXX] duration: 30.108336561s trace: https://jaeger.dev.storj.io/trace/4064dde73875f604 FAIL (2026-03-18 19:45:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 6401 (195h33m46.022365991s) [seg_pos={0 0}, upload=187849]: Done begin upload segment. Mar 18 19:44:43.006745086 (195h33m46.022414689s) [noise=true , address=145.79.227.31:4803 , piece_id=4B3IHNQU4ZTY4JNPRKEPBNLCH2KS6FKIJ6AHE34YGAMPKLTO7VBA, seg_pos={0 0}, upload=187849]: Uploading piece... Mar 18 19:44:43.006760792 (195h33m46.022430392s) [noise=true , address=145.79.227.21:4476 , piece_id=2U6QT45GGINBVYTH7UDJQS7VLSLTA6LMQFXAR3QMJHYUKZC6KO2Q, seg_pos={0 0}, upload=187849]: Uploading piece... Mar 18 19:44:43.007293871 (195h33m46.022963499s) [noise=true , address=145.79.227.50:4374 , piece_id=HPIEAFXBUT43KXMXVPYX2BX72EVOHFAMDDMFHF77ERLWPVTOCZNA, seg_pos={0 0}, upload=187849]: Uploading piece... Mar 18 19:44:43.007447718 (195h33m46.023117326s) [noise=true , address=45.91.116.9:4261 , piece_id=DSBU3SQWN3Z3ZY2OW46NTWCR7OBFJOWJZBRRRZ5H6L2XLZBWPWJQ, seg_pos={0 0}, upload=187849]: Uploading piece... Mar 18 19:44:43.007925870 (195h33m46.023595473s) [noise=true , address=145.79.227.35:4481 , piece_id=3IB3JWPWCDXZZQAYN7UJJ4XBJQ5Z2G7GYNRXKGJPUS73KDAJTCDA, seg_pos={0 0}, upload=187849]: Uploading piece... Mar 18 19:44:43.008274832 (195h33m46.023944435s) [noise=true , address=145.79.227.30:4765 , piece_id=ZGKOLXZ3IGVMT6QKZJISLTVDYWXCVJTB4CYNZPSRWEM2VZKXOPLA, seg_pos={0 0}, upload=187849]: Uploading piece... Mar 18 19:44:43.008600816 (195h33m46.024270418s) [noise=true , address=145.79.227.23:4551 , piece_id=DCHL7ALBT7I3CS4K2JEEVHL6MAN3OUT4ZBNSKNTS7SN3L77HB3RQ, seg_pos={0 0}, upload=187849]: Uploading piece... Mar 18 19:44:43.008957474 (195h33m46.024627077s) [noise=true , address=145.79.227.13:4806 , piece_id=4HYSCSMZ4JPQOGHDY3WNZLT5JNUJ33WBUOLYKF4SB77M24GWVWEA, seg_pos={0 0}, upload=187849]: Uploading piece... Mar 18 19:44:43.009278929 (195h33m46.024948532s) [noise=true , address=145.79.227.51:4410 , piece_id=TK7J6SXFXDIIVLHSCZRCQ4YSAY4TS6RIMULL4FUJIVT65R3YSJMQ, seg_pos={0 0}, upload=187849]: Uploading piece... Mar 18 19:44:43.009656041 (195h33m46.02532565s) [noise=true , address=145.79.227.3:4155 , piece_id=Q3UM2OEKLUG55ZY6FUUEUUIDAQ7DLX7HMXE7OQ2ID6AVACI67OLQ, seg_pos={0 0}, upload=187849]: Uploading piece... Mar 18 19:44:43.010119816 (195h33m46.025790028s) [noise=true , address=45.91.116.11:4117 , piece_id=4UJQR52RTYEJZDCVMWHQXECUWMR7KYKHTYESOJMM7CQATD3ESW6Q, seg_pos={0 0}, upload=187849]: Uploading piece... Mar 18 19:44:43.010253458 (195h33m46.025923071s) [noise=true , address=77.90.31.11:4555 , piece_id=QXPEQWFP7MWB2N3A7356JSUIXCCHW5RZ6TNL2QWPJWJ7AMFJS53Q, seg_pos={0 0}, upload=187849]: Uploading piece... Mar 18 19:44:43.010523328 (195h33m46.026192922s) [noise=true , address=45.91.116.7:4010 , piece_id=SXLSIU7UST36U3VRNTJH7LV5ORXAKZCMFFJ2N4HS47E7LJ6R4T6A, seg_pos={0 0}, upload=187849]: Uploading piece... Mar 18 19:44:43.010639701 (195h33m46.026309305s) [noise=true , address=145.79.227.29:4942 , piece_id=YBOAQOMY4TEPI56OVM7GGOQWPHTIJM2CI2VA47MY22EO53LSEH3A, seg_pos={0 0}, upload=187849]: Uploading piece... Mar 18 19:44:43.011032179 (195h33m46.02670179s) [noise=true , address=145.79.227.34:4905 , piece_id=AHF4TLOFIMJT5II3KXCGQNILFMU4DDPSVFHZUOA6YSSFACEMCNYA, seg_pos={0 0}, upload=187849]: Uploading piece... Mar 18 19:44:43.011500269 (195h33m46.027169879s) [noise=true , address=77.90.31.13:4629 , piece_id=YRLON22AAP4PBTEJBSHLCBCZCOALKQZU5WQCKKNB3JKRTVPW4D7A, seg_pos={0 0}, upload=187849]: Uploading piece... Mar 18 19:44:43.606952739 (195h33m46.622622341s) [upload=187849]: Next returned nil segment Mar 18 19:44:43.606971755 (195h33m46.622641371s) [upload=187849]: Waiting for error group managing segments... Mar 18 19:44:43.745897388 (195h33m46.761566988s) [noise=true , address=45.91.116.7:4010 , piece_id=SXLSIU7UST36U3VRNTJH7LV5ORXAKZCMFFJ2N4HS47E7LJ6R4T6A, seg_pos={0 0}, upload=187849]: Done uploading piece. err: <nil> Mar 18 19:44:43.750600126 (195h33m46.766269717s) [noise=true , address=45.91.116.9:4261 , piece_id=DSBU3SQWN3Z3ZY2OW46NTWCR7OBFJOWJZBRRRZ5H6L2XLZBWPWJQ, seg_pos={0 0}, upload=187849]: Done uploading piece. err: <nil> Mar 18 19:44:43.753837586 (195h33m46.769507207s) [noise=true , address=77.90.31.11:4555 , piece_id=QXPEQWFP7MWB2N3A7356JSUIXCCHW5RZ6TNL2QWPJWJ7AMFJS53Q, seg_pos={0 0}, upload=187849]: Done uploading piece. err: <nil> Mar 18 19:44:43.781822668 (195h33m46.797492274s) [noise=true , address=45.91.116.11:4117 , piece_id=4UJQR52RTYEJZDCVMWHQXECUWMR7KYKHTYESOJMM7CQATD3ESW6Q, seg_pos={0 0}, upload=187849]: Done uploading piece. err: <nil> Mar 18 19:44:43.830438127 (195h33m46.84610773s) [noise=true , address=77.90.31.13:4629 , piece_id=YRLON22AAP4PBTEJBSHLCBCZCOALKQZU5WQCKKNB3JKRTVPW4D7A, seg_pos={0 0}, upload=187849]: Done uploading piece. err: <nil> Mar 18 19:45:12.606849132 (195h34m15.622518744s) [noise=true , address=145.79.227.50:4374 , piece_id=HPIEAFXBUT43KXMXVPYX2BX72EVOHFAMDDMFHF77ERLWPVTOCZNA, seg_pos={0 0}, upload=187849]: Done uploading piece. err: ecclient: upload failed (node:12QRixhWaUwXPdjcWKYHbiknME5JVomWYABY6egDD6VKwaTPB8K, address:145.79.227.50:4374): protocol: rpc: dial tcp 145.79.227.50:4374: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:45:12.606948622 (195h34m15.622618231s) [noise=true , address=145.79.227.3:4155 , piece_id=Q3UM2OEKLUG55ZY6FUUEUUIDAQ7DLX7HMXE7OQ2ID6AVACI67OLQ, seg_pos={0 0}, upload=187849]: Done uploading piece. err: ecclient: upload failed (node:1yiUBiCEpAs5yDmgVfFGuxgW4R53jzMJugo8Piff3p79EEZdWr, address:145.79.227.3:4155): protocol: rpc: dial tcp 145.79.227.3:4155: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:45:12.607143577 (195h34m15.62281318s) [noise=true , address=145.79.227.31:4803 , piece_id=4B3IHNQU4ZTY4JNPRKEPBNLCH2KS6FKIJ6AHE34YGAMPKLTO7VBA, seg_pos={0 0}, upload=187849]: Done uploading piece. err: ecclient: upload failed (node:12GFk3GkhB88UKQdZM8vZ9VLNuuYDrS5ThZjVAdbL2CtTQVMDEB, address:145.79.227.31:4803): protocol: rpc: dial tcp 145.79.227.31:4803: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:45:12.607326377 (195h34m15.622995987s) [noise=true , address=145.79.227.34:4905 , piece_id=AHF4TLOFIMJT5II3KXCGQNILFMU4DDPSVFHZUOA6YSSFACEMCNYA, seg_pos={0 0}, upload=187849]: Done uploading piece. err: ecclient: upload failed (node:12ffKESNSVpXQzbwJinP6vvgE6zoF3RFz2NRUuuJHvd7bvk1gc9, address:145.79.227.34:4905): protocol: rpc: dial tcp 145.79.227.34:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:45:12.607694239 (195h34m15.62336385s) [noise=true , address=145.79.227.29:4942 , piece_id=YBOAQOMY4TEPI56OVM7GGOQWPHTIJM2CI2VA47MY22EO53LSEH3A, seg_pos={0 0}, upload=187849]: Done uploading piece. err: ecclient: upload failed (node:17o881pUMuG7K2dv83k7RkrJCrmM7kgUiWesgam4GEw21KVUn7, address:145.79.227.29:4942): protocol: rpc: dial tcp 145.79.227.29:4942: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:45:12.608039650 (195h34m15.623709252s) [noise=true , address=145.79.227.30:4765 , piece_id=ZGKOLXZ3IGVMT6QKZJISLTVDYWXCVJTB4CYNZPSRWEM2VZKXOPLA, seg_pos={0 0}, upload=187849]: Done uploading piece. err: ecclient: upload failed (node:12SRRQBu9efFwqpKMvLBQhaZ9s3XQXVekdmefCBXuQPgRdik2QT, address:145.79.227.30:4765): protocol: rpc: dial tcp 145.79.227.30:4765: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:45:12.608240629 (195h34m15.623910238s) [noise=true , address=145.79.227.35:4481 , piece_id=3IB3JWPWCDXZZQAYN7UJJ4XBJQ5Z2G7GYNRXKGJPUS73KDAJTCDA, seg_pos={0 0}, upload=187849]: Done uploading piece. err: ecclient: upload failed (node:12bRK8WWmAnWUwzCfHu8JtZgrruYSJCvhnsYTKcaLccJcv5RQjp, address:145.79.227.35:4481): protocol: rpc: dial tcp 145.79.227.35:4481: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:45:12.608287351 (195h34m15.62395696s) [noise=true , address=145.79.227.13:4806 , piece_id=4HYSCSMZ4JPQOGHDY3WNZLT5JNUJ33WBUOLYKF4SB77M24GWVWEA, seg_pos={0 0}, upload=187849]: Done uploading piece. err: ecclient: upload failed (node:12qUyweKQhw15RjewCgKi1oLyCvN3VUB1VMpeGhj2Q2hszvJBsQ, address:145.79.227.13:4806): protocol: rpc: dial tcp 145.79.227.13:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:45:12.608545809 (195h34m15.624215421s) [noise=true , address=145.79.227.23:4551 , piece_id=DCHL7ALBT7I3CS4K2JEEVHL6MAN3OUT4ZBNSKNTS7SN3L77HB3RQ, seg_pos={0 0}, upload=187849]: Done uploading piece. err: ecclient: upload failed (node:12vR5WRMGoYyzYDZzwELFw6kLG6vtKsUeFM9kUhgZDLGKB6vcXs, address:145.79.227.23:4551): protocol: rpc: dial tcp 145.79.227.23:4551: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:45:12.608801675 (195h34m15.624471285s) [noise=true , address=145.79.227.51:4410 , piece_id=TK7J6SXFXDIIVLHSCZRCQ4YSAY4TS6RIMULL4FUJIVT65R3YSJMQ, seg_pos={0 0}, upload=187849]: Done uploading piece. err: ecclient: upload failed (node:12HWX3mC14qJg7WdCetaBtS2GcGQFhzSaqUbKrYFBHZKdyFAAgX, address:145.79.227.51:4410): protocol: rpc: dial tcp 145.79.227.51:4410: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:45:12.609050061 (195h34m15.624719671s) [noise=true , address=145.79.227.21:4476 , piece_id=2U6QT45GGINBVYTH7UDJQS7VLSLTA6LMQFXAR3QMJHYUKZC6KO2Q, seg_pos={0 0}, upload=187849]: Done uploading piece. err: ecclient: upload failed (node:12NKwRvjiEpY7DrbqFGaDHQ8no5YV4cUALnpsvQ8icUPoLvMYen, address:145.79.227.21:4476): protocol: rpc: dial tcp 145.79.227.21:4476: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:45:12.609086929 (195h34m15.62475653s) [seg_pos={0 0}, upload=187849]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Mar 18 19:45:12.713376362 (195h34m15.729045963s) [upload=187849]: Done uploading segments... Mar 18 19:45:12.713418039 (195h34m15.72908764s) [upload=187849]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 30.108460379s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [................................................XXX.XXX.....] duration: 3.338072808s trace: https://jaeger.dev.storj.io/trace/4d6a8ab787f335c0 FAIL (2026-03-18 19:44:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.338165437s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............................................X..X...X..X..] duration: 3.232483472s trace: https://jaeger.dev.storj.io/trace/1d504dbde839e584 FAIL (2026-03-18 19:44:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ad=187818]: Uploading segments... Mar 18 19:42:30.164297246 (195h31m33.179966859s) [upload=187818]: Got next segment. Inline: false Mar 18 19:42:30.164335872 (195h31m33.180005484s) [upload=187818]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 19:42:30.164342115 (195h31m33.180011726s) [upload=187818]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 19:42:30.295740352 (195h31m33.311409959s) [seg_pos={0 0}, upload=187818]: Begin upload segment... Mar 18 19:42:30.295819504 (195h31m33.311489098s) [seg_pos={0 0}, upload=187818]: Done begin upload segment. Mar 18 19:42:30.295854279 (195h31m33.311523884s) [noise=true , address=167.17.72.70:4747 , piece_id=DT7Q6F5FL35GOLB6R3WJUSE5ABMJUX4YOSQFBOEQP4L3GLKFDCSA, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.295928902 (195h31m33.311598512s) [noise=true , address=79.127.205.234:4178 , piece_id=VYRVMFVMHSXGUJKMYG4VCDY5PWCKY3GZOOM7LFSGJT5D6G6XX7VA, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.296687216 (195h31m33.312356824s) [noise=true , address=193.149.164.69:4074 , piece_id=6MEFX5DNI5BZHKOSRSJYUV62AMDE3XNPHRP5PSNJLC7SRO7CCXKQ, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.297153710 (195h31m33.312823316s) [noise=true , address=193.149.164.67:4008 , piece_id=XMMA6UYSOZY7JYR335QRO6SIP2XEGTT4UNKXY76HMG6MSXXLY4LA, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.298321958 (195h31m33.313991566s) [noise=true , address=79.127.219.38:4837 , piece_id=BZPNGMT5X3EFWKEDIMZPG4YHVD6JM6WFW5J6QATLOI6BG6L4XRRA, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.299001863 (195h31m33.31467149s) [noise=true , address=45.91.116.13:4184 , piece_id=OW6D74CRZTL6LYO3A3U6DMP32NJ7KRW3UY7OCQBDHOIFFJ6JBKMQ, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.299642047 (195h31m33.315311653s) [noise=true , address=145.79.227.36:4550 , piece_id=A6XR2NEO2W3JRZJ3GUKLMDAUOBPR4I6GEYF3JKDZ4JE33GCE3AIQ, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.300289082 (195h31m33.315958687s) [noise=true , address=77.90.31.12:4615 , piece_id=QFLHODZO25DLOEZDAHMT63RX3RZRXJ5SZZHKPKHK2W6ZT7SPSVZQ, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.300949020 (195h31m33.316618634s) [noise=true , address=193.149.164.68:4059 , piece_id=X5RS553XBHMHURC7FTV2NQ675FVLZ367PV7R4265PP6ETHZWGSMA, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.301598022 (195h31m33.317267633s) [noise=true , address=77.90.31.14:4676 , piece_id=UZ5Y2AEXU4RTTNCKRW75EISDVD6CVUKPZ4ETIS2KSFAG2XT5PE2Q, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.302260217 (195h31m33.317929827s) [noise=true , address=45.91.116.12:4236 , piece_id=CMZSQBTLTEW32Z6UZQ4N5DLQGCQMOQJ7JDMRF7N2W4M7XMIBSKVQ, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.303604800 (195h31m33.319274407s) [noise=true , address=64.49.13.169:4711 , piece_id=COVJUEOQG643A5AUPWJHK3TX4IR47GDC4Y5M45XBTWMXSNFZ7CAQ, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.304038174 (195h31m33.319707946s) [noise=true , address=145.79.227.25:4641 , piece_id=CGZGZ5TOF6DXNDA6QCUOOVHJKUAMB5HQKM6UIAF36MIF7UN2P3DA, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.304560775 (195h31m33.32023039s) [noise=true , address=167.17.72.71:4788 , piece_id=UP6PNPDDONOBO4PCV6TQQGGFX752JIH4U2UBGRNRHDQE3MRJ2WUA, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.307928965 (195h31m33.323598574s) [noise=true , address=145.79.227.34:4924 , piece_id=FFGMVVL5SZ6MGHXLFCSRINV6ZXLJI6P5VMXAIYIIZC5PPWRAMQ6A, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.308369712 (195h31m33.324039326s) [noise=true , address=145.79.227.41:4117 , piece_id=I33OGI4XUXWR2PJMOWF5T63ZLZA2EHG3GZNKO5JFUITN7BJ73ATA, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.308803746 (195h31m33.324473349s) [noise=true , address=193.149.164.70:4119 , piece_id=N3UUPE3G5KST5ONZ3CKLSXAH3WRAMPRKO4RHUP5NVRMOJMGC5U6Q, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.309322931 (195h31m33.324992538s) [noise=true , address=145.79.227.18:4017 , piece_id=DFHRQXQCJHWCAIOB4PXG7VFAQIY4VZ2EWK4UOGYOCWOPGRO4OOCQ, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.860587835 (195h31m33.876257441s) [upload=187818]: Next returned nil segment Mar 18 19:42:30.860606189 (195h31m33.876275801s) [upload=187818]: Waiting for error group managing segments... Mar 18 19:42:31.016475363 (195h31m34.032144963s) [noise=true , address=45.91.116.13:4184 , piece_id=OW6D74CRZTL6LYO3A3U6DMP32NJ7KRW3UY7OCQBDHOIFFJ6JBKMQ, seg_pos={0 0}, upload=187818]: Done uploading piece. err: <nil> Mar 18 19:42:31.033202195 (195h31m34.048871788s) [noise=true , address=45.91.116.12:4236 , piece_id=CMZSQBTLTEW32Z6UZQ4N5DLQGCQMOQJ7JDMRF7N2W4M7XMIBSKVQ, seg_pos={0 0}, upload=187818]: Done uploading piece. err: <nil> Mar 18 19:42:31.035461867 (195h31m34.051131475s) [noise=true , address=77.90.31.12:4615 , piece_id=QFLHODZO25DLOEZDAHMT63RX3RZRXJ5SZZHKPKHK2W6ZT7SPSVZQ, seg_pos={0 0}, upload=187818]: Done uploading piece. err: <nil> Mar 18 19:42:31.035622532 (195h31m34.051292123s) [noise=true , address=77.90.31.14:4676 , piece_id=UZ5Y2AEXU4RTTNCKRW75EISDVD6CVUKPZ4ETIS2KSFAG2XT5PE2Q, seg_pos={0 0}, upload=187818]: Done uploading piece. err: <nil> Mar 18 19:42:31.097484593 (195h31m34.113154201s) [noise=true , address=79.127.219.38:4837 , piece_id=BZPNGMT5X3EFWKEDIMZPG4YHVD6JM6WFW5J6QATLOI6BG6L4XRRA, seg_pos={0 0}, upload=187818]: Done uploading piece. err: <nil> Mar 18 19:42:31.374543059 (195h31m34.390212652s) [noise=true , address=167.17.72.71:4788 , piece_id=UP6PNPDDONOBO4PCV6TQQGGFX752JIH4U2UBGRNRHDQE3MRJ2WUA, seg_pos={0 0}, upload=187818]: Done uploading piece. err: <nil> Mar 18 19:42:31.522469434 (195h31m34.538139092s) [noise=true , address=193.149.164.69:4074 , piece_id=6MEFX5DNI5BZHKOSRSJYUV62AMDE3XNPHRP5PSNJLC7SRO7CCXKQ, seg_pos={0 0}, upload=187818]: Done uploading piece. err: <nil> Mar 18 19:42:31.529007081 (195h31m34.544676671s) [noise=true , address=64.49.13.169:4711 , piece_id=COVJUEOQG643A5AUPWJHK3TX4IR47GDC4Y5M45XBTWMXSNFZ7CAQ, seg_pos={0 0}, upload=187818]: Done uploading piece. err: <nil> Mar 18 19:42:31.532525886 (195h31m34.548195475s) [noise=true , address=193.149.164.68:4059 , piece_id=X5RS553XBHMHURC7FTV2NQ675FVLZ367PV7R4265PP6ETHZWGSMA, seg_pos={0 0}, upload=187818]: Done uploading piece. err: <nil> Mar 18 19:42:31.535856266 (195h31m34.551525859s) [noise=true , address=193.149.164.67:4008 , piece_id=XMMA6UYSOZY7JYR335QRO6SIP2XEGTT4UNKXY76HMG6MSXXLY4LA, seg_pos={0 0}, upload=187818]: Done uploading piece. err: <nil> Mar 18 19:42:31.651555273 (195h31m34.667224874s) [noise=true , address=79.127.205.234:4178 , piece_id=VYRVMFVMHSXGUJKMYG4VCDY5PWCKY3GZOOM7LFSGJT5D6G6XX7VA, seg_pos={0 0}, upload=187818]: Done uploading piece. err: <nil> Mar 18 19:42:31.845292574 (195h31m34.860962177s) [noise=true , address=193.149.164.70:4119 , piece_id=N3UUPE3G5KST5ONZ3CKLSXAH3WRAMPRKO4RHUP5NVRMOJMGC5U6Q, seg_pos={0 0}, upload=187818]: Done uploading piece. err: <nil> Mar 18 19:42:32.069813118 (195h31m35.085482719s) [noise=true , address=167.17.72.70:4747 , piece_id=DT7Q6F5FL35GOLB6R3WJUSE5ABMJUX4YOSQFBOEQP4L3GLKFDCSA, seg_pos={0 0}, upload=187818]: Done uploading piece. err: <nil> Mar 18 19:43:29.853808856 (195h32m32.869478464s) [noise=true , address=145.79.227.25:4641 , piece_id=CGZGZ5TOF6DXNDA6QCUOOVHJKUAMB5HQKM6UIAF36MIF7UN2P3DA, seg_pos={0 0}, upload=187818]: Done uploading piece. err: ecclient: upload failed (node:1FzqefKwAXK6Cs7uV8peWLN9m7cifXv8Xu7JTxdTPRe4UzenWr, address:145.79.227.25:4641): protocol: rpc: dial tcp 145.79.227.25:4641: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:43:29.853902541 (195h32m32.869572159s) [noise=true , address=145.79.227.41:4117 , piece_id=I33OGI4XUXWR2PJMOWF5T63ZLZA2EHG3GZNKO5JFUITN7BJ73ATA, seg_pos={0 0}, upload=187818]: Done uploading piece. err: ecclient: upload failed (node:1QVEhFpjfLubGsZmoY1XQdDWevLy2xWdjtPq4JnbSYuBgEXgz7, address:145.79.227.41:4117): protocol: rpc: dial tcp 145.79.227.41:4117: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:43:29.854201648 (195h32m32.869871257s) [noise=true , address=145.79.227.34:4924 , piece_id=FFGMVVL5SZ6MGHXLFCSRINV6ZXLJI6P5VMXAIYIIZC5PPWRAMQ6A, seg_pos={0 0}, upload=187818]: Done uploading piece. err: ecclient: upload failed (node:19TbrQjuN1EhBNVtha9EcKPSCAWyttfmHpec6QeF1S5zW6DRZs, address:145.79.227.34:4924): protocol: rpc: dial tcp 145.79.227.34:4924: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:43:29.854317854 (195h32m32.869987468s) [noise=true , address=145.79.227.18:4017 , piece_id=DFHRQXQCJHWCAIOB4PXG7VFAQIY4VZ2EWK4UOGYOCWOPGRO4OOCQ, seg_pos={0 0}, upload=187818]: Done uploading piece. err: ecclient: upload failed (node:12wS9YpUnsjgkdTPSowozp8txmqAhfF4n8s7JVpLb2tdRgQeLw8, address:145.79.227.18:4017): protocol: rpc: dial tcp 145.79.227.18:4017: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:43:29.854642668 (195h32m32.870312283s) [noise=true , address=145.79.227.36:4550 , piece_id=A6XR2NEO2W3JRZJ3GUKLMDAUOBPR4I6GEYF3JKDZ4JE33GCE3AIQ, seg_pos={0 0}, upload=187818]: Done uploading piece. err: ecclient: upload failed (node:1Fz8JANjsHWJkHj7Y2JAkr7NTKqDtcKjRW4kDaYGLJYwgWnxbS, address:145.79.227.36:4550): protocol: rpc: dial tcp 145.79.227.36:4550: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:43:29.854688745 (195h32m32.87035834s) [seg_pos={0 0}, upload=187818]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Mar 18 19:43:29.967337043 (195h32m32.983006644s) [upload=187818]: Done uploading segments... Mar 18 19:43:29.967385303 (195h32m32.983054961s) [upload=187818]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 3.23251204s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 10 out of 60 is failed (threshold=6) [.............................................XXXXXXXX....XX.] duration: 4.129360583s trace: https://jaeger.dev.storj.io/trace/e8be83e2b643f78 FAIL (2026-03-18 19:44:34 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.129386813s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 10 out of 60 is failed (threshold=6) [..............................................XXXXXXXX....XX] duration: 5.000975151s trace: https://jaeger.dev.storj.io/trace/e8be83e2b643f78 FAIL (2026-03-18 19:44:09 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001000191s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [...............................................XXXXXXXXXXXXX] duration: 30.132155252s trace: https://jaeger.dev.storj.io/trace/5618a88508453d92 FAIL (2026-03-18 19:44:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== Mar 18 19:43:38.068143449 (195h32m41.08381305s) [upload=187832]: Starting upload Mar 18 19:43:38.068227506 (195h32m41.083897107s) [upload=187832]: Done starting upload Mar 18 19:43:38.068287270 (195h32m41.083956863s) [upload=187832]: Uploading segments... Mar 18 19:43:38.068725955 (195h32m41.084395555s) [upload=187832]: Got next segment. Inline: false Mar 18 19:43:38.068785580 (195h32m41.084455185s) [upload=187832]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 19:43:38.068798576 (195h32m41.084468176s) [upload=187832]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 19:43:38.188278887 (195h32m41.203948497s) [seg_pos={0 0}, upload=187832]: Begin upload segment... Mar 18 19:43:38.188354919 (195h32m41.204024522s) [noise=true , address=145.79.227.31:4802 , piece_id=VP25QEQSQX5ULTS2WKKAP66FWBIVCO6EW27XT26HEQVMPYOBW7EQ, seg_pos={0 0}, upload=187832]: Uploading piece... Mar 18 19:43:38.190015110 (195h32m41.2056847s) [seg_pos={0 0}, upload=187832]: Done begin upload segment. Mar 18 19:43:38.190090767 (195h32m41.205760384s) [noise=true , address=145.79.227.49:4337 , piece_id=AX7L2K2E5EJ2N7JXX42VO3XXYY3PT7Y4MNBKEFKDLFSVYOBLQMQA, seg_pos={0 0}, upload=187832]: Uploading piece... Mar 18 19:43:38.190717347 (195h32m41.206386952s) [noise=true , address=145.79.227.20:4439 , piece_id=I3JES3WGTKAVTJ6WCEF2JVQ7CLEKUIMATJWHYYGU4665VO6T4X2Q, seg_pos={0 0}, upload=187832]: Uploading piece... Mar 18 19:43:38.191392628 (195h32m41.207062231s) [noise=true , address=145.79.227.51:4413 , piece_id=FOYTEWXPK6Z2VAEJUOQM7E2FSQCCUQLIR6LQQEFCFR5XEZHEK4EQ, seg_pos={0 0}, upload=187832]: Uploading piece... Mar 18 19:43:38.192004406 (195h32m41.207674011s) [noise=true , address=45.91.116.11:4119 , piece_id=XDVRBLBUGHG4RT6SV4YGNPFX3GZ4H6KUNGE6VUGGDS3U45TYDZFA, seg_pos={0 0}, upload=187832]: Uploading piece... Mar 18 19:43:38.192575458 (195h32m41.208245064s) [noise=true , address=79.127.219.45:4853 , piece_id=WEQP3HIL4OUKFKDWGOKSAKZ6TIUCJOTQKUQAZ63O3P36YUAGTU2A, seg_pos={0 0}, upload=187832]: Uploading piece... Mar 18 19:43:38.193108991 (195h32m41.208778594s) [noise=true , address=45.91.116.10:4085 , piece_id=QUHYLV4ZORFTPNBVX3CJASWHUV4YACETAMSOQ2W65S2BEGDNXYPQ, seg_pos={0 0}, upload=187832]: Uploading piece... Mar 18 19:43:38.194977253 (195h32m41.210646859s) [noise=true , address=45.91.116.14:4216 , piece_id=NFC2CVN7IKJONK7BFWYMSWCLAIADRJQXNWOJIACH2GASOBC7LZJA, seg_pos={0 0}, upload=187832]: Uploading piece... Mar 18 19:43:38.196928794 (195h32m41.212598403s) [noise=true , address=145.79.227.52:4448 , piece_id=A67BR2V2GKWWLVCQG6453SQ42NTUUC6B2EQVRP7MKDNQZVQE6GFQ, seg_pos={0 0}, upload=187832]: Uploading piece... Mar 18 19:43:38.198177411 (195h32m41.21384702s) [noise=true , address=79.127.219.46:4891 , piece_id=2JH6GFYCDUSXU4VCDVUJNVSONHOUZN4NTMVEZ7ABCS7H6MQOF6WA, seg_pos={0 0}, upload=187832]: Uploading piece... Mar 18 19:43:38.201565738 (195h32m41.217235347s) [noise=true , address=45.91.116.5:4297 , piece_id=7V4GPHXSQUWWWF4BKRHDHI2WRSYB4OGLNKFBXW3KTQII5ULO4QPA, seg_pos={0 0}, upload=187832]: Uploading piece... Mar 18 19:43:38.204527246 (195h32m41.220196848s) [noise=true , address=45.91.116.8:4047 , piece_id=LPMSQ7256LFVCXKEXD2KP4ODQ5ZWPZKGOAEE5KAXIP3ROB2ZEFRA, seg_pos={0 0}, upload=187832]: Uploading piece... Mar 18 19:43:38.206318228 (195h32m41.221987834s) [noise=true , address=145.79.227.3:4152 , piece_id=XVSLIBA6LP3SM5XZG3G7HJD3GJ2LP5WMJUJS6Q3PGOMTE3P3BKOA, seg_pos={0 0}, upload=187832]: Uploading piece... Mar 18 19:43:38.208088825 (195h32m41.223758434s) [noise=true , address=77.90.31.21:4878 , piece_id=6MDB2DAUH4Q44AGS4ZIDW5RPPI3EQF7EFVI2CBW3MNSIMRUIL7PA, seg_pos={0 0}, upload=187832]: Uploading piece... Mar 18 19:43:38.208456366 (195h32m41.224125966s) [noise=true , address=145.79.227.38:4979 , piece_id=AAJ4ADDNDJBTQHYKGRN2L5CQKKCVPG4RHWZVFF47224GO3PDQ46Q, seg_pos={0 0}, upload=187832]: Uploading piece... Mar 18 19:43:38.211811123 (195h32m41.227480717s) [noise=true , address=145.79.227.34:4905 , piece_id=UVJEY6YGB3ILRY6SKLR463ZPGYG7L3NVH72QV6ZVXVOAPOZVXZEA, seg_pos={0 0}, upload=187832]: Uploading piece... Mar 18 19:43:38.809618281 (195h32m41.825287881s) [upload=187832]: Next returned nil segment Mar 18 19:43:38.809644442 (195h32m41.825314047s) [upload=187832]: Waiting for error group managing segments... Mar 18 19:43:38.998695831 (195h32m42.014365443s) [noise=true , address=45.91.116.10:4085 , piece_id=QUHYLV4ZORFTPNBVX3CJASWHUV4YACETAMSOQ2W65S2BEGDNXYPQ, seg_pos={0 0}, upload=187832]: Done uploading piece. err: <nil> Mar 18 19:43:38.998902694 (195h32m42.014572298s) [noise=true , address=45.91.116.11:4119 , piece_id=XDVRBLBUGHG4RT6SV4YGNPFX3GZ4H6KUNGE6VUGGDS3U45TYDZFA, seg_pos={0 0}, upload=187832]: Done uploading piece. err: <nil> Mar 18 19:43:38.999248618 (195h32m42.014918217s) [noise=true , address=45.91.116.5:4297 , piece_id=7V4GPHXSQUWWWF4BKRHDHI2WRSYB4OGLNKFBXW3KTQII5ULO4QPA, seg_pos={0 0}, upload=187832]: Done uploading piece. err: <nil> Mar 18 19:43:38.999647139 (195h32m42.015316744s) [noise=true , address=45.91.116.8:4047 , piece_id=LPMSQ7256LFVCXKEXD2KP4ODQ5ZWPZKGOAEE5KAXIP3ROB2ZEFRA, seg_pos={0 0}, upload=187832]: Done uploading piece. err: <nil> Mar 18 19:43:38.999731432 (195h32m42.015401031s) [noise=true , address=45.91.116.14:4216 , piece_id=NFC2CVN7IKJONK7BFWYMSWCLAIADRJQXNWOJIACH2GASOBC7LZJA, seg_pos={0 0}, upload=187832]: Done uploading piece. err: <nil> Mar 18 19:43:39.105078170 (195h32m42.120747773s) [noise=true , address=79.127.219.46:4891 , piece_id=2JH6GFYCDUSXU4VCDVUJNVSONHOUZN4NTMVEZ7ABCS7H6MQOF6WA, seg_pos={0 0}, upload=187832]: Done uploading piece. err: <nil> Mar 18 19:43:39.122737404 (195h32m42.138407014s) [noise=true , address=79.127.219.45:4853 , piece_id=WEQP3HIL4OUKFKDWGOKSAKZ6TIUCJOTQKUQAZ63O3P36YUAGTU2A, seg_pos={0 0}, upload=187832]: Done uploading piece. err: <nil> Mar 18 19:43:39.138321034 (195h32m42.153990625s) [noise=true , address=77.90.31.21:4878 , piece_id=6MDB2DAUH4Q44AGS4ZIDW5RPPI3EQF7EFVI2CBW3MNSIMRUIL7PA, seg_pos={0 0}, upload=187832]: Done uploading piece. err: <nil> Mar 18 19:44:07.787145967 (195h33m10.802815578s) [noise=true , address=145.79.227.3:4152 , piece_id=XVSLIBA6LP3SM5XZG3G7HJD3GJ2LP5WMJUJS6Q3PGOMTE3P3BKOA, seg_pos={0 0}, upload=187832]: Done uploading piece. err: ecclient: upload failed (node:122rsZGFcSywYKfw4DPU7duA9TGtj1iXmxS4ewGGGSXGRf7Q7mX, address:145.79.227.3:4152): protocol: rpc: dial tcp 145.79.227.3:4152: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:44:07.787921731 (195h33m10.803591336s) [noise=true , address=145.79.227.52:4448 , piece_id=A67BR2V2GKWWLVCQG6453SQ42NTUUC6B2EQVRP7MKDNQZVQE6GFQ, seg_pos={0 0}, upload=187832]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: rpc: dial tcp 145.79.227.52:4448: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:44:07.788768602 (195h33m10.80443821s) [noise=true , address=145.79.227.31:4802 , piece_id=VP25QEQSQX5ULTS2WKKAP66FWBIVCO6EW27XT26HEQVMPYOBW7EQ, seg_pos={0 0}, upload=187832]: Done uploading piece. err: ecclient: upload failed (node:1iMEuZv8uHpYjTqP9JVpznKPPdEyyy4xMTsgVUGMuCjZkV4cri, address:145.79.227.31:4802): protocol: rpc: dial tcp 145.79.227.31:4802: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:44:07.789015182 (195h33m10.804684791s) [noise=true , address=145.79.227.20:4439 , piece_id=I3JES3WGTKAVTJ6WCEF2JVQ7CLEKUIMATJWHYYGU4665VO6T4X2Q, seg_pos={0 0}, upload=187832]: Done uploading piece. err: ecclient: upload failed (node:1wgkbc3Mkej7supxhYMtUdBvn9t45i6kmAokAUkqFxEpehMjAn, address:145.79.227.20:4439): protocol: rpc: dial tcp 145.79.227.20:4439: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:44:07.789235056 (195h33m10.804904666s) [noise=true , address=145.79.227.49:4337 , piece_id=AX7L2K2E5EJ2N7JXX42VO3XXYY3PT7Y4MNBKEFKDLFSVYOBLQMQA, seg_pos={0 0}, upload=187832]: Done uploading piece. err: ecclient: upload failed (node:1nTZFfzNCcVAjut7NXf9seifysJDfvXkS8aZSnXKfw6gh1TrjE, address:145.79.227.49:4337): protocol: rpc: dial tcp 145.79.227.49:4337: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:44:07.789596524 (195h33m10.805266134s) [noise=true , address=145.79.227.51:4413 , piece_id=FOYTEWXPK6Z2VAEJUOQM7E2FSQCCUQLIR6LQQEFCFR5XEZHEK4EQ, seg_pos={0 0}, upload=187832]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: rpc: dial tcp 145.79.227.51:4413: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:44:07.790171861 (195h33m10.805841474s) [noise=true , address=145.79.227.34:4905 , piece_id=UVJEY6YGB3ILRY6SKLR463ZPGYG7L3NVH72QV6ZVXVOAPOZVXZEA, seg_pos={0 0}, upload=187832]: Done uploading piece. err: ecclient: upload failed (node:12ffKESNSVpXQzbwJinP6vvgE6zoF3RFz2NRUuuJHvd7bvk1gc9, address:145.79.227.34:4905): protocol: rpc: dial tcp 145.79.227.34:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:44:07.790524532 (195h33m10.806194141s) [noise=true , address=145.79.227.38:4979 , piece_id=AAJ4ADDNDJBTQHYKGRN2L5CQKKCVPG4RHWZVFF47224GO3PDQ46Q, seg_pos={0 0}, upload=187832]: Done uploading piece. err: ecclient: upload failed (node:12B518TvbbRhj6LiXxV4GwUgAoUYmWMWeTpyYerKnKuZYBjXoma, address:145.79.227.38:4979): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:44:07.790560643 (195h33m10.806230237s) [seg_pos={0 0}, upload=187832]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Mar 18 19:44:07.917007797 (195h33m10.932677387s) [upload=187832]: Done uploading segments... Mar 18 19:44:07.917058471 (195h33m10.932728071s) [upload=187832]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 30.132185226s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 9 out of 60 is failed (threshold=6) [...............................................XXXXXXXX....X] duration: 5.001015933s trace: https://jaeger.dev.storj.io/trace/7574dfe2a665391f FAIL (2026-03-18 19:43:40 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001040085s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [.................................................XXX.XXX....] duration: 3.018887091s trace: https://jaeger.dev.storj.io/trace/4d6a8ab787f335c0 FAIL (2026-03-18 19:43:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.018915706s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [................................................X..X...X..X.] duration: 3.685780898s trace: https://jaeger.dev.storj.io/trace/1d504dbde839e584 FAIL (2026-03-18 19:43:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ad=187818]: Uploading segments... Mar 18 19:42:30.164297246 (195h31m33.179966859s) [upload=187818]: Got next segment. Inline: false Mar 18 19:42:30.164335872 (195h31m33.180005484s) [upload=187818]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 19:42:30.164342115 (195h31m33.180011726s) [upload=187818]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 19:42:30.295740352 (195h31m33.311409959s) [seg_pos={0 0}, upload=187818]: Begin upload segment... Mar 18 19:42:30.295819504 (195h31m33.311489098s) [seg_pos={0 0}, upload=187818]: Done begin upload segment. Mar 18 19:42:30.295854279 (195h31m33.311523884s) [noise=true , address=167.17.72.70:4747 , piece_id=DT7Q6F5FL35GOLB6R3WJUSE5ABMJUX4YOSQFBOEQP4L3GLKFDCSA, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.295928902 (195h31m33.311598512s) [noise=true , address=79.127.205.234:4178 , piece_id=VYRVMFVMHSXGUJKMYG4VCDY5PWCKY3GZOOM7LFSGJT5D6G6XX7VA, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.296687216 (195h31m33.312356824s) [noise=true , address=193.149.164.69:4074 , piece_id=6MEFX5DNI5BZHKOSRSJYUV62AMDE3XNPHRP5PSNJLC7SRO7CCXKQ, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.297153710 (195h31m33.312823316s) [noise=true , address=193.149.164.67:4008 , piece_id=XMMA6UYSOZY7JYR335QRO6SIP2XEGTT4UNKXY76HMG6MSXXLY4LA, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.298321958 (195h31m33.313991566s) [noise=true , address=79.127.219.38:4837 , piece_id=BZPNGMT5X3EFWKEDIMZPG4YHVD6JM6WFW5J6QATLOI6BG6L4XRRA, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.299001863 (195h31m33.31467149s) [noise=true , address=45.91.116.13:4184 , piece_id=OW6D74CRZTL6LYO3A3U6DMP32NJ7KRW3UY7OCQBDHOIFFJ6JBKMQ, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.299642047 (195h31m33.315311653s) [noise=true , address=145.79.227.36:4550 , piece_id=A6XR2NEO2W3JRZJ3GUKLMDAUOBPR4I6GEYF3JKDZ4JE33GCE3AIQ, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.300289082 (195h31m33.315958687s) [noise=true , address=77.90.31.12:4615 , piece_id=QFLHODZO25DLOEZDAHMT63RX3RZRXJ5SZZHKPKHK2W6ZT7SPSVZQ, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.300949020 (195h31m33.316618634s) [noise=true , address=193.149.164.68:4059 , piece_id=X5RS553XBHMHURC7FTV2NQ675FVLZ367PV7R4265PP6ETHZWGSMA, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.301598022 (195h31m33.317267633s) [noise=true , address=77.90.31.14:4676 , piece_id=UZ5Y2AEXU4RTTNCKRW75EISDVD6CVUKPZ4ETIS2KSFAG2XT5PE2Q, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.302260217 (195h31m33.317929827s) [noise=true , address=45.91.116.12:4236 , piece_id=CMZSQBTLTEW32Z6UZQ4N5DLQGCQMOQJ7JDMRF7N2W4M7XMIBSKVQ, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.303604800 (195h31m33.319274407s) [noise=true , address=64.49.13.169:4711 , piece_id=COVJUEOQG643A5AUPWJHK3TX4IR47GDC4Y5M45XBTWMXSNFZ7CAQ, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.304038174 (195h31m33.319707946s) [noise=true , address=145.79.227.25:4641 , piece_id=CGZGZ5TOF6DXNDA6QCUOOVHJKUAMB5HQKM6UIAF36MIF7UN2P3DA, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.304560775 (195h31m33.32023039s) [noise=true , address=167.17.72.71:4788 , piece_id=UP6PNPDDONOBO4PCV6TQQGGFX752JIH4U2UBGRNRHDQE3MRJ2WUA, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.307928965 (195h31m33.323598574s) [noise=true , address=145.79.227.34:4924 , piece_id=FFGMVVL5SZ6MGHXLFCSRINV6ZXLJI6P5VMXAIYIIZC5PPWRAMQ6A, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.308369712 (195h31m33.324039326s) [noise=true , address=145.79.227.41:4117 , piece_id=I33OGI4XUXWR2PJMOWF5T63ZLZA2EHG3GZNKO5JFUITN7BJ73ATA, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.308803746 (195h31m33.324473349s) [noise=true , address=193.149.164.70:4119 , piece_id=N3UUPE3G5KST5ONZ3CKLSXAH3WRAMPRKO4RHUP5NVRMOJMGC5U6Q, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.309322931 (195h31m33.324992538s) [noise=true , address=145.79.227.18:4017 , piece_id=DFHRQXQCJHWCAIOB4PXG7VFAQIY4VZ2EWK4UOGYOCWOPGRO4OOCQ, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.860587835 (195h31m33.876257441s) [upload=187818]: Next returned nil segment Mar 18 19:42:30.860606189 (195h31m33.876275801s) [upload=187818]: Waiting for error group managing segments... Mar 18 19:42:31.016475363 (195h31m34.032144963s) [noise=true , address=45.91.116.13:4184 , piece_id=OW6D74CRZTL6LYO3A3U6DMP32NJ7KRW3UY7OCQBDHOIFFJ6JBKMQ, seg_pos={0 0}, upload=187818]: Done uploading piece. err: <nil> Mar 18 19:42:31.033202195 (195h31m34.048871788s) [noise=true , address=45.91.116.12:4236 , piece_id=CMZSQBTLTEW32Z6UZQ4N5DLQGCQMOQJ7JDMRF7N2W4M7XMIBSKVQ, seg_pos={0 0}, upload=187818]: Done uploading piece. err: <nil> Mar 18 19:42:31.035461867 (195h31m34.051131475s) [noise=true , address=77.90.31.12:4615 , piece_id=QFLHODZO25DLOEZDAHMT63RX3RZRXJ5SZZHKPKHK2W6ZT7SPSVZQ, seg_pos={0 0}, upload=187818]: Done uploading piece. err: <nil> Mar 18 19:42:31.035622532 (195h31m34.051292123s) [noise=true , address=77.90.31.14:4676 , piece_id=UZ5Y2AEXU4RTTNCKRW75EISDVD6CVUKPZ4ETIS2KSFAG2XT5PE2Q, seg_pos={0 0}, upload=187818]: Done uploading piece. err: <nil> Mar 18 19:42:31.097484593 (195h31m34.113154201s) [noise=true , address=79.127.219.38:4837 , piece_id=BZPNGMT5X3EFWKEDIMZPG4YHVD6JM6WFW5J6QATLOI6BG6L4XRRA, seg_pos={0 0}, upload=187818]: Done uploading piece. err: <nil> Mar 18 19:42:31.374543059 (195h31m34.390212652s) [noise=true , address=167.17.72.71:4788 , piece_id=UP6PNPDDONOBO4PCV6TQQGGFX752JIH4U2UBGRNRHDQE3MRJ2WUA, seg_pos={0 0}, upload=187818]: Done uploading piece. err: <nil> Mar 18 19:42:31.522469434 (195h31m34.538139092s) [noise=true , address=193.149.164.69:4074 , piece_id=6MEFX5DNI5BZHKOSRSJYUV62AMDE3XNPHRP5PSNJLC7SRO7CCXKQ, seg_pos={0 0}, upload=187818]: Done uploading piece. err: <nil> Mar 18 19:42:31.529007081 (195h31m34.544676671s) [noise=true , address=64.49.13.169:4711 , piece_id=COVJUEOQG643A5AUPWJHK3TX4IR47GDC4Y5M45XBTWMXSNFZ7CAQ, seg_pos={0 0}, upload=187818]: Done uploading piece. err: <nil> Mar 18 19:42:31.532525886 (195h31m34.548195475s) [noise=true , address=193.149.164.68:4059 , piece_id=X5RS553XBHMHURC7FTV2NQ675FVLZ367PV7R4265PP6ETHZWGSMA, seg_pos={0 0}, upload=187818]: Done uploading piece. err: <nil> Mar 18 19:42:31.535856266 (195h31m34.551525859s) [noise=true , address=193.149.164.67:4008 , piece_id=XMMA6UYSOZY7JYR335QRO6SIP2XEGTT4UNKXY76HMG6MSXXLY4LA, seg_pos={0 0}, upload=187818]: Done uploading piece. err: <nil> Mar 18 19:42:31.651555273 (195h31m34.667224874s) [noise=true , address=79.127.205.234:4178 , piece_id=VYRVMFVMHSXGUJKMYG4VCDY5PWCKY3GZOOM7LFSGJT5D6G6XX7VA, seg_pos={0 0}, upload=187818]: Done uploading piece. err: <nil> Mar 18 19:42:31.845292574 (195h31m34.860962177s) [noise=true , address=193.149.164.70:4119 , piece_id=N3UUPE3G5KST5ONZ3CKLSXAH3WRAMPRKO4RHUP5NVRMOJMGC5U6Q, seg_pos={0 0}, upload=187818]: Done uploading piece. err: <nil> Mar 18 19:42:32.069813118 (195h31m35.085482719s) [noise=true , address=167.17.72.70:4747 , piece_id=DT7Q6F5FL35GOLB6R3WJUSE5ABMJUX4YOSQFBOEQP4L3GLKFDCSA, seg_pos={0 0}, upload=187818]: Done uploading piece. err: <nil> Mar 18 19:43:29.853808856 (195h32m32.869478464s) [noise=true , address=145.79.227.25:4641 , piece_id=CGZGZ5TOF6DXNDA6QCUOOVHJKUAMB5HQKM6UIAF36MIF7UN2P3DA, seg_pos={0 0}, upload=187818]: Done uploading piece. err: ecclient: upload failed (node:1FzqefKwAXK6Cs7uV8peWLN9m7cifXv8Xu7JTxdTPRe4UzenWr, address:145.79.227.25:4641): protocol: rpc: dial tcp 145.79.227.25:4641: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:43:29.853902541 (195h32m32.869572159s) [noise=true , address=145.79.227.41:4117 , piece_id=I33OGI4XUXWR2PJMOWF5T63ZLZA2EHG3GZNKO5JFUITN7BJ73ATA, seg_pos={0 0}, upload=187818]: Done uploading piece. err: ecclient: upload failed (node:1QVEhFpjfLubGsZmoY1XQdDWevLy2xWdjtPq4JnbSYuBgEXgz7, address:145.79.227.41:4117): protocol: rpc: dial tcp 145.79.227.41:4117: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:43:29.854201648 (195h32m32.869871257s) [noise=true , address=145.79.227.34:4924 , piece_id=FFGMVVL5SZ6MGHXLFCSRINV6ZXLJI6P5VMXAIYIIZC5PPWRAMQ6A, seg_pos={0 0}, upload=187818]: Done uploading piece. err: ecclient: upload failed (node:19TbrQjuN1EhBNVtha9EcKPSCAWyttfmHpec6QeF1S5zW6DRZs, address:145.79.227.34:4924): protocol: rpc: dial tcp 145.79.227.34:4924: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:43:29.854317854 (195h32m32.869987468s) [noise=true , address=145.79.227.18:4017 , piece_id=DFHRQXQCJHWCAIOB4PXG7VFAQIY4VZ2EWK4UOGYOCWOPGRO4OOCQ, seg_pos={0 0}, upload=187818]: Done uploading piece. err: ecclient: upload failed (node:12wS9YpUnsjgkdTPSowozp8txmqAhfF4n8s7JVpLb2tdRgQeLw8, address:145.79.227.18:4017): protocol: rpc: dial tcp 145.79.227.18:4017: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:43:29.854642668 (195h32m32.870312283s) [noise=true , address=145.79.227.36:4550 , piece_id=A6XR2NEO2W3JRZJ3GUKLMDAUOBPR4I6GEYF3JKDZ4JE33GCE3AIQ, seg_pos={0 0}, upload=187818]: Done uploading piece. err: ecclient: upload failed (node:1Fz8JANjsHWJkHj7Y2JAkr7NTKqDtcKjRW4kDaYGLJYwgWnxbS, address:145.79.227.36:4550): protocol: rpc: dial tcp 145.79.227.36:4550: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:43:29.854688745 (195h32m32.87035834s) [seg_pos={0 0}, upload=187818]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Mar 18 19:43:29.967337043 (195h32m32.983006644s) [upload=187818]: Done uploading segments... Mar 18 19:43:29.967385303 (195h32m32.983054961s) [upload=187818]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 3.685810268s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.................................................X..X...X..X] duration: 1m0.115336587s trace: https://jaeger.dev.storj.io/trace/1d504dbde839e584 FAIL (2026-03-18 19:43:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ad=187818]: Uploading segments... Mar 18 19:42:30.164297246 (195h31m33.179966859s) [upload=187818]: Got next segment. Inline: false Mar 18 19:42:30.164335872 (195h31m33.180005484s) [upload=187818]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 19:42:30.164342115 (195h31m33.180011726s) [upload=187818]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 19:42:30.295740352 (195h31m33.311409959s) [seg_pos={0 0}, upload=187818]: Begin upload segment... Mar 18 19:42:30.295819504 (195h31m33.311489098s) [seg_pos={0 0}, upload=187818]: Done begin upload segment. Mar 18 19:42:30.295854279 (195h31m33.311523884s) [noise=true , address=167.17.72.70:4747 , piece_id=DT7Q6F5FL35GOLB6R3WJUSE5ABMJUX4YOSQFBOEQP4L3GLKFDCSA, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.295928902 (195h31m33.311598512s) [noise=true , address=79.127.205.234:4178 , piece_id=VYRVMFVMHSXGUJKMYG4VCDY5PWCKY3GZOOM7LFSGJT5D6G6XX7VA, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.296687216 (195h31m33.312356824s) [noise=true , address=193.149.164.69:4074 , piece_id=6MEFX5DNI5BZHKOSRSJYUV62AMDE3XNPHRP5PSNJLC7SRO7CCXKQ, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.297153710 (195h31m33.312823316s) [noise=true , address=193.149.164.67:4008 , piece_id=XMMA6UYSOZY7JYR335QRO6SIP2XEGTT4UNKXY76HMG6MSXXLY4LA, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.298321958 (195h31m33.313991566s) [noise=true , address=79.127.219.38:4837 , piece_id=BZPNGMT5X3EFWKEDIMZPG4YHVD6JM6WFW5J6QATLOI6BG6L4XRRA, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.299001863 (195h31m33.31467149s) [noise=true , address=45.91.116.13:4184 , piece_id=OW6D74CRZTL6LYO3A3U6DMP32NJ7KRW3UY7OCQBDHOIFFJ6JBKMQ, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.299642047 (195h31m33.315311653s) [noise=true , address=145.79.227.36:4550 , piece_id=A6XR2NEO2W3JRZJ3GUKLMDAUOBPR4I6GEYF3JKDZ4JE33GCE3AIQ, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.300289082 (195h31m33.315958687s) [noise=true , address=77.90.31.12:4615 , piece_id=QFLHODZO25DLOEZDAHMT63RX3RZRXJ5SZZHKPKHK2W6ZT7SPSVZQ, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.300949020 (195h31m33.316618634s) [noise=true , address=193.149.164.68:4059 , piece_id=X5RS553XBHMHURC7FTV2NQ675FVLZ367PV7R4265PP6ETHZWGSMA, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.301598022 (195h31m33.317267633s) [noise=true , address=77.90.31.14:4676 , piece_id=UZ5Y2AEXU4RTTNCKRW75EISDVD6CVUKPZ4ETIS2KSFAG2XT5PE2Q, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.302260217 (195h31m33.317929827s) [noise=true , address=45.91.116.12:4236 , piece_id=CMZSQBTLTEW32Z6UZQ4N5DLQGCQMOQJ7JDMRF7N2W4M7XMIBSKVQ, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.303604800 (195h31m33.319274407s) [noise=true , address=64.49.13.169:4711 , piece_id=COVJUEOQG643A5AUPWJHK3TX4IR47GDC4Y5M45XBTWMXSNFZ7CAQ, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.304038174 (195h31m33.319707946s) [noise=true , address=145.79.227.25:4641 , piece_id=CGZGZ5TOF6DXNDA6QCUOOVHJKUAMB5HQKM6UIAF36MIF7UN2P3DA, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.304560775 (195h31m33.32023039s) [noise=true , address=167.17.72.71:4788 , piece_id=UP6PNPDDONOBO4PCV6TQQGGFX752JIH4U2UBGRNRHDQE3MRJ2WUA, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.307928965 (195h31m33.323598574s) [noise=true , address=145.79.227.34:4924 , piece_id=FFGMVVL5SZ6MGHXLFCSRINV6ZXLJI6P5VMXAIYIIZC5PPWRAMQ6A, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.308369712 (195h31m33.324039326s) [noise=true , address=145.79.227.41:4117 , piece_id=I33OGI4XUXWR2PJMOWF5T63ZLZA2EHG3GZNKO5JFUITN7BJ73ATA, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.308803746 (195h31m33.324473349s) [noise=true , address=193.149.164.70:4119 , piece_id=N3UUPE3G5KST5ONZ3CKLSXAH3WRAMPRKO4RHUP5NVRMOJMGC5U6Q, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.309322931 (195h31m33.324992538s) [noise=true , address=145.79.227.18:4017 , piece_id=DFHRQXQCJHWCAIOB4PXG7VFAQIY4VZ2EWK4UOGYOCWOPGRO4OOCQ, seg_pos={0 0}, upload=187818]: Uploading piece... Mar 18 19:42:30.860587835 (195h31m33.876257441s) [upload=187818]: Next returned nil segment Mar 18 19:42:30.860606189 (195h31m33.876275801s) [upload=187818]: Waiting for error group managing segments... Mar 18 19:42:31.016475363 (195h31m34.032144963s) [noise=true , address=45.91.116.13:4184 , piece_id=OW6D74CRZTL6LYO3A3U6DMP32NJ7KRW3UY7OCQBDHOIFFJ6JBKMQ, seg_pos={0 0}, upload=187818]: Done uploading piece. err: <nil> Mar 18 19:42:31.033202195 (195h31m34.048871788s) [noise=true , address=45.91.116.12:4236 , piece_id=CMZSQBTLTEW32Z6UZQ4N5DLQGCQMOQJ7JDMRF7N2W4M7XMIBSKVQ, seg_pos={0 0}, upload=187818]: Done uploading piece. err: <nil> Mar 18 19:42:31.035461867 (195h31m34.051131475s) [noise=true , address=77.90.31.12:4615 , piece_id=QFLHODZO25DLOEZDAHMT63RX3RZRXJ5SZZHKPKHK2W6ZT7SPSVZQ, seg_pos={0 0}, upload=187818]: Done uploading piece. err: <nil> Mar 18 19:42:31.035622532 (195h31m34.051292123s) [noise=true , address=77.90.31.14:4676 , piece_id=UZ5Y2AEXU4RTTNCKRW75EISDVD6CVUKPZ4ETIS2KSFAG2XT5PE2Q, seg_pos={0 0}, upload=187818]: Done uploading piece. err: <nil> Mar 18 19:42:31.097484593 (195h31m34.113154201s) [noise=true , address=79.127.219.38:4837 , piece_id=BZPNGMT5X3EFWKEDIMZPG4YHVD6JM6WFW5J6QATLOI6BG6L4XRRA, seg_pos={0 0}, upload=187818]: Done uploading piece. err: <nil> Mar 18 19:42:31.374543059 (195h31m34.390212652s) [noise=true , address=167.17.72.71:4788 , piece_id=UP6PNPDDONOBO4PCV6TQQGGFX752JIH4U2UBGRNRHDQE3MRJ2WUA, seg_pos={0 0}, upload=187818]: Done uploading piece. err: <nil> Mar 18 19:42:31.522469434 (195h31m34.538139092s) [noise=true , address=193.149.164.69:4074 , piece_id=6MEFX5DNI5BZHKOSRSJYUV62AMDE3XNPHRP5PSNJLC7SRO7CCXKQ, seg_pos={0 0}, upload=187818]: Done uploading piece. err: <nil> Mar 18 19:42:31.529007081 (195h31m34.544676671s) [noise=true , address=64.49.13.169:4711 , piece_id=COVJUEOQG643A5AUPWJHK3TX4IR47GDC4Y5M45XBTWMXSNFZ7CAQ, seg_pos={0 0}, upload=187818]: Done uploading piece. err: <nil> Mar 18 19:42:31.532525886 (195h31m34.548195475s) [noise=true , address=193.149.164.68:4059 , piece_id=X5RS553XBHMHURC7FTV2NQ675FVLZ367PV7R4265PP6ETHZWGSMA, seg_pos={0 0}, upload=187818]: Done uploading piece. err: <nil> Mar 18 19:42:31.535856266 (195h31m34.551525859s) [noise=true , address=193.149.164.67:4008 , piece_id=XMMA6UYSOZY7JYR335QRO6SIP2XEGTT4UNKXY76HMG6MSXXLY4LA, seg_pos={0 0}, upload=187818]: Done uploading piece. err: <nil> Mar 18 19:42:31.651555273 (195h31m34.667224874s) [noise=true , address=79.127.205.234:4178 , piece_id=VYRVMFVMHSXGUJKMYG4VCDY5PWCKY3GZOOM7LFSGJT5D6G6XX7VA, seg_pos={0 0}, upload=187818]: Done uploading piece. err: <nil> Mar 18 19:42:31.845292574 (195h31m34.860962177s) [noise=true , address=193.149.164.70:4119 , piece_id=N3UUPE3G5KST5ONZ3CKLSXAH3WRAMPRKO4RHUP5NVRMOJMGC5U6Q, seg_pos={0 0}, upload=187818]: Done uploading piece. err: <nil> Mar 18 19:42:32.069813118 (195h31m35.085482719s) [noise=true , address=167.17.72.70:4747 , piece_id=DT7Q6F5FL35GOLB6R3WJUSE5ABMJUX4YOSQFBOEQP4L3GLKFDCSA, seg_pos={0 0}, upload=187818]: Done uploading piece. err: <nil> Mar 18 19:43:29.853808856 (195h32m32.869478464s) [noise=true , address=145.79.227.25:4641 , piece_id=CGZGZ5TOF6DXNDA6QCUOOVHJKUAMB5HQKM6UIAF36MIF7UN2P3DA, seg_pos={0 0}, upload=187818]: Done uploading piece. err: ecclient: upload failed (node:1FzqefKwAXK6Cs7uV8peWLN9m7cifXv8Xu7JTxdTPRe4UzenWr, address:145.79.227.25:4641): protocol: rpc: dial tcp 145.79.227.25:4641: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:43:29.853902541 (195h32m32.869572159s) [noise=true , address=145.79.227.41:4117 , piece_id=I33OGI4XUXWR2PJMOWF5T63ZLZA2EHG3GZNKO5JFUITN7BJ73ATA, seg_pos={0 0}, upload=187818]: Done uploading piece. err: ecclient: upload failed (node:1QVEhFpjfLubGsZmoY1XQdDWevLy2xWdjtPq4JnbSYuBgEXgz7, address:145.79.227.41:4117): protocol: rpc: dial tcp 145.79.227.41:4117: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:43:29.854201648 (195h32m32.869871257s) [noise=true , address=145.79.227.34:4924 , piece_id=FFGMVVL5SZ6MGHXLFCSRINV6ZXLJI6P5VMXAIYIIZC5PPWRAMQ6A, seg_pos={0 0}, upload=187818]: Done uploading piece. err: ecclient: upload failed (node:19TbrQjuN1EhBNVtha9EcKPSCAWyttfmHpec6QeF1S5zW6DRZs, address:145.79.227.34:4924): protocol: rpc: dial tcp 145.79.227.34:4924: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:43:29.854317854 (195h32m32.869987468s) [noise=true , address=145.79.227.18:4017 , piece_id=DFHRQXQCJHWCAIOB4PXG7VFAQIY4VZ2EWK4UOGYOCWOPGRO4OOCQ, seg_pos={0 0}, upload=187818]: Done uploading piece. err: ecclient: upload failed (node:12wS9YpUnsjgkdTPSowozp8txmqAhfF4n8s7JVpLb2tdRgQeLw8, address:145.79.227.18:4017): protocol: rpc: dial tcp 145.79.227.18:4017: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:43:29.854642668 (195h32m32.870312283s) [noise=true , address=145.79.227.36:4550 , piece_id=A6XR2NEO2W3JRZJ3GUKLMDAUOBPR4I6GEYF3JKDZ4JE33GCE3AIQ, seg_pos={0 0}, upload=187818]: Done uploading piece. err: ecclient: upload failed (node:1Fz8JANjsHWJkHj7Y2JAkr7NTKqDtcKjRW4kDaYGLJYwgWnxbS, address:145.79.227.36:4550): protocol: rpc: dial tcp 145.79.227.36:4550: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:43:29.854688745 (195h32m32.87035834s) [seg_pos={0 0}, upload=187818]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Mar 18 19:43:29.967337043 (195h32m32.983006644s) [upload=187818]: Done uploading segments... Mar 18 19:43:29.967385303 (195h32m32.983054961s) [upload=187818]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1m0.115354598s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [................................................XXXXXXXX....] duration: 2.352983215s trace: https://jaeger.dev.storj.io/trace/27979b4cc9984479 FAIL (2026-03-18 19:43:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.353106691s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [................................................XXXXXXXXXXXX] duration: 30.136514663s trace: https://jaeger.dev.storj.io/trace/310a783f73457e6e FAIL (2026-03-18 19:42:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 18 19:42:28.482570276 (195h31m31.498239867s) [seg_pos={0 0}, upload=187816]: Done begin upload segment. Mar 18 19:42:28.482614108 (195h31m31.498283714s) [noise=true , address=77.90.31.19:4841 , piece_id=FYNNOPEBFJZKWP23YC4XAECICECOASRA5RA6TBL26YJL2O7QRLQQ, seg_pos={0 0}, upload=187816]: Uploading piece... Mar 18 19:42:28.482643487 (195h31m31.498313098s) [noise=true , address=145.79.227.22:4513 , piece_id=ZOUAPQ2S5XSRMKK5GJCCTGORYVRXQPQMWKJEHQPWTE7JV6CMKO7Q, seg_pos={0 0}, upload=187816]: Uploading piece... Mar 18 19:42:28.483327673 (195h31m31.498997276s) [noise=true , address=145.79.227.8:4333 , piece_id=WXDJNZOT2PRYZSFNNHNE4NBLNSR7KFAOULUHLYJSZ7CXRB3WCJXQ, seg_pos={0 0}, upload=187816]: Uploading piece... Mar 18 19:42:28.483348599 (195h31m31.499018211s) [noise=true , address=145.79.227.17:4974 , piece_id=VX6MW3AE3YDKE6TLPDRS735TTLOGLMK43M2LRHY5TN34HQN46LQQ, seg_pos={0 0}, upload=187816]: Uploading piece... Mar 18 19:42:28.483730076 (195h31m31.499399676s) [noise=true , address=145.79.227.2:4113 , piece_id=W3Z3O7FCZLMPLX3Q5KE3XDRZZ64EJGAITS7DXMJ4MPDD3YVL3DLQ, seg_pos={0 0}, upload=187816]: Uploading piece... Mar 18 19:42:28.483805052 (195h31m31.499474666s) [noise=true , address=145.79.227.36:4520 , piece_id=VOFBNAFOM7TZNJ6DGVV47W2OWKTZ56YCPJUCQRTWOQKKALS45CRQ, seg_pos={0 0}, upload=187816]: Uploading piece... Mar 18 19:42:28.484063366 (195h31m31.499732969s) [noise=true , address=145.79.227.21:4477 , piece_id=QACVE2R3J443YK7LJW4IUFA3SAFP2BFJI5Z77H4CKBL27QPBUI7A, seg_pos={0 0}, upload=187816]: Uploading piece... Mar 18 19:42:28.484213337 (195h31m31.499882947s) [noise=true , address=45.91.116.5:4297 , piece_id=OKH5CCFQVP77CIJ6CPTYOJJNLPI633SWAR7V2YYK63PCGQLZVNLA, seg_pos={0 0}, upload=187816]: Uploading piece... Mar 18 19:42:28.484395820 (195h31m31.50006542s) [noise=true , address=145.79.227.3:4149 , piece_id=ZYOWANL4FUWM5MNBCDXKAYCOBZH65NZY3SMIDLAHNMWSGMHU2ABA, seg_pos={0 0}, upload=187816]: Uploading piece... Mar 18 19:42:28.484619048 (195h31m31.500288659s) [noise=true , address=145.79.227.46:4602 , piece_id=F4V3QVK4QTH2CKBYHO7H6SGXYY6C4RQCTTTT74X4DXSS2IFIEGRQ, seg_pos={0 0}, upload=187816]: Uploading piece... Mar 18 19:42:28.484734099 (195h31m31.500403702s) [noise=true , address=77.90.31.11:4556 , piece_id=CMNG66HJGZLSKBQCCPGQ77PLJB2KR26CWG5U6YILC4ELSNZO2B2A, seg_pos={0 0}, upload=187816]: Uploading piece... Mar 18 19:42:28.485024279 (195h31m31.500693891s) [noise=true , address=45.91.116.11:4119 , piece_id=6AKKVG3V2HJVX5GQA6FFE7LARC76J3SKOJ7QE7MPP3AKO7SBP6AQ, seg_pos={0 0}, upload=187816]: Uploading piece... Mar 18 19:42:28.485127504 (195h31m31.500797107s) [noise=true , address=145.79.227.9:4371 , piece_id=ACUKY7QRCFANMJNHB6DL6QXBE4VHWBQY245AWR54PMDX4PSTEUFQ, seg_pos={0 0}, upload=187816]: Uploading piece... Mar 18 19:42:28.485427649 (195h31m31.501097259s) [noise=true , address=145.79.227.51:4414 , piece_id=KPIC4O2GFDUNGJJLTMCEALGNIUZY5TJXAS2ZGD7CBM7MAEJPJZHQ, seg_pos={0 0}, upload=187816]: Uploading piece... Mar 18 19:42:28.485454581 (195h31m31.501124185s) [noise=true , address=145.79.227.43:4162 , piece_id=TTDWE6AB6TZ2N3JLFM4BPGDKEOSYZDV5HATR2EM6OEBEXWV47XDA, seg_pos={0 0}, upload=187816]: Uploading piece... Mar 18 19:42:28.485803027 (195h31m31.501472614s) [noise=true , address=45.91.116.4:4940 , piece_id=Z54DTDR6VVPXNZRLVYPXMVLQ2MOUHM3JXSK5QU54J4QVJBST5O7A, seg_pos={0 0}, upload=187816]: Uploading piece... Mar 18 19:42:29.078162678 (195h31m32.093832271s) [upload=187816]: Next returned nil segment Mar 18 19:42:29.078187609 (195h31m32.093857209s) [upload=187816]: Waiting for error group managing segments... Mar 18 19:42:29.216046318 (195h31m32.231715922s) [noise=true , address=77.90.31.11:4556 , piece_id=CMNG66HJGZLSKBQCCPGQ77PLJB2KR26CWG5U6YILC4ELSNZO2B2A, seg_pos={0 0}, upload=187816]: Done uploading piece. err: <nil> Mar 18 19:42:29.217377095 (195h31m32.233046697s) [noise=true , address=45.91.116.4:4940 , piece_id=Z54DTDR6VVPXNZRLVYPXMVLQ2MOUHM3JXSK5QU54J4QVJBST5O7A, seg_pos={0 0}, upload=187816]: Done uploading piece. err: <nil> Mar 18 19:42:29.239563518 (195h31m32.255233123s) [noise=true , address=45.91.116.11:4119 , piece_id=6AKKVG3V2HJVX5GQA6FFE7LARC76J3SKOJ7QE7MPP3AKO7SBP6AQ, seg_pos={0 0}, upload=187816]: Done uploading piece. err: <nil> Mar 18 19:42:29.244079022 (195h31m32.259748652s) [noise=true , address=77.90.31.19:4841 , piece_id=FYNNOPEBFJZKWP23YC4XAECICECOASRA5RA6TBL26YJL2O7QRLQQ, seg_pos={0 0}, upload=187816]: Done uploading piece. err: <nil> Mar 18 19:42:29.248020011 (195h31m32.263689612s) [noise=true , address=45.91.116.5:4297 , piece_id=OKH5CCFQVP77CIJ6CPTYOJJNLPI633SWAR7V2YYK63PCGQLZVNLA, seg_pos={0 0}, upload=187816]: Done uploading piece. err: <nil> Mar 18 19:42:58.005891092 (195h32m1.021560702s) [noise=true , address=145.79.227.2:4113 , piece_id=W3Z3O7FCZLMPLX3Q5KE3XDRZZ64EJGAITS7DXMJ4MPDD3YVL3DLQ, seg_pos={0 0}, upload=187816]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: rpc: dial tcp 145.79.227.2:4113: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:42:58.006127761 (195h32m1.021797365s) [noise=true , address=145.79.227.43:4162 , piece_id=TTDWE6AB6TZ2N3JLFM4BPGDKEOSYZDV5HATR2EM6OEBEXWV47XDA, seg_pos={0 0}, upload=187816]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: rpc: dial tcp 145.79.227.43:4162: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:42:58.006491433 (195h32m1.022161058s) [noise=true , address=145.79.227.9:4371 , piece_id=ACUKY7QRCFANMJNHB6DL6QXBE4VHWBQY245AWR54PMDX4PSTEUFQ, seg_pos={0 0}, upload=187816]: Done uploading piece. err: ecclient: upload failed (node:12cSqcGZqvZqs1QKBi6WK4fUDKgvQcmTupR1C9ZkWz879XeWPEL, address:145.79.227.9:4371): protocol: rpc: dial tcp 145.79.227.9:4371: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:42:58.006674263 (195h32m1.022343874s) [noise=true , address=145.79.227.3:4149 , piece_id=ZYOWANL4FUWM5MNBCDXKAYCOBZH65NZY3SMIDLAHNMWSGMHU2ABA, seg_pos={0 0}, upload=187816]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: rpc: dial tcp 145.79.227.3:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:42:58.006971207 (195h32m1.022640814s) [noise=true , address=145.79.227.21:4477 , piece_id=QACVE2R3J443YK7LJW4IUFA3SAFP2BFJI5Z77H4CKBL27QPBUI7A, seg_pos={0 0}, upload=187816]: Done uploading piece. err: ecclient: upload failed (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY, address:145.79.227.21:4477): protocol: rpc: dial tcp 145.79.227.21:4477: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:42:58.007144455 (195h32m1.022814065s) [noise=true , address=145.79.227.8:4333 , piece_id=WXDJNZOT2PRYZSFNNHNE4NBLNSR7KFAOULUHLYJSZ7CXRB3WCJXQ, seg_pos={0 0}, upload=187816]: Done uploading piece. err: ecclient: upload failed (node:1xRdTZoEk3NsnS445GcdEvwAdDzNLwUYSfgjB7bBSGRx9L9mrU, address:145.79.227.8:4333): protocol: rpc: dial tcp 145.79.227.8:4333: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:42:58.007786445 (195h32m1.023456054s) [noise=true , address=145.79.227.46:4602 , piece_id=F4V3QVK4QTH2CKBYHO7H6SGXYY6C4RQCTTTT74X4DXSS2IFIEGRQ, seg_pos={0 0}, upload=187816]: Done uploading piece. err: ecclient: upload failed (node:1kgUqvw1r3gSKw1bNjzys3AiAJXdkAnSafmvcPgNVPBbN9JS1d, address:145.79.227.46:4602): protocol: rpc: dial tcp 145.79.227.46:4602: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:42:58.007808212 (195h32m1.023477813s) [noise=true , address=145.79.227.22:4513 , piece_id=ZOUAPQ2S5XSRMKK5GJCCTGORYVRXQPQMWKJEHQPWTE7JV6CMKO7Q, seg_pos={0 0}, upload=187816]: Done uploading piece. err: ecclient: upload failed (node:12BykEXT7spgn4jkPucFmPvjvgKPKi5fZgRAeMTbfecZ7jCNori, address:145.79.227.22:4513): protocol: rpc: dial tcp 145.79.227.22:4513: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:42:58.007993852 (195h32m1.023663452s) [noise=true , address=145.79.227.17:4974 , piece_id=VX6MW3AE3YDKE6TLPDRS735TTLOGLMK43M2LRHY5TN34HQN46LQQ, seg_pos={0 0}, upload=187816]: Done uploading piece. err: ecclient: upload failed (node:1aR2XEwckLo7ufnCtsmyG2PS3vNiqpZs9b66UhuH7vUixcePS1, address:145.79.227.17:4974): protocol: rpc: dial tcp 145.79.227.17:4974: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:42:58.008058328 (195h32m1.023727939s) [noise=true , address=145.79.227.36:4520 , piece_id=VOFBNAFOM7TZNJ6DGVV47W2OWKTZ56YCPJUCQRTWOQKKALS45CRQ, seg_pos={0 0}, upload=187816]: Done uploading piece. err: ecclient: upload failed (node:12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy, address:145.79.227.36:4520): protocol: rpc: dial tcp 145.79.227.36:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:42:58.008239090 (195h32m1.023908695s) [noise=true , address=145.79.227.51:4414 , piece_id=KPIC4O2GFDUNGJJLTMCEALGNIUZY5TJXAS2ZGD7CBM7MAEJPJZHQ, seg_pos={0 0}, upload=187816]: Done uploading piece. err: ecclient: upload failed (node:1Gh17XmEkfiFPqqR1ASnC7LoMz6ik3a5EHitxG67GVQkDxAHc2, address:145.79.227.51:4414): protocol: rpc: dial tcp 145.79.227.51:4414: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:42:58.008279215 (195h32m1.023948802s) [seg_pos={0 0}, upload=187816]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Mar 18 19:42:58.140754080 (195h32m1.156423692s) [upload=187816]: Done uploading segments... Mar 18 19:42:58.140798973 (195h32m1.156468581s) [upload=187816]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 30.136537816s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [..................................................XXX.XXX...] duration: 4.642745265s trace: https://jaeger.dev.storj.io/trace/4d6a8ab787f335c0 FAIL (2026-03-18 19:42:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.642770703s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [.................................................XXXXXXXX...] duration: 2.828768395s trace: https://jaeger.dev.storj.io/trace/27979b4cc9984479 FAIL (2026-03-18 19:42:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.82879078s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [..................................................XXXXXXXX..] duration: 2.030501568s trace: https://jaeger.dev.storj.io/trace/27979b4cc9984479 FAIL (2026-03-18 19:42:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.030529042s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [...................................................XXX.XXX..] duration: 4.009071321s trace: https://jaeger.dev.storj.io/trace/4d6a8ab787f335c0 FAIL (2026-03-18 19:42:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.009101273s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.................................................XXXXXXXXXXX] duration: 30.180624045s trace: https://jaeger.dev.storj.io/trace/7a74688dbc8b5bb FAIL (2026-03-18 19:41:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Mar 18 19:41:19.270400354 (195h30m22.286069946s) [upload=187798]: Starting upload Mar 18 19:41:19.270500171 (195h30m22.28616978s) [upload=187798]: Done starting upload Mar 18 19:41:19.270588136 (195h30m22.286257727s) [upload=187798]: Uploading segments... Mar 18 19:41:19.270977152 (195h30m22.286646743s) [upload=187798]: Got next segment. Inline: false Mar 18 19:41:19.271045484 (195h30m22.286715085s) [upload=187798]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 19:41:19.271052971 (195h30m22.286722564s) [upload=187798]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 19:41:19.428116146 (195h30m22.443785749s) [seg_pos={0 0}, upload=187798]: Begin upload segment... Mar 18 19:41:19.428183410 (195h30m22.443853s) [seg_pos={0 0}, upload=187798]: Done begin upload segment. Mar 18 19:41:19.428554401 (195h30m22.444224001s) [noise=true , address=45.91.116.6:4989 , piece_id=GDWNGB6GPLV5RQ3NJSMUSY2HEGGMLZDSKZNM43BQBYRIPE63GDWQ, seg_pos={0 0}, upload=187798]: Uploading piece... Mar 18 19:41:19.430044325 (195h30m22.445713934s) [noise=true , address=45.91.116.9:4296 , piece_id=VL555JHY73F2H54NTNGFRVZTHKC4MHU6D7BULPPPMBZXORGLHJCQ, seg_pos={0 0}, upload=187798]: Uploading piece... Mar 18 19:41:19.431034812 (195h30m22.446704415s) [noise=true , address=45.91.116.10:4082 , piece_id=SAWK3MUZ7ANYEWQ5MDXBIHYXG3LU4TBFN74XJ2OTDFBAU34HL63Q, seg_pos={0 0}, upload=187798]: Uploading piece... Mar 18 19:41:19.431431413 (195h30m22.447101019s) [noise=true , address=145.79.227.15:4878 , piece_id=3KLYW42QEC3DBPJYGZI3YLAYAVOW4S2SNLANZ2LEBAEI2DHCXWQQ, seg_pos={0 0}, upload=187798]: Uploading piece... Mar 18 19:41:19.431814474 (195h30m22.447484078s) [noise=true , address=145.79.227.32:4833 , piece_id=PHERHUSCRBPYL3O57PI4TKQ4XFQKKFZSMTLI54HXQFBURRLBV4ZA, seg_pos={0 0}, upload=187798]: Uploading piece... Mar 18 19:41:19.432749981 (195h30m22.448419593s) [noise=true , address=145.79.227.26:4657 , piece_id=EZF2OKWSCAFAABWUZMMJFRFILIWLYVXPHEZPONK7CODTFR3UZ2OQ, seg_pos={0 0}, upload=187798]: Uploading piece... Mar 18 19:41:19.433234511 (195h30m22.44890412s) [noise=true , address=45.91.116.4:4972 , piece_id=P555QUI7DSHBCINH5HG2UR5JI4RPHOXZ2IMGJBQEKIQMTPLHN5KA, seg_pos={0 0}, upload=187798]: Uploading piece... Mar 18 19:41:19.433604305 (195h30m22.449273911s) [noise=true , address=145.79.227.16:4911 , piece_id=XE4Y3HQYKZ43Q2SNGJ473QJBXIG3YUN3N47N53R2QFWCRJHIFM2Q, seg_pos={0 0}, upload=187798]: Uploading piece... Mar 18 19:41:19.434047482 (195h30m22.449717088s) [noise=true , address=77.90.31.14:4664 , piece_id=IRNNUASSGL3KAZX53Y5IHQFBSBNBSBPXAOX6MECWE7U7JMRNWTDA, seg_pos={0 0}, upload=187798]: Uploading piece... Mar 18 19:41:19.434958548 (195h30m22.450628154s) [noise=true , address=79.127.219.41:4927 , piece_id=KIRCGPZO75PUQDE6XZ2VMIPM3MYUSSQWWN7DFNS2PEI4RRQFATZA, seg_pos={0 0}, upload=187798]: Uploading piece... Mar 18 19:41:19.435377797 (195h30m22.451047401s) [noise=true , address=145.79.227.50:4374 , piece_id=QBRM2WEMFVOE7WIY5D5P7O6HQ4JTPXKBPDNTQNZQNXXO6C3WV7KQ, seg_pos={0 0}, upload=187798]: Uploading piece... Mar 18 19:41:19.435751511 (195h30m22.451421119s) [noise=true , address=77.90.31.15:4697 , piece_id=ZBU73XV34P3F7ICILD566TIUESFDVNIAMB7SH2IOVS3B6A52G2LA, seg_pos={0 0}, upload=187798]: Uploading piece... Mar 18 19:41:19.436115439 (195h30m22.451785045s) [noise=true , address=45.91.116.14:4224 , piece_id=BST3D4BW3RSJWJ4TZFTELJVIFP7AU6B5C6Y64TRWPSKY6LHOAKPQ, seg_pos={0 0}, upload=187798]: Uploading piece... Mar 18 19:41:19.436455571 (195h30m22.452125181s) [noise=true , address=145.79.227.20:4442 , piece_id=LSZRXL3PVICKKM4RZHFHJ2JFXWK5PZEJIIYITXPAXYUKJWHHYOMQ, seg_pos={0 0}, upload=187798]: Uploading piece... Mar 18 19:41:19.436941662 (195h30m22.45261127s) [noise=true , address=45.91.116.5:4298 , piece_id=B3N77D466R63HKYU2VTWIC2FVUOWRFT37VMHOVI6HB6CJNUOKIDA, seg_pos={0 0}, upload=187798]: Uploading piece... Mar 18 19:41:19.441348046 (195h30m22.457017655s) [noise=true , address=45.91.116.11:4120 , piece_id=MICN2SZG65FJSUU5KQPPF7TJJMSV6LULYXUY5IUS3NGG26ZPBTXQ, seg_pos={0 0}, upload=187798]: Uploading piece... Mar 18 19:41:20.032318751 (195h30m23.047988355s) [upload=187798]: Next returned nil segment Mar 18 19:41:20.032364989 (195h30m23.048034602s) [upload=187798]: Waiting for error group managing segments... Mar 18 19:41:20.163480555 (195h30m23.179150155s) [noise=true , address=77.90.31.14:4664 , piece_id=IRNNUASSGL3KAZX53Y5IHQFBSBNBSBPXAOX6MECWE7U7JMRNWTDA, seg_pos={0 0}, upload=187798]: Done uploading piece. err: <nil> Mar 18 19:41:20.177534008 (195h30m23.193203611s) [noise=true , address=45.91.116.11:4120 , piece_id=MICN2SZG65FJSUU5KQPPF7TJJMSV6LULYXUY5IUS3NGG26ZPBTXQ, seg_pos={0 0}, upload=187798]: Done uploading piece. err: <nil> Mar 18 19:41:20.177669283 (195h30m23.193338887s) [noise=true , address=45.91.116.9:4296 , piece_id=VL555JHY73F2H54NTNGFRVZTHKC4MHU6D7BULPPPMBZXORGLHJCQ, seg_pos={0 0}, upload=187798]: Done uploading piece. err: <nil> Mar 18 19:41:20.178604317 (195h30m23.194273919s) [noise=true , address=45.91.116.14:4224 , piece_id=BST3D4BW3RSJWJ4TZFTELJVIFP7AU6B5C6Y64TRWPSKY6LHOAKPQ, seg_pos={0 0}, upload=187798]: Done uploading piece. err: <nil> Mar 18 19:41:20.181412142 (195h30m23.197081742s) [noise=true , address=45.91.116.10:4082 , piece_id=SAWK3MUZ7ANYEWQ5MDXBIHYXG3LU4TBFN74XJ2OTDFBAU34HL63Q, seg_pos={0 0}, upload=187798]: Done uploading piece. err: <nil> Mar 18 19:41:20.184543982 (195h30m23.200213582s) [noise=true , address=45.91.116.4:4972 , piece_id=P555QUI7DSHBCINH5HG2UR5JI4RPHOXZ2IMGJBQEKIQMTPLHN5KA, seg_pos={0 0}, upload=187798]: Done uploading piece. err: <nil> Mar 18 19:41:20.185526927 (195h30m23.20119653s) [noise=true , address=45.91.116.5:4298 , piece_id=B3N77D466R63HKYU2VTWIC2FVUOWRFT37VMHOVI6HB6CJNUOKIDA, seg_pos={0 0}, upload=187798]: Done uploading piece. err: <nil> Mar 18 19:41:20.197278733 (195h30m23.212948342s) [noise=true , address=77.90.31.15:4697 , piece_id=ZBU73XV34P3F7ICILD566TIUESFDVNIAMB7SH2IOVS3B6A52G2LA, seg_pos={0 0}, upload=187798]: Done uploading piece. err: <nil> Mar 18 19:41:20.201529094 (195h30m23.217198704s) [noise=true , address=45.91.116.6:4989 , piece_id=GDWNGB6GPLV5RQ3NJSMUSY2HEGGMLZDSKZNM43BQBYRIPE63GDWQ, seg_pos={0 0}, upload=187798]: Done uploading piece. err: <nil> Mar 18 19:41:20.284165629 (195h30m23.299835221s) [noise=true , address=79.127.219.41:4927 , piece_id=KIRCGPZO75PUQDE6XZ2VMIPM3MYUSSQWWN7DFNS2PEI4RRQFATZA, seg_pos={0 0}, upload=187798]: Done uploading piece. err: <nil> Mar 18 19:41:48.988577919 (195h30m52.004247529s) [noise=true , address=145.79.227.20:4442 , piece_id=LSZRXL3PVICKKM4RZHFHJ2JFXWK5PZEJIIYITXPAXYUKJWHHYOMQ, seg_pos={0 0}, upload=187798]: Done uploading piece. err: ecclient: upload failed (node:125KyoqeSzbjP8XPhXqw2AenTDiLtVmt5KpLBD7ZW7YC9e9zJu9, address:145.79.227.20:4442): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:41:48.988707306 (195h30m52.004376916s) [noise=true , address=145.79.227.26:4657 , piece_id=EZF2OKWSCAFAABWUZMMJFRFILIWLYVXPHEZPONK7CODTFR3UZ2OQ, seg_pos={0 0}, upload=187798]: Done uploading piece. err: ecclient: upload failed (node:129MyUjh5Nw3NhQxefnBzxYUK3mSXqwpW3nLa3KerYrjpuJiUfq, address:145.79.227.26:4657): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:41:48.988862434 (195h30m52.004532044s) [noise=true , address=145.79.227.32:4833 , piece_id=PHERHUSCRBPYL3O57PI4TKQ4XFQKKFZSMTLI54HXQFBURRLBV4ZA, seg_pos={0 0}, upload=187798]: Done uploading piece. err: ecclient: upload failed (node:127L5Mr53vCqKr4gy7N4rAaVCC9FvU8XdYKFrxWrJL728dt3LmR, address:145.79.227.32:4833): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:41:48.988959065 (195h30m52.004628675s) [noise=true , address=145.79.227.50:4374 , piece_id=QBRM2WEMFVOE7WIY5D5P7O6HQ4JTPXKBPDNTQNZQNXXO6C3WV7KQ, seg_pos={0 0}, upload=187798]: Done uploading piece. err: ecclient: upload failed (node:12QRixhWaUwXPdjcWKYHbiknME5JVomWYABY6egDD6VKwaTPB8K, address:145.79.227.50:4374): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:41:48.989056323 (195h30m52.00472591s) [noise=true , address=145.79.227.15:4878 , piece_id=3KLYW42QEC3DBPJYGZI3YLAYAVOW4S2SNLANZ2LEBAEI2DHCXWQQ, seg_pos={0 0}, upload=187798]: Done uploading piece. err: ecclient: upload failed (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC, address:145.79.227.15:4878): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:41:48.989103517 (195h30m52.004773104s) [noise=true , address=145.79.227.16:4911 , piece_id=XE4Y3HQYKZ43Q2SNGJ473QJBXIG3YUN3N47N53R2QFWCRJHIFM2Q, seg_pos={0 0}, upload=187798]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: context deadline exceeded; context deadline exceeded Mar 18 19:41:48.989125631 (195h30m52.004795233s) [seg_pos={0 0}, upload=187798]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Mar 18 19:41:49.167497719 (195h30m52.183167329s) [upload=187798]: Done uploading segments... Mar 18 19:41:49.167541122 (195h30m52.183210733s) [upload=187798]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 30.180641418s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................................................X..X...X..] duration: 3.303801774s trace: https://jaeger.dev.storj.io/trace/7bb5f3f9442e399e FAIL (2026-03-18 19:41:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== oad=187775]: Uploading segments... Mar 18 19:39:51.245373273 (195h28m54.261042883s) [upload=187775]: Got next segment. Inline: false Mar 18 19:39:51.245450148 (195h28m54.261119759s) [upload=187775]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 19:39:51.245457117 (195h28m54.261126725s) [upload=187775]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 19:39:51.356664730 (195h28m54.372334331s) [seg_pos={0 0}, upload=187775]: Begin upload segment... Mar 18 19:39:51.356741101 (195h28m54.372410707s) [seg_pos={0 0}, upload=187775]: Done begin upload segment. Mar 18 19:39:51.356816902 (195h28m54.372486505s) [noise=true , address=45.91.116.12:4257 , piece_id=YDLJDFP3Q7VL2JANZL366KGZXB3EI4MG5PWEBDPYPJW7IXY77NRQ, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.356876970 (195h28m54.372546579s) [noise=true , address=167.17.72.71:4811 , piece_id=GDYHOQWV5KNLDWZRWX2S7YXGQ77KMN65ECAMSXGHEAIMJMIDIKVQ, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.357042162 (195h28m54.372711762s) [noise=true , address=145.79.227.37:4574 , piece_id=XRLKWPUQPPQJOCHD7BSZMS3EZGIXLRK2WFVZARB2T5IPDFQB3SDA, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.357958364 (195h28m54.373627987s) [noise=true , address=79.127.219.46:4919 , piece_id=7E5AC5BG7OUFDQXKAQVVFIKCNMBCH44YHGPHJXV6WLL6VZNAYVCQ, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.357989629 (195h28m54.373659229s) [noise=true , address=45.91.116.10:4087 , piece_id=ZS6W5PCUKBUVCNWLGIUP24WZ4Z6JQYNHSPGEI7RVV367NFBU324A, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.358579404 (195h28m54.37424901s) [noise=true , address=167.17.72.69:4741 , piece_id=2UMXKHMNVA52AOVHBHO2YHJ6BPHBQ4NL3O4S32LFVZKY524GGKUA, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.358785953 (195h28m54.374455555s) [noise=true , address=145.79.227.31:4821 , piece_id=2RY5OW2QLCFMX4Q3ZYHS3LBKHRUAOGGWDDGZCDWBRXGLTRJZGQNQ, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.359325146 (195h28m54.374994746s) [noise=true , address=193.149.164.71:4293 , piece_id=U3Y47GEFDQZZYSQWZPXLIKPQX7KOSATYYXSSQ2LLZRN5R5A4X43A, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.359888775 (195h28m54.375558385s) [noise=true , address=145.79.227.50:4393 , piece_id=MKQZD5DRQKJGDJRKUUPIHIBHQ5HAVABPTTQDYUQ76L4V5IXE2XIQ, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.360658986 (195h28m54.376328586s) [noise=true , address=45.91.116.11:4141 , piece_id=ALRFXBSV356RLTAAODD6LCRGVMIW4YCLWLV3J7CCDLQ4KFQWGJKA, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.360990163 (195h28m54.376659767s) [noise=true , address=45.91.116.13:4185 , piece_id=DCRQDPZMQB3BR6ST4OSI2UURGDC5E2PUUZVU4GJMOJDMLHQ2EFOA, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.361395094 (195h28m54.377064703s) [noise=true , address=167.17.72.70:4752 , piece_id=ZWHC3OAFEVAHYOMXNP22YLHVJOTR72NAMCZKMNKY6EP7ZY6GMP4A, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.361842432 (195h28m54.377512034s) [noise=true , address=64.49.13.27:4957 , piece_id=37IAXM3YBV6IOZGC3DM5QEDYQTM6HSAHS5UFADOIMPEGK3QF3BTQ, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.362135261 (195h28m54.377804865s) [noise=true , address=145.79.227.33:4882 , piece_id=KEOJNYHAH5QPIFGGDB4PQNKJ4N5VQY3USQD2NWFLCH3TJKIIAP3A, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.362663307 (195h28m54.378332918s) [noise=true , address=193.149.164.66:4977 , piece_id=7HOES66K6XAZBO5IF3O6K4HDX27SYMMYWBMSMLX2URPKN4SSFEZA, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.362795143 (195h28m54.378464749s) [noise=true , address=145.79.227.43:4180 , piece_id=GGTUQNDLFS2QXTKOOFAA3QIIES7OW32QKK3V4VLQ2F2F74ITNWFQ, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.363148171 (195h28m54.378817777s) [noise=true , address=45.91.116.5:4323 , piece_id=M4IXPUQQZSM75YMO567TTF3OEQV2Y5AY723HLLLKVSYVK2J4N4PQ, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.363481817 (195h28m54.379151417s) [noise=true , address=79.127.219.41:4958 , piece_id=OM62E6RDF7K5BUXBTZ4YFWRLEQ46W2ZCAY27WZ75ZEAYYI4ANKTQ, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.914452486 (195h28m54.930122086s) [upload=187775]: Next returned nil segment Mar 18 19:39:51.914477088 (195h28m54.930146689s) [upload=187775]: Waiting for error group managing segments... Mar 18 19:39:52.065063812 (195h28m55.080733412s) [noise=true , address=45.91.116.10:4087 , piece_id=ZS6W5PCUKBUVCNWLGIUP24WZ4Z6JQYNHSPGEI7RVV367NFBU324A, seg_pos={0 0}, upload=187775]: Done uploading piece. err: <nil> Mar 18 19:39:52.075576360 (195h28m55.09124596s) [noise=true , address=45.91.116.5:4323 , piece_id=M4IXPUQQZSM75YMO567TTF3OEQV2Y5AY723HLLLKVSYVK2J4N4PQ, seg_pos={0 0}, upload=187775]: Done uploading piece. err: <nil> Mar 18 19:39:52.076559539 (195h28m55.09222914s) [noise=true , address=45.91.116.13:4185 , piece_id=DCRQDPZMQB3BR6ST4OSI2UURGDC5E2PUUZVU4GJMOJDMLHQ2EFOA, seg_pos={0 0}, upload=187775]: Done uploading piece. err: <nil> Mar 18 19:39:52.079832594 (195h28m55.095502205s) [noise=true , address=45.91.116.11:4141 , piece_id=ALRFXBSV356RLTAAODD6LCRGVMIW4YCLWLV3J7CCDLQ4KFQWGJKA, seg_pos={0 0}, upload=187775]: Done uploading piece. err: <nil> Mar 18 19:39:52.094457996 (195h28m55.110127595s) [noise=true , address=79.127.219.46:4919 , piece_id=7E5AC5BG7OUFDQXKAQVVFIKCNMBCH44YHGPHJXV6WLL6VZNAYVCQ, seg_pos={0 0}, upload=187775]: Done uploading piece. err: <nil> Mar 18 19:39:52.116381598 (195h28m55.132051201s) [noise=true , address=79.127.219.41:4958 , piece_id=OM62E6RDF7K5BUXBTZ4YFWRLEQ46W2ZCAY27WZ75ZEAYYI4ANKTQ, seg_pos={0 0}, upload=187775]: Done uploading piece. err: <nil> Mar 18 19:39:52.253087456 (195h28m55.268757064s) [noise=true , address=45.91.116.12:4257 , piece_id=YDLJDFP3Q7VL2JANZL366KGZXB3EI4MG5PWEBDPYPJW7IXY77NRQ, seg_pos={0 0}, upload=187775]: Done uploading piece. err: <nil> Mar 18 19:39:52.483396325 (195h28m55.499065936s) [noise=true , address=167.17.72.71:4811 , piece_id=GDYHOQWV5KNLDWZRWX2S7YXGQ77KMN65ECAMSXGHEAIMJMIDIKVQ, seg_pos={0 0}, upload=187775]: Done uploading piece. err: <nil> Mar 18 19:39:52.500200438 (195h28m55.515870037s) [noise=true , address=167.17.72.69:4741 , piece_id=2UMXKHMNVA52AOVHBHO2YHJ6BPHBQ4NL3O4S32LFVZKY524GGKUA, seg_pos={0 0}, upload=187775]: Done uploading piece. err: <nil> Mar 18 19:39:52.565217723 (195h28m55.580887332s) [noise=true , address=167.17.72.70:4752 , piece_id=ZWHC3OAFEVAHYOMXNP22YLHVJOTR72NAMCZKMNKY6EP7ZY6GMP4A, seg_pos={0 0}, upload=187775]: Done uploading piece. err: <nil> Mar 18 19:39:52.590846650 (195h28m55.606516251s) [noise=true , address=64.49.13.27:4957 , piece_id=37IAXM3YBV6IOZGC3DM5QEDYQTM6HSAHS5UFADOIMPEGK3QF3BTQ, seg_pos={0 0}, upload=187775]: Done uploading piece. err: <nil> Mar 18 19:39:52.598323339 (195h28m55.613992939s) [noise=true , address=193.149.164.66:4977 , piece_id=7HOES66K6XAZBO5IF3O6K4HDX27SYMMYWBMSMLX2URPKN4SSFEZA, seg_pos={0 0}, upload=187775]: Done uploading piece. err: <nil> Mar 18 19:39:52.746791574 (195h28m55.762461183s) [noise=true , address=193.149.164.71:4293 , piece_id=U3Y47GEFDQZZYSQWZPXLIKPQX7KOSATYYXSSQ2LLZRN5R5A4X43A, seg_pos={0 0}, upload=187775]: Done uploading piece. err: <nil> Mar 18 19:40:50.958698173 (195h29m53.974367782s) [noise=true , address=145.79.227.50:4393 , piece_id=MKQZD5DRQKJGDJRKUUPIHIBHQ5HAVABPTTQDYUQ76L4V5IXE2XIQ, seg_pos={0 0}, upload=187775]: Done uploading piece. err: ecclient: upload failed (node:1xfCSdYjRRxMaTw47dQmaXzGmMWyV1oCCGx7JdhvGqaaoN14Ff, address:145.79.227.50:4393): protocol: rpc: dial tcp 145.79.227.50:4393: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:40:50.958929147 (195h29m53.974598757s) [noise=true , address=145.79.227.33:4882 , piece_id=KEOJNYHAH5QPIFGGDB4PQNKJ4N5VQY3USQD2NWFLCH3TJKIIAP3A, seg_pos={0 0}, upload=187775]: Done uploading piece. err: ecclient: upload failed (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc, address:145.79.227.33:4882): protocol: rpc: dial tcp 145.79.227.33:4882: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:40:50.959152478 (195h29m53.974822088s) [noise=true , address=145.79.227.37:4574 , piece_id=XRLKWPUQPPQJOCHD7BSZMS3EZGIXLRK2WFVZARB2T5IPDFQB3SDA, seg_pos={0 0}, upload=187775]: Done uploading piece. err: ecclient: upload failed (node:1tNoERpZQJ2K3F56AhqUErsCMoF2adJfye5rrEoSi8dnjimcnw, address:145.79.227.37:4574): protocol: rpc: dial tcp 145.79.227.37:4574: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:40:50.959352897 (195h29m53.975022507s) [noise=true , address=145.79.227.31:4821 , piece_id=2RY5OW2QLCFMX4Q3ZYHS3LBKHRUAOGGWDDGZCDWBRXGLTRJZGQNQ, seg_pos={0 0}, upload=187775]: Done uploading piece. err: ecclient: upload failed (node:1Q2vaBr3JNY3jv8jVUTg9uzfWUtNjrd5c5Z9Yfi54D3JMRM1af, address:145.79.227.31:4821): protocol: rpc: dial tcp 145.79.227.31:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:40:50.959577488 (195h29m53.975247097s) [noise=true , address=145.79.227.43:4180 , piece_id=GGTUQNDLFS2QXTKOOFAA3QIIES7OW32QKK3V4VLQ2F2F74ITNWFQ, seg_pos={0 0}, upload=187775]: Done uploading piece. err: ecclient: upload failed (node:1rhkcipPBZNFScBjBhWdigzSMwENSmzhCucnC2iuyc9udpweE3, address:145.79.227.43:4180): protocol: rpc: dial tcp 145.79.227.43:4180: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:40:50.959606832 (195h29m53.975276422s) [seg_pos={0 0}, upload=187775]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Mar 18 19:40:51.064815222 (195h29m54.080484816s) [upload=187775]: Done uploading segments... Mar 18 19:40:51.064854034 (195h29m54.080523634s) [upload=187775]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 3.303844847s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [..................................................XXXXXXXXXX] duration: 30.127367481s trace: https://jaeger.dev.storj.io/trace/1a10a20328dc7c59 FAIL (2026-03-18 19:41:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== _pos={0 0}, upload=187789]: Uploading piece... Mar 18 19:40:49.143832862 (195h29m52.159502481s) [noise=true , address=77.90.31.21:4880 , piece_id=2GBXWQK6KLGR23DHJAWRIOJ26IVS2ZA67TPKP6WSNEU6HEH4CAQQ, seg_pos={0 0}, upload=187789]: Uploading piece... Mar 18 19:40:49.144020951 (195h29m52.159690558s) [noise=true , address=45.91.116.5:4299 , piece_id=3JMXESOFFX34M5APWNTIHWOJNFSNWZUWDJ2M75FB4BC3O67DGI4Q, seg_pos={0 0}, upload=187789]: Uploading piece... Mar 18 19:40:49.146243409 (195h29m52.161913028s) [noise=true , address=145.79.227.10:4404 , piece_id=USIYPXJ4PUIBNF33AACAUHCDQWEA2TYCSNODKPJMQ3WEHR4FWE6A, seg_pos={0 0}, upload=187789]: Uploading piece... Mar 18 19:40:49.147706945 (195h29m52.163376555s) [noise=true , address=145.79.227.26:4656 , piece_id=K26SLR2MITYNRJIBSYPZSIGHLASEKZAPNL2LIQ3ZP4V7ZI27AOWA, seg_pos={0 0}, upload=187789]: Uploading piece... Mar 18 19:40:49.148286426 (195h29m52.163956032s) [noise=true , address=145.79.227.40:4054 , piece_id=L64BPXJXBYUY5KRY4KJUXLA7344CDFS5FP5WODCXDVMIKS4FMHQA, seg_pos={0 0}, upload=187789]: Uploading piece... Mar 18 19:40:49.148906198 (195h29m52.164575873s) [noise=true , address=145.79.227.41:4085 , piece_id=R7TEBXPH5XPDVEPGIGDPEHJICQYUWUCIJT2RNSEGDMPNPDOSTXPQ, seg_pos={0 0}, upload=187789]: Uploading piece... Mar 18 19:40:49.149572067 (195h29m52.165241668s) [noise=true , address=145.79.227.16:4911 , piece_id=GEGCH2DAMKOX5KQQZ7EXNNNRIQBFNY2BY4GTJUI2UOMXPAPZV5VQ, seg_pos={0 0}, upload=187789]: Uploading piece... Mar 18 19:40:49.151383237 (195h29m52.16705284s) [noise=true , address=145.79.227.48:4303 , piece_id=E2E3LRCQJQ23NR2MYL4TECQFPSTBA7PX23GMP4BIC5O3N7QAYQYA, seg_pos={0 0}, upload=187789]: Uploading piece... Mar 18 19:40:49.152691793 (195h29m52.168361399s) [noise=true , address=145.79.227.20:4438 , piece_id=5MO2KAHKWKX3RMBZFI3T5KQUYJ26C5SDXYV5JMHO4M3HLSFWIZWA, seg_pos={0 0}, upload=187789]: Uploading piece... Mar 18 19:40:49.156414777 (195h29m52.172084377s) [noise=true , address=145.79.227.46:4602 , piece_id=5FWXO7QFJFHPD36MW5HNX3WTBFYR2KQD7LYZVFKO7H5FOW4J6WZA, seg_pos={0 0}, upload=187789]: Uploading piece... Mar 18 19:40:49.159383231 (195h29m52.175052842s) [noise=true , address=145.79.227.50:4378 , piece_id=5OFIICDCHFAS37MXJ7LG66VLPLOYZEDEAGIJQ2TZERHYWTIC6DWQ, seg_pos={0 0}, upload=187789]: Uploading piece... Mar 18 19:40:49.163276087 (195h29m52.178945693s) [noise=true , address=145.79.227.22:4514 , piece_id=UPUVWD4DI2HEFAUZ7TWMRW6BEZRA6EMU4A6XWMCXDT26BFTUAM4A, seg_pos={0 0}, upload=187789]: Uploading piece... Mar 18 19:40:49.164787437 (195h29m52.180457048s) [noise=true , address=145.79.227.47:4270 , piece_id=2T3WFSFTAZ7O54EWPQDFAQVAUVWZ3OFXNVF56JHWWU4SQDRGGEOQ, seg_pos={0 0}, upload=187789]: Uploading piece... Mar 18 19:40:49.166608838 (195h29m52.182278449s) [noise=true , address=145.79.227.5:4225 , piece_id=TOS76O4VF3WPADTPCM5ZXTEDRGY372XNODESV7EQTYOGZX6V4EQQ, seg_pos={0 0}, upload=187789]: Uploading piece... Mar 18 19:40:49.742825545 (195h29m52.758495145s) [upload=187789]: Next returned nil segment Mar 18 19:40:49.742846795 (195h29m52.758516403s) [upload=187789]: Waiting for error group managing segments... Mar 18 19:40:49.880267726 (195h29m52.895937327s) [noise=true , address=77.90.31.21:4880 , piece_id=2GBXWQK6KLGR23DHJAWRIOJ26IVS2ZA67TPKP6WSNEU6HEH4CAQQ, seg_pos={0 0}, upload=187789]: Done uploading piece. err: <nil> Mar 18 19:40:49.904382460 (195h29m52.920052065s) [noise=true , address=45.91.116.5:4299 , piece_id=3JMXESOFFX34M5APWNTIHWOJNFSNWZUWDJ2M75FB4BC3O67DGI4Q, seg_pos={0 0}, upload=187789]: Done uploading piece. err: <nil> Mar 18 19:40:49.943461395 (195h29m52.959130997s) [noise=true , address=79.127.219.41:4930 , piece_id=CXM7X2SPNLTP2PJL55NRWQPSXZGZM5DQ2JBV62BUOVUHPDVKHNJQ, seg_pos={0 0}, upload=187789]: Done uploading piece. err: <nil> Mar 18 19:41:18.729608706 (195h30m21.745278316s) [noise=true , address=145.79.227.33:4874 , piece_id=F2EAHFET72274LQUM7KJQOT7552ZSRDEVO4J2DT7CAM2HZ2SIVLQ, seg_pos={0 0}, upload=187789]: Done uploading piece. err: ecclient: upload failed (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq, address:145.79.227.33:4874): protocol: rpc: dial tcp 145.79.227.33:4874: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:41:18.730050173 (195h30m21.745719783s) [noise=true , address=145.79.227.20:4438 , piece_id=5MO2KAHKWKX3RMBZFI3T5KQUYJ26C5SDXYV5JMHO4M3HLSFWIZWA, seg_pos={0 0}, upload=187789]: Done uploading piece. err: ecclient: upload failed (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT, address:145.79.227.20:4438): protocol: rpc: dial tcp 145.79.227.20:4438: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:41:18.730173098 (195h30m21.745842708s) [noise=true , address=145.79.227.5:4225 , piece_id=TOS76O4VF3WPADTPCM5ZXTEDRGY372XNODESV7EQTYOGZX6V4EQQ, seg_pos={0 0}, upload=187789]: Done uploading piece. err: ecclient: upload failed (node:12QtY3yN6J1H8htRneePHu4RmbN7A1xkgywphMDVsVxhTczGupY, address:145.79.227.5:4225): protocol: rpc: dial tcp 145.79.227.5:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:41:18.730321512 (195h30m21.745991123s) [noise=true , address=145.79.227.46:4602 , piece_id=5FWXO7QFJFHPD36MW5HNX3WTBFYR2KQD7LYZVFKO7H5FOW4J6WZA, seg_pos={0 0}, upload=187789]: Done uploading piece. err: ecclient: upload failed (node:1kgUqvw1r3gSKw1bNjzys3AiAJXdkAnSafmvcPgNVPBbN9JS1d, address:145.79.227.46:4602): protocol: rpc: dial tcp 145.79.227.46:4602: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:41:18.730467836 (195h30m21.746137446s) [noise=true , address=145.79.227.40:4054 , piece_id=L64BPXJXBYUY5KRY4KJUXLA7344CDFS5FP5WODCXDVMIKS4FMHQA, seg_pos={0 0}, upload=187789]: Done uploading piece. err: ecclient: upload failed (node:1opDYSgBqBFTN6wZ5mrN5pJsnt9AH4y5sDr1yxE3Cjxg3vbq3v, address:145.79.227.40:4054): protocol: rpc: dial tcp 145.79.227.40:4054: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:41:18.730577367 (195h30m21.746246978s) [noise=true , address=145.79.227.47:4270 , piece_id=2T3WFSFTAZ7O54EWPQDFAQVAUVWZ3OFXNVF56JHWWU4SQDRGGEOQ, seg_pos={0 0}, upload=187789]: Done uploading piece. err: ecclient: upload failed (node:12sJfMr1nMTpizd5PZ2bZ6M8tZWhFL2ukdJ3cNSRQtdUHNJhg81, address:145.79.227.47:4270): protocol: rpc: dial tcp 145.79.227.47:4270: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:41:18.730839874 (195h30m21.746509485s) [noise=true , address=145.79.227.41:4085 , piece_id=R7TEBXPH5XPDVEPGIGDPEHJICQYUWUCIJT2RNSEGDMPNPDOSTXPQ, seg_pos={0 0}, upload=187789]: Done uploading piece. err: ecclient: upload failed (node:17x7QtnZsENMZPU3tqkh8jEd29JdowLvT855W4kuourgUA6y5C, address:145.79.227.41:4085): protocol: rpc: dial tcp 145.79.227.41:4085: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:41:18.731064758 (195h30m21.746734369s) [noise=true , address=145.79.227.16:4911 , piece_id=GEGCH2DAMKOX5KQQZ7EXNNNRIQBFNY2BY4GTJUI2UOMXPAPZV5VQ, seg_pos={0 0}, upload=187789]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: rpc: dial tcp 145.79.227.16:4911: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:41:18.731320732 (195h30m21.746990343s) [noise=true , address=145.79.227.50:4378 , piece_id=5OFIICDCHFAS37MXJ7LG66VLPLOYZEDEAGIJQ2TZERHYWTIC6DWQ, seg_pos={0 0}, upload=187789]: Done uploading piece. err: ecclient: upload failed (node:1DcpXz1fKrWhdPxTpGUFmydMUuzMQpNFMgGB75BAVrjdMiSkhW, address:145.79.227.50:4378): protocol: rpc: dial tcp 145.79.227.50:4378: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:41:18.731629084 (195h30m21.747298694s) [noise=true , address=145.79.227.22:4514 , piece_id=UPUVWD4DI2HEFAUZ7TWMRW6BEZRA6EMU4A6XWMCXDT26BFTUAM4A, seg_pos={0 0}, upload=187789]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: rpc: dial tcp 145.79.227.22:4514: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:41:18.731681443 (195h30m21.747351044s) [noise=true , address=145.79.227.48:4303 , piece_id=E2E3LRCQJQ23NR2MYL4TECQFPSTBA7PX23GMP4BIC5O3N7QAYQYA, seg_pos={0 0}, upload=187789]: Done uploading piece. err: ecclient: upload failed (node:12gJKrCBLJWN1SBKibfeqBtcqqiv4veKTGmtnycjZyb3VN9kwoT, address:145.79.227.48:4303): protocol: rpc: dial tcp 145.79.227.48:4303: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:41:18.731754222 (195h30m21.747423833s) [noise=true , address=145.79.227.10:4404 , piece_id=USIYPXJ4PUIBNF33AACAUHCDQWEA2TYCSNODKPJMQ3WEHR4FWE6A, seg_pos={0 0}, upload=187789]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: rpc: dial tcp 145.79.227.10:4404: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:41:18.731964080 (195h30m21.747633692s) [noise=true , address=145.79.227.26:4656 , piece_id=K26SLR2MITYNRJIBSYPZSIGHLASEKZAPNL2LIQ3ZP4V7ZI27AOWA, seg_pos={0 0}, upload=187789]: Done uploading piece. err: ecclient: upload failed (node:1u6e2ZjrPg2q3EDN2ZjDj9hzEmCAoRvt39HMeM79gcVmMVm53j, address:145.79.227.26:4656): protocol: rpc: dial tcp 145.79.227.26:4656: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:41:18.731994895 (195h30m21.747664496s) [seg_pos={0 0}, upload=187789]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Mar 18 19:41:18.855455980 (195h30m21.871125589s) [upload=187789]: Done uploading segments... Mar 18 19:41:18.855505095 (195h30m21.871174704s) [upload=187789]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 30.127392209s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [...................................................XXXXXXXX.] duration: 3.657762033s trace: https://jaeger.dev.storj.io/trace/27979b4cc9984479 FAIL (2026-03-18 19:41:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.657791116s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [....................................................XXX.XXX.] duration: 3.196873117s trace: https://jaeger.dev.storj.io/trace/4d6a8ab787f335c0 FAIL (2026-03-18 19:41:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.196889578s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [.....................................................XXX.XXX] duration: 1m0.001431543s trace: https://jaeger.dev.storj.io/trace/4d6a8ab787f335c0 FAIL (2026-03-18 19:40:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001454386s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................................................X..X...X.] duration: 3.338057171s trace: https://jaeger.dev.storj.io/trace/7bb5f3f9442e399e FAIL (2026-03-18 19:40:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== oad=187775]: Uploading segments... Mar 18 19:39:51.245373273 (195h28m54.261042883s) [upload=187775]: Got next segment. Inline: false Mar 18 19:39:51.245450148 (195h28m54.261119759s) [upload=187775]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 19:39:51.245457117 (195h28m54.261126725s) [upload=187775]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 19:39:51.356664730 (195h28m54.372334331s) [seg_pos={0 0}, upload=187775]: Begin upload segment... Mar 18 19:39:51.356741101 (195h28m54.372410707s) [seg_pos={0 0}, upload=187775]: Done begin upload segment. Mar 18 19:39:51.356816902 (195h28m54.372486505s) [noise=true , address=45.91.116.12:4257 , piece_id=YDLJDFP3Q7VL2JANZL366KGZXB3EI4MG5PWEBDPYPJW7IXY77NRQ, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.356876970 (195h28m54.372546579s) [noise=true , address=167.17.72.71:4811 , piece_id=GDYHOQWV5KNLDWZRWX2S7YXGQ77KMN65ECAMSXGHEAIMJMIDIKVQ, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.357042162 (195h28m54.372711762s) [noise=true , address=145.79.227.37:4574 , piece_id=XRLKWPUQPPQJOCHD7BSZMS3EZGIXLRK2WFVZARB2T5IPDFQB3SDA, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.357958364 (195h28m54.373627987s) [noise=true , address=79.127.219.46:4919 , piece_id=7E5AC5BG7OUFDQXKAQVVFIKCNMBCH44YHGPHJXV6WLL6VZNAYVCQ, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.357989629 (195h28m54.373659229s) [noise=true , address=45.91.116.10:4087 , piece_id=ZS6W5PCUKBUVCNWLGIUP24WZ4Z6JQYNHSPGEI7RVV367NFBU324A, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.358579404 (195h28m54.37424901s) [noise=true , address=167.17.72.69:4741 , piece_id=2UMXKHMNVA52AOVHBHO2YHJ6BPHBQ4NL3O4S32LFVZKY524GGKUA, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.358785953 (195h28m54.374455555s) [noise=true , address=145.79.227.31:4821 , piece_id=2RY5OW2QLCFMX4Q3ZYHS3LBKHRUAOGGWDDGZCDWBRXGLTRJZGQNQ, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.359325146 (195h28m54.374994746s) [noise=true , address=193.149.164.71:4293 , piece_id=U3Y47GEFDQZZYSQWZPXLIKPQX7KOSATYYXSSQ2LLZRN5R5A4X43A, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.359888775 (195h28m54.375558385s) [noise=true , address=145.79.227.50:4393 , piece_id=MKQZD5DRQKJGDJRKUUPIHIBHQ5HAVABPTTQDYUQ76L4V5IXE2XIQ, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.360658986 (195h28m54.376328586s) [noise=true , address=45.91.116.11:4141 , piece_id=ALRFXBSV356RLTAAODD6LCRGVMIW4YCLWLV3J7CCDLQ4KFQWGJKA, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.360990163 (195h28m54.376659767s) [noise=true , address=45.91.116.13:4185 , piece_id=DCRQDPZMQB3BR6ST4OSI2UURGDC5E2PUUZVU4GJMOJDMLHQ2EFOA, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.361395094 (195h28m54.377064703s) [noise=true , address=167.17.72.70:4752 , piece_id=ZWHC3OAFEVAHYOMXNP22YLHVJOTR72NAMCZKMNKY6EP7ZY6GMP4A, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.361842432 (195h28m54.377512034s) [noise=true , address=64.49.13.27:4957 , piece_id=37IAXM3YBV6IOZGC3DM5QEDYQTM6HSAHS5UFADOIMPEGK3QF3BTQ, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.362135261 (195h28m54.377804865s) [noise=true , address=145.79.227.33:4882 , piece_id=KEOJNYHAH5QPIFGGDB4PQNKJ4N5VQY3USQD2NWFLCH3TJKIIAP3A, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.362663307 (195h28m54.378332918s) [noise=true , address=193.149.164.66:4977 , piece_id=7HOES66K6XAZBO5IF3O6K4HDX27SYMMYWBMSMLX2URPKN4SSFEZA, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.362795143 (195h28m54.378464749s) [noise=true , address=145.79.227.43:4180 , piece_id=GGTUQNDLFS2QXTKOOFAA3QIIES7OW32QKK3V4VLQ2F2F74ITNWFQ, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.363148171 (195h28m54.378817777s) [noise=true , address=45.91.116.5:4323 , piece_id=M4IXPUQQZSM75YMO567TTF3OEQV2Y5AY723HLLLKVSYVK2J4N4PQ, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.363481817 (195h28m54.379151417s) [noise=true , address=79.127.219.41:4958 , piece_id=OM62E6RDF7K5BUXBTZ4YFWRLEQ46W2ZCAY27WZ75ZEAYYI4ANKTQ, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.914452486 (195h28m54.930122086s) [upload=187775]: Next returned nil segment Mar 18 19:39:51.914477088 (195h28m54.930146689s) [upload=187775]: Waiting for error group managing segments... Mar 18 19:39:52.065063812 (195h28m55.080733412s) [noise=true , address=45.91.116.10:4087 , piece_id=ZS6W5PCUKBUVCNWLGIUP24WZ4Z6JQYNHSPGEI7RVV367NFBU324A, seg_pos={0 0}, upload=187775]: Done uploading piece. err: <nil> Mar 18 19:39:52.075576360 (195h28m55.09124596s) [noise=true , address=45.91.116.5:4323 , piece_id=M4IXPUQQZSM75YMO567TTF3OEQV2Y5AY723HLLLKVSYVK2J4N4PQ, seg_pos={0 0}, upload=187775]: Done uploading piece. err: <nil> Mar 18 19:39:52.076559539 (195h28m55.09222914s) [noise=true , address=45.91.116.13:4185 , piece_id=DCRQDPZMQB3BR6ST4OSI2UURGDC5E2PUUZVU4GJMOJDMLHQ2EFOA, seg_pos={0 0}, upload=187775]: Done uploading piece. err: <nil> Mar 18 19:39:52.079832594 (195h28m55.095502205s) [noise=true , address=45.91.116.11:4141 , piece_id=ALRFXBSV356RLTAAODD6LCRGVMIW4YCLWLV3J7CCDLQ4KFQWGJKA, seg_pos={0 0}, upload=187775]: Done uploading piece. err: <nil> Mar 18 19:39:52.094457996 (195h28m55.110127595s) [noise=true , address=79.127.219.46:4919 , piece_id=7E5AC5BG7OUFDQXKAQVVFIKCNMBCH44YHGPHJXV6WLL6VZNAYVCQ, seg_pos={0 0}, upload=187775]: Done uploading piece. err: <nil> Mar 18 19:39:52.116381598 (195h28m55.132051201s) [noise=true , address=79.127.219.41:4958 , piece_id=OM62E6RDF7K5BUXBTZ4YFWRLEQ46W2ZCAY27WZ75ZEAYYI4ANKTQ, seg_pos={0 0}, upload=187775]: Done uploading piece. err: <nil> Mar 18 19:39:52.253087456 (195h28m55.268757064s) [noise=true , address=45.91.116.12:4257 , piece_id=YDLJDFP3Q7VL2JANZL366KGZXB3EI4MG5PWEBDPYPJW7IXY77NRQ, seg_pos={0 0}, upload=187775]: Done uploading piece. err: <nil> Mar 18 19:39:52.483396325 (195h28m55.499065936s) [noise=true , address=167.17.72.71:4811 , piece_id=GDYHOQWV5KNLDWZRWX2S7YXGQ77KMN65ECAMSXGHEAIMJMIDIKVQ, seg_pos={0 0}, upload=187775]: Done uploading piece. err: <nil> Mar 18 19:39:52.500200438 (195h28m55.515870037s) [noise=true , address=167.17.72.69:4741 , piece_id=2UMXKHMNVA52AOVHBHO2YHJ6BPHBQ4NL3O4S32LFVZKY524GGKUA, seg_pos={0 0}, upload=187775]: Done uploading piece. err: <nil> Mar 18 19:39:52.565217723 (195h28m55.580887332s) [noise=true , address=167.17.72.70:4752 , piece_id=ZWHC3OAFEVAHYOMXNP22YLHVJOTR72NAMCZKMNKY6EP7ZY6GMP4A, seg_pos={0 0}, upload=187775]: Done uploading piece. err: <nil> Mar 18 19:39:52.590846650 (195h28m55.606516251s) [noise=true , address=64.49.13.27:4957 , piece_id=37IAXM3YBV6IOZGC3DM5QEDYQTM6HSAHS5UFADOIMPEGK3QF3BTQ, seg_pos={0 0}, upload=187775]: Done uploading piece. err: <nil> Mar 18 19:39:52.598323339 (195h28m55.613992939s) [noise=true , address=193.149.164.66:4977 , piece_id=7HOES66K6XAZBO5IF3O6K4HDX27SYMMYWBMSMLX2URPKN4SSFEZA, seg_pos={0 0}, upload=187775]: Done uploading piece. err: <nil> Mar 18 19:39:52.746791574 (195h28m55.762461183s) [noise=true , address=193.149.164.71:4293 , piece_id=U3Y47GEFDQZZYSQWZPXLIKPQX7KOSATYYXSSQ2LLZRN5R5A4X43A, seg_pos={0 0}, upload=187775]: Done uploading piece. err: <nil> Mar 18 19:40:50.958698173 (195h29m53.974367782s) [noise=true , address=145.79.227.50:4393 , piece_id=MKQZD5DRQKJGDJRKUUPIHIBHQ5HAVABPTTQDYUQ76L4V5IXE2XIQ, seg_pos={0 0}, upload=187775]: Done uploading piece. err: ecclient: upload failed (node:1xfCSdYjRRxMaTw47dQmaXzGmMWyV1oCCGx7JdhvGqaaoN14Ff, address:145.79.227.50:4393): protocol: rpc: dial tcp 145.79.227.50:4393: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:40:50.958929147 (195h29m53.974598757s) [noise=true , address=145.79.227.33:4882 , piece_id=KEOJNYHAH5QPIFGGDB4PQNKJ4N5VQY3USQD2NWFLCH3TJKIIAP3A, seg_pos={0 0}, upload=187775]: Done uploading piece. err: ecclient: upload failed (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc, address:145.79.227.33:4882): protocol: rpc: dial tcp 145.79.227.33:4882: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:40:50.959152478 (195h29m53.974822088s) [noise=true , address=145.79.227.37:4574 , piece_id=XRLKWPUQPPQJOCHD7BSZMS3EZGIXLRK2WFVZARB2T5IPDFQB3SDA, seg_pos={0 0}, upload=187775]: Done uploading piece. err: ecclient: upload failed (node:1tNoERpZQJ2K3F56AhqUErsCMoF2adJfye5rrEoSi8dnjimcnw, address:145.79.227.37:4574): protocol: rpc: dial tcp 145.79.227.37:4574: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:40:50.959352897 (195h29m53.975022507s) [noise=true , address=145.79.227.31:4821 , piece_id=2RY5OW2QLCFMX4Q3ZYHS3LBKHRUAOGGWDDGZCDWBRXGLTRJZGQNQ, seg_pos={0 0}, upload=187775]: Done uploading piece. err: ecclient: upload failed (node:1Q2vaBr3JNY3jv8jVUTg9uzfWUtNjrd5c5Z9Yfi54D3JMRM1af, address:145.79.227.31:4821): protocol: rpc: dial tcp 145.79.227.31:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:40:50.959577488 (195h29m53.975247097s) [noise=true , address=145.79.227.43:4180 , piece_id=GGTUQNDLFS2QXTKOOFAA3QIIES7OW32QKK3V4VLQ2F2F74ITNWFQ, seg_pos={0 0}, upload=187775]: Done uploading piece. err: ecclient: upload failed (node:1rhkcipPBZNFScBjBhWdigzSMwENSmzhCucnC2iuyc9udpweE3, address:145.79.227.43:4180): protocol: rpc: dial tcp 145.79.227.43:4180: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:40:50.959606832 (195h29m53.975276422s) [seg_pos={0 0}, upload=187775]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Mar 18 19:40:51.064815222 (195h29m54.080484816s) [upload=187775]: Done uploading segments... Mar 18 19:40:51.064854034 (195h29m54.080523634s) [upload=187775]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 3.338076288s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [....................................................X..X...X] duration: 1m0.107980341s trace: https://jaeger.dev.storj.io/trace/7bb5f3f9442e399e FAIL (2026-03-18 19:40:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== oad=187775]: Uploading segments... Mar 18 19:39:51.245373273 (195h28m54.261042883s) [upload=187775]: Got next segment. Inline: false Mar 18 19:39:51.245450148 (195h28m54.261119759s) [upload=187775]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 19:39:51.245457117 (195h28m54.261126725s) [upload=187775]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 19:39:51.356664730 (195h28m54.372334331s) [seg_pos={0 0}, upload=187775]: Begin upload segment... Mar 18 19:39:51.356741101 (195h28m54.372410707s) [seg_pos={0 0}, upload=187775]: Done begin upload segment. Mar 18 19:39:51.356816902 (195h28m54.372486505s) [noise=true , address=45.91.116.12:4257 , piece_id=YDLJDFP3Q7VL2JANZL366KGZXB3EI4MG5PWEBDPYPJW7IXY77NRQ, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.356876970 (195h28m54.372546579s) [noise=true , address=167.17.72.71:4811 , piece_id=GDYHOQWV5KNLDWZRWX2S7YXGQ77KMN65ECAMSXGHEAIMJMIDIKVQ, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.357042162 (195h28m54.372711762s) [noise=true , address=145.79.227.37:4574 , piece_id=XRLKWPUQPPQJOCHD7BSZMS3EZGIXLRK2WFVZARB2T5IPDFQB3SDA, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.357958364 (195h28m54.373627987s) [noise=true , address=79.127.219.46:4919 , piece_id=7E5AC5BG7OUFDQXKAQVVFIKCNMBCH44YHGPHJXV6WLL6VZNAYVCQ, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.357989629 (195h28m54.373659229s) [noise=true , address=45.91.116.10:4087 , piece_id=ZS6W5PCUKBUVCNWLGIUP24WZ4Z6JQYNHSPGEI7RVV367NFBU324A, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.358579404 (195h28m54.37424901s) [noise=true , address=167.17.72.69:4741 , piece_id=2UMXKHMNVA52AOVHBHO2YHJ6BPHBQ4NL3O4S32LFVZKY524GGKUA, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.358785953 (195h28m54.374455555s) [noise=true , address=145.79.227.31:4821 , piece_id=2RY5OW2QLCFMX4Q3ZYHS3LBKHRUAOGGWDDGZCDWBRXGLTRJZGQNQ, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.359325146 (195h28m54.374994746s) [noise=true , address=193.149.164.71:4293 , piece_id=U3Y47GEFDQZZYSQWZPXLIKPQX7KOSATYYXSSQ2LLZRN5R5A4X43A, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.359888775 (195h28m54.375558385s) [noise=true , address=145.79.227.50:4393 , piece_id=MKQZD5DRQKJGDJRKUUPIHIBHQ5HAVABPTTQDYUQ76L4V5IXE2XIQ, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.360658986 (195h28m54.376328586s) [noise=true , address=45.91.116.11:4141 , piece_id=ALRFXBSV356RLTAAODD6LCRGVMIW4YCLWLV3J7CCDLQ4KFQWGJKA, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.360990163 (195h28m54.376659767s) [noise=true , address=45.91.116.13:4185 , piece_id=DCRQDPZMQB3BR6ST4OSI2UURGDC5E2PUUZVU4GJMOJDMLHQ2EFOA, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.361395094 (195h28m54.377064703s) [noise=true , address=167.17.72.70:4752 , piece_id=ZWHC3OAFEVAHYOMXNP22YLHVJOTR72NAMCZKMNKY6EP7ZY6GMP4A, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.361842432 (195h28m54.377512034s) [noise=true , address=64.49.13.27:4957 , piece_id=37IAXM3YBV6IOZGC3DM5QEDYQTM6HSAHS5UFADOIMPEGK3QF3BTQ, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.362135261 (195h28m54.377804865s) [noise=true , address=145.79.227.33:4882 , piece_id=KEOJNYHAH5QPIFGGDB4PQNKJ4N5VQY3USQD2NWFLCH3TJKIIAP3A, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.362663307 (195h28m54.378332918s) [noise=true , address=193.149.164.66:4977 , piece_id=7HOES66K6XAZBO5IF3O6K4HDX27SYMMYWBMSMLX2URPKN4SSFEZA, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.362795143 (195h28m54.378464749s) [noise=true , address=145.79.227.43:4180 , piece_id=GGTUQNDLFS2QXTKOOFAA3QIIES7OW32QKK3V4VLQ2F2F74ITNWFQ, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.363148171 (195h28m54.378817777s) [noise=true , address=45.91.116.5:4323 , piece_id=M4IXPUQQZSM75YMO567TTF3OEQV2Y5AY723HLLLKVSYVK2J4N4PQ, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.363481817 (195h28m54.379151417s) [noise=true , address=79.127.219.41:4958 , piece_id=OM62E6RDF7K5BUXBTZ4YFWRLEQ46W2ZCAY27WZ75ZEAYYI4ANKTQ, seg_pos={0 0}, upload=187775]: Uploading piece... Mar 18 19:39:51.914452486 (195h28m54.930122086s) [upload=187775]: Next returned nil segment Mar 18 19:39:51.914477088 (195h28m54.930146689s) [upload=187775]: Waiting for error group managing segments... Mar 18 19:39:52.065063812 (195h28m55.080733412s) [noise=true , address=45.91.116.10:4087 , piece_id=ZS6W5PCUKBUVCNWLGIUP24WZ4Z6JQYNHSPGEI7RVV367NFBU324A, seg_pos={0 0}, upload=187775]: Done uploading piece. err: <nil> Mar 18 19:39:52.075576360 (195h28m55.09124596s) [noise=true , address=45.91.116.5:4323 , piece_id=M4IXPUQQZSM75YMO567TTF3OEQV2Y5AY723HLLLKVSYVK2J4N4PQ, seg_pos={0 0}, upload=187775]: Done uploading piece. err: <nil> Mar 18 19:39:52.076559539 (195h28m55.09222914s) [noise=true , address=45.91.116.13:4185 , piece_id=DCRQDPZMQB3BR6ST4OSI2UURGDC5E2PUUZVU4GJMOJDMLHQ2EFOA, seg_pos={0 0}, upload=187775]: Done uploading piece. err: <nil> Mar 18 19:39:52.079832594 (195h28m55.095502205s) [noise=true , address=45.91.116.11:4141 , piece_id=ALRFXBSV356RLTAAODD6LCRGVMIW4YCLWLV3J7CCDLQ4KFQWGJKA, seg_pos={0 0}, upload=187775]: Done uploading piece. err: <nil> Mar 18 19:39:52.094457996 (195h28m55.110127595s) [noise=true , address=79.127.219.46:4919 , piece_id=7E5AC5BG7OUFDQXKAQVVFIKCNMBCH44YHGPHJXV6WLL6VZNAYVCQ, seg_pos={0 0}, upload=187775]: Done uploading piece. err: <nil> Mar 18 19:39:52.116381598 (195h28m55.132051201s) [noise=true , address=79.127.219.41:4958 , piece_id=OM62E6RDF7K5BUXBTZ4YFWRLEQ46W2ZCAY27WZ75ZEAYYI4ANKTQ, seg_pos={0 0}, upload=187775]: Done uploading piece. err: <nil> Mar 18 19:39:52.253087456 (195h28m55.268757064s) [noise=true , address=45.91.116.12:4257 , piece_id=YDLJDFP3Q7VL2JANZL366KGZXB3EI4MG5PWEBDPYPJW7IXY77NRQ, seg_pos={0 0}, upload=187775]: Done uploading piece. err: <nil> Mar 18 19:39:52.483396325 (195h28m55.499065936s) [noise=true , address=167.17.72.71:4811 , piece_id=GDYHOQWV5KNLDWZRWX2S7YXGQ77KMN65ECAMSXGHEAIMJMIDIKVQ, seg_pos={0 0}, upload=187775]: Done uploading piece. err: <nil> Mar 18 19:39:52.500200438 (195h28m55.515870037s) [noise=true , address=167.17.72.69:4741 , piece_id=2UMXKHMNVA52AOVHBHO2YHJ6BPHBQ4NL3O4S32LFVZKY524GGKUA, seg_pos={0 0}, upload=187775]: Done uploading piece. err: <nil> Mar 18 19:39:52.565217723 (195h28m55.580887332s) [noise=true , address=167.17.72.70:4752 , piece_id=ZWHC3OAFEVAHYOMXNP22YLHVJOTR72NAMCZKMNKY6EP7ZY6GMP4A, seg_pos={0 0}, upload=187775]: Done uploading piece. err: <nil> Mar 18 19:39:52.590846650 (195h28m55.606516251s) [noise=true , address=64.49.13.27:4957 , piece_id=37IAXM3YBV6IOZGC3DM5QEDYQTM6HSAHS5UFADOIMPEGK3QF3BTQ, seg_pos={0 0}, upload=187775]: Done uploading piece. err: <nil> Mar 18 19:39:52.598323339 (195h28m55.613992939s) [noise=true , address=193.149.164.66:4977 , piece_id=7HOES66K6XAZBO5IF3O6K4HDX27SYMMYWBMSMLX2URPKN4SSFEZA, seg_pos={0 0}, upload=187775]: Done uploading piece. err: <nil> Mar 18 19:39:52.746791574 (195h28m55.762461183s) [noise=true , address=193.149.164.71:4293 , piece_id=U3Y47GEFDQZZYSQWZPXLIKPQX7KOSATYYXSSQ2LLZRN5R5A4X43A, seg_pos={0 0}, upload=187775]: Done uploading piece. err: <nil> Mar 18 19:40:50.958698173 (195h29m53.974367782s) [noise=true , address=145.79.227.50:4393 , piece_id=MKQZD5DRQKJGDJRKUUPIHIBHQ5HAVABPTTQDYUQ76L4V5IXE2XIQ, seg_pos={0 0}, upload=187775]: Done uploading piece. err: ecclient: upload failed (node:1xfCSdYjRRxMaTw47dQmaXzGmMWyV1oCCGx7JdhvGqaaoN14Ff, address:145.79.227.50:4393): protocol: rpc: dial tcp 145.79.227.50:4393: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:40:50.958929147 (195h29m53.974598757s) [noise=true , address=145.79.227.33:4882 , piece_id=KEOJNYHAH5QPIFGGDB4PQNKJ4N5VQY3USQD2NWFLCH3TJKIIAP3A, seg_pos={0 0}, upload=187775]: Done uploading piece. err: ecclient: upload failed (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc, address:145.79.227.33:4882): protocol: rpc: dial tcp 145.79.227.33:4882: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:40:50.959152478 (195h29m53.974822088s) [noise=true , address=145.79.227.37:4574 , piece_id=XRLKWPUQPPQJOCHD7BSZMS3EZGIXLRK2WFVZARB2T5IPDFQB3SDA, seg_pos={0 0}, upload=187775]: Done uploading piece. err: ecclient: upload failed (node:1tNoERpZQJ2K3F56AhqUErsCMoF2adJfye5rrEoSi8dnjimcnw, address:145.79.227.37:4574): protocol: rpc: dial tcp 145.79.227.37:4574: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:40:50.959352897 (195h29m53.975022507s) [noise=true , address=145.79.227.31:4821 , piece_id=2RY5OW2QLCFMX4Q3ZYHS3LBKHRUAOGGWDDGZCDWBRXGLTRJZGQNQ, seg_pos={0 0}, upload=187775]: Done uploading piece. err: ecclient: upload failed (node:1Q2vaBr3JNY3jv8jVUTg9uzfWUtNjrd5c5Z9Yfi54D3JMRM1af, address:145.79.227.31:4821): protocol: rpc: dial tcp 145.79.227.31:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:40:50.959577488 (195h29m53.975247097s) [noise=true , address=145.79.227.43:4180 , piece_id=GGTUQNDLFS2QXTKOOFAA3QIIES7OW32QKK3V4VLQ2F2F74ITNWFQ, seg_pos={0 0}, upload=187775]: Done uploading piece. err: ecclient: upload failed (node:1rhkcipPBZNFScBjBhWdigzSMwENSmzhCucnC2iuyc9udpweE3, address:145.79.227.43:4180): protocol: rpc: dial tcp 145.79.227.43:4180: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:40:50.959606832 (195h29m53.975276422s) [seg_pos={0 0}, upload=187775]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Mar 18 19:40:51.064815222 (195h29m54.080484816s) [upload=187775]: Done uploading segments... Mar 18 19:40:51.064854034 (195h29m54.080523634s) [upload=187775]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1m0.108004287s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [...................................................XXXXXXXXX] duration: 30.128007243s trace: https://jaeger.dev.storj.io/trace/6984de30a40911c8 FAIL (2026-03-18 19:40:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 9:56.497104947 (195h28m59.512774544s) [seg_pos={0 0}, upload=187778]: Done begin upload segment. Mar 18 19:39:56.497123579 (195h28m59.512793185s) [noise=true , address=145.79.227.3:4154 , piece_id=TXI4H6PQHBC5JXT5WMH4TIDTHCYCN7CXNHRMJ24LF3GB3PS65UZQ, seg_pos={0 0}, upload=187778]: Uploading piece... Mar 18 19:39:56.497151043 (195h28m59.512820647s) [noise=true , address=145.79.227.39:4016 , piece_id=4YME4I3GKUXXOR52CMVFACJ3F26FC3DDELIEMINIMKFKL6MXFEIQ, seg_pos={0 0}, upload=187778]: Uploading piece... Mar 18 19:39:56.497830334 (195h28m59.513499942s) [noise=true , address=145.79.227.4:4187 , piece_id=SFG6R4AIEDBFY6YBDMQQYQBI64S6KDR7DPDYKCYAWQSNVGPEWDHQ, seg_pos={0 0}, upload=187778]: Uploading piece... Mar 18 19:39:56.498297193 (195h28m59.513966793s) [noise=true , address=145.79.227.14:4841 , piece_id=JBADJPVVMMIX4VXCFZTFD2UQTBDAO2T4AALIZIMASVJQWGJCWZKA, seg_pos={0 0}, upload=187778]: Uploading piece... Mar 18 19:39:56.498643739 (195h28m59.51431334s) [noise=true , address=145.79.227.38:4980 , piece_id=7ASQBCWOAQIOIYGXAEB6DUZFQD5HFXZSVFBSZCNQRAWXNER47AZA, seg_pos={0 0}, upload=187778]: Uploading piece... Mar 18 19:39:56.499031086 (195h28m59.514700693s) [noise=true , address=145.79.227.2:4115 , piece_id=2HCAIXKSTW3QG3QDPZPLDSDQJJK6XK66INOJBUEASQZVXWOU7GIQ, seg_pos={0 0}, upload=187778]: Uploading piece... Mar 18 19:39:56.499480491 (195h28m59.515150096s) [noise=true , address=77.90.31.21:4881 , piece_id=UHTYPALT3K4UFWUAJR2JKHPRYCC53GR2MYCRCLDDBLGDN77XCCWQ, seg_pos={0 0}, upload=187778]: Uploading piece... Mar 18 19:39:56.499954880 (195h28m59.515624485s) [noise=true , address=145.79.227.52:4448 , piece_id=7RY44XVRABPIH4XDBHST5PNHCPFMWHHMOSNMNUBI3R3DR5TYHJZA, seg_pos={0 0}, upload=187778]: Uploading piece... Mar 18 19:39:56.500402178 (195h28m59.516071781s) [noise=true , address=145.79.227.26:4659 , piece_id=VLMJ6BKPFYDIAUVRTRIRZE5EEKAKO7UP3QKFX2S4MKHKVFGYVH5A, seg_pos={0 0}, upload=187778]: Uploading piece... Mar 18 19:39:56.500912087 (195h28m59.516581699s) [noise=true , address=77.90.31.17:4774 , piece_id=BV6DUZLPJ3R4YQBIRKB7RKYUE7XPET56ATMROEDR6UOB2SJRCIXQ, seg_pos={0 0}, upload=187778]: Uploading piece... Mar 18 19:39:56.501132597 (195h28m59.516802206s) [noise=true , address=45.91.116.9:4295 , piece_id=QWIJBOZTZGCIBOFJSDQ7CCYDWDFCZ2UJD4OKLCTTK575NCMOCS6Q, seg_pos={0 0}, upload=187778]: Uploading piece... Mar 18 19:39:56.501618346 (195h28m59.517287947s) [noise=true , address=145.79.227.51:4409 , piece_id=B262DBGHGZLM45LNIJCPLSJA7DIII6DDYWWGXD3V6LIXZHYWNPQA, seg_pos={0 0}, upload=187778]: Uploading piece... Mar 18 19:39:56.502156808 (195h28m59.517826415s) [noise=true , address=79.127.219.45:4854 , piece_id=AQSJBZOS2QJ4QQVEXAHW5SD3LMBWBDWFZVLII2NMU7YO542NJ5DQ, seg_pos={0 0}, upload=187778]: Uploading piece... Mar 18 19:39:56.502711624 (195h28m59.518381228s) [noise=true , address=145.79.227.27:4693 , piece_id=IB33LVJEUTACPDY2Z4WBRF5Z6H7DMHHJPOEXFPA4DOWSYTVCZM5A, seg_pos={0 0}, upload=187778]: Uploading piece... Mar 18 19:39:56.503260616 (195h28m59.518930226s) [noise=true , address=45.91.116.4:4940 , piece_id=VR2YO4DAZ5OKAUPDZUW32BFN2LKBSUPZUZ6M2BWDZBS5PRODAPHQ, seg_pos={0 0}, upload=187778]: Uploading piece... Mar 18 19:39:56.504115185 (195h28m59.519784792s) [noise=true , address=145.79.227.9:4370 , piece_id=XSF4GY2BYV3GDYCDBEECAVEAC7R7WBLP6MDM32YPVXQ2SBM43CLA, seg_pos={0 0}, upload=187778]: Uploading piece... Mar 18 19:39:57.106240793 (195h29m0.121910403s) [upload=187778]: Next returned nil segment Mar 18 19:39:57.106263043 (195h29m0.12193269s) [upload=187778]: Waiting for error group managing segments... Mar 18 19:39:57.240476472 (195h29m0.256146078s) [noise=true , address=45.91.116.9:4295 , piece_id=QWIJBOZTZGCIBOFJSDQ7CCYDWDFCZ2UJD4OKLCTTK575NCMOCS6Q, seg_pos={0 0}, upload=187778]: Done uploading piece. err: <nil> Mar 18 19:39:57.248765484 (195h29m0.264435094s) [noise=true , address=45.91.116.4:4940 , piece_id=VR2YO4DAZ5OKAUPDZUW32BFN2LKBSUPZUZ6M2BWDZBS5PRODAPHQ, seg_pos={0 0}, upload=187778]: Done uploading piece. err: <nil> Mar 18 19:39:57.257016798 (195h29m0.272686402s) [noise=true , address=77.90.31.17:4774 , piece_id=BV6DUZLPJ3R4YQBIRKB7RKYUE7XPET56ATMROEDR6UOB2SJRCIXQ, seg_pos={0 0}, upload=187778]: Done uploading piece. err: <nil> Mar 18 19:39:57.260722185 (195h29m0.276391786s) [noise=true , address=77.90.31.21:4881 , piece_id=UHTYPALT3K4UFWUAJR2JKHPRYCC53GR2MYCRCLDDBLGDN77XCCWQ, seg_pos={0 0}, upload=187778]: Done uploading piece. err: <nil> Mar 18 19:39:57.340986314 (195h29m0.356655917s) [noise=true , address=79.127.219.45:4854 , piece_id=AQSJBZOS2QJ4QQVEXAHW5SD3LMBWBDWFZVLII2NMU7YO542NJ5DQ, seg_pos={0 0}, upload=187778]: Done uploading piece. err: <nil> Mar 18 19:40:26.094105132 (195h29m29.109774738s) [noise=true , address=145.79.227.9:4370 , piece_id=XSF4GY2BYV3GDYCDBEECAVEAC7R7WBLP6MDM32YPVXQ2SBM43CLA, seg_pos={0 0}, upload=187778]: Done uploading piece. err: ecclient: upload failed (node:12SXZN7WJvcHys9BRd4Lq8NAZ3MSgHf9ZVv7SgaiJNRosdmFVpm, address:145.79.227.9:4370): protocol: rpc: dial tcp 145.79.227.9:4370: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:40:26.094313377 (195h29m29.109982989s) [noise=true , address=145.79.227.38:4980 , piece_id=7ASQBCWOAQIOIYGXAEB6DUZFQD5HFXZSVFBSZCNQRAWXNER47AZA, seg_pos={0 0}, upload=187778]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: rpc: dial tcp 145.79.227.38:4980: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:40:26.094355479 (195h29m29.110025086s) [noise=true , address=145.79.227.2:4115 , piece_id=2HCAIXKSTW3QG3QDPZPLDSDQJJK6XK66INOJBUEASQZVXWOU7GIQ, seg_pos={0 0}, upload=187778]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: rpc: dial tcp 145.79.227.2:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:40:26.094533145 (195h29m29.110202755s) [noise=true , address=145.79.227.27:4693 , piece_id=IB33LVJEUTACPDY2Z4WBRF5Z6H7DMHHJPOEXFPA4DOWSYTVCZM5A, seg_pos={0 0}, upload=187778]: Done uploading piece. err: ecclient: upload failed (node:11E39ZZcpnJR43M2RU3Ggb8neyXj6bbSJkANxdJNGuZJiH8c9d, address:145.79.227.27:4693): protocol: rpc: dial tcp 145.79.227.27:4693: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:40:26.094557836 (195h29m29.110227439s) [noise=true , address=145.79.227.39:4016 , piece_id=4YME4I3GKUXXOR52CMVFACJ3F26FC3DDELIEMINIMKFKL6MXFEIQ, seg_pos={0 0}, upload=187778]: Done uploading piece. err: ecclient: upload failed (node:1jC98jEWn6K8jFHPA9knJeHAkuNEmcS7ThxUvbySDfbq6FmKVR, address:145.79.227.39:4016): protocol: rpc: dial tcp 145.79.227.39:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:40:26.094740251 (195h29m29.110409863s) [noise=true , address=145.79.227.14:4841 , piece_id=JBADJPVVMMIX4VXCFZTFD2UQTBDAO2T4AALIZIMASVJQWGJCWZKA, seg_pos={0 0}, upload=187778]: Done uploading piece. err: ecclient: upload failed (node:12ndNj9Dy8qQSbHXwVWw31sz19ki1qy6Zgetjvj1A5Q3odkzmDm, address:145.79.227.14:4841): protocol: rpc: dial tcp 145.79.227.14:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:40:26.094772328 (195h29m29.110441919s) [noise=true , address=145.79.227.52:4448 , piece_id=7RY44XVRABPIH4XDBHST5PNHCPFMWHHMOSNMNUBI3R3DR5TYHJZA, seg_pos={0 0}, upload=187778]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: rpc: dial tcp 145.79.227.52:4448: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:40:26.094878748 (195h29m29.110548357s) [noise=true , address=145.79.227.3:4154 , piece_id=TXI4H6PQHBC5JXT5WMH4TIDTHCYCN7CXNHRMJ24LF3GB3PS65UZQ, seg_pos={0 0}, upload=187778]: Done uploading piece. err: ecclient: upload failed (node:12S7Xsfnti3QuD9apBq1sgSjmahnAYA6LYbXqyckbLY4UU84ZQF, address:145.79.227.3:4154): protocol: rpc: dial tcp 145.79.227.3:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:40:26.095000823 (195h29m29.110670425s) [noise=true , address=145.79.227.26:4659 , piece_id=VLMJ6BKPFYDIAUVRTRIRZE5EEKAKO7UP3QKFX2S4MKHKVFGYVH5A, seg_pos={0 0}, upload=187778]: Done uploading piece. err: ecclient: upload failed (node:12UBuWLogtN8mv4nwXDpzGdQtbKDz1EfVENSjXC54QdFp2fJg99, address:145.79.227.26:4659): protocol: rpc: dial tcp 145.79.227.26:4659: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:40:26.095022119 (195h29m29.110691722s) [noise=true , address=145.79.227.51:4409 , piece_id=B262DBGHGZLM45LNIJCPLSJA7DIII6DDYWWGXD3V6LIXZHYWNPQA, seg_pos={0 0}, upload=187778]: Done uploading piece. err: ecclient: upload failed (node:12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj, address:145.79.227.51:4409): protocol: rpc: dial tcp 145.79.227.51:4409: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:40:26.095341973 (195h29m29.111011582s) [noise=true , address=145.79.227.4:4187 , piece_id=SFG6R4AIEDBFY6YBDMQQYQBI64S6KDR7DPDYKCYAWQSNVGPEWDHQ, seg_pos={0 0}, upload=187778]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:40:26.095658227 (195h29m29.111327827s) [seg_pos={0 0}, upload=187778]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Mar 18 19:40:26.220376998 (195h29m29.2360466s) [upload=187778]: Done uploading segments... Mar 18 19:40:26.220416784 (195h29m29.236086387s) [upload=187778]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 30.128031033s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [....................................................XXXXXXXX] duration: 5.000664897s trace: https://jaeger.dev.storj.io/trace/27979b4cc9984479 FAIL (2026-03-18 19:40:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.000690658s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [......................................................XXX.XX] duration: 1m0.001116964s trace: https://jaeger.dev.storj.io/trace/3ab1023d88eaf413 FAIL (2026-03-18 19:39:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001141876s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................................X..X...] duration: 3.251231341s trace: https://jaeger.dev.storj.io/trace/2ca59d04a6d58ba0 FAIL (2026-03-18 19:39:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== pload=187737]: Uploading segments... Mar 18 19:37:12.981533109 (195h26m15.997202698s) [upload=187737]: Got next segment. Inline: false Mar 18 19:37:12.981598888 (195h26m15.997268489s) [upload=187737]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 19:37:12.981604720 (195h26m15.997274321s) [upload=187737]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 19:37:13.094870952 (195h26m16.110540559s) [seg_pos={0 0}, upload=187737]: Begin upload segment... Mar 18 19:37:13.094952447 (195h26m16.110622051s) [seg_pos={0 0}, upload=187737]: Done begin upload segment. Mar 18 19:37:13.095058924 (195h26m16.110728528s) [noise=true , address=77.90.31.19:4859 , piece_id=DSDO54RPJDJL74QLHBW5CWHNMQEUOWQXCPEX2KTA3YJXCJRH7Z2Q, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.095069831 (195h26m16.110739444s) [noise=true , address=77.90.31.11:4567 , piece_id=HUJKKLGH32QQAFHNPKPX4Q6ZTOT6PE2U3LGU2YGR3ZO53EOVZZDQ, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.095349501 (195h26m16.111019107s) [noise=true , address=79.127.219.45:4883 , piece_id=7NB4IV6P2KFYO62WORCI3OICZWN2QD4LVTXMP2AAXO2Q5Y65JGKQ, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.095876166 (195h26m16.11154576s) [noise=true , address=193.149.164.71:4298 , piece_id=JJ7AR3N4XQHCDCPGNWRU5VH5UBSBBLBPVLVPGR7UZGFB6WBF2DTA, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.095979098 (195h26m16.111648712s) [noise=true , address=145.79.227.4:4203 , piece_id=JQ7NCR6S3UVYLZ27OGYRIC75O2EGDRMS7VCZX5SADZAHAKR2IXIA, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.096545094 (195h26m16.112214708s) [noise=true , address=193.149.164.68:4052 , piece_id=DJN4FAGSUCIPGE4TP2CH2LANKMAQXLR3JYR3KONIEFWA5S3JGMTA, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.097159648 (195h26m16.11282925s) [noise=true , address=45.91.116.12:4237 , piece_id=XYQGCI2ET6MHS3YHU7NX5BAZKS2KNTOWRS5YTH24KPUOJGZMG3EA, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.097411902 (195h26m16.11308151s) [noise=true , address=79.127.219.46:4907 , piece_id=F552ZCGYCFVHQAVBL2UMTAR4JVFRX7HHBLUS7ZFL6XAQZVT7UPIQ, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.098280639 (195h26m16.113950249s) [noise=true , address=145.79.227.17:4977 , piece_id=EGN563W6WKAQ5XEMCVOWG62HDX6LRGDA4SNGFNIINCKS6IGDCYKQ, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.098832187 (195h26m16.114501796s) [noise=true , address=77.90.31.12:4609 , piece_id=VWCJAFHE5CQV6QKNCSXP4JG3YFWKOCC7LRGQGI3YNNGBLWKOECYQ, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.099348103 (195h26m16.115017706s) [noise=true , address=145.79.227.38:4986 , piece_id=POHAECJMSBQTPCZSBCUN3OTVKWTYFJHWTJ6KG3TQ4NJXLT4OYDMA, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.099809042 (195h26m16.115478648s) [noise=true , address=193.149.164.67:4010 , piece_id=6DIUZIS2OU3C55O52CQDWCAJVRHNYMQN4SXZXDXDYVK2B6LVDXLQ, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.100280986 (195h26m16.115950599s) [noise=true , address=167.17.72.71:4808 , piece_id=YOSBTFC3H6LTHYDW2I2KGJ3GH4ITNXJF6EUMMIWG3XJ2T4W4IJZA, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.101049801 (195h26m16.116719407s) [noise=true , address=193.149.164.66:4977 , piece_id=FYH4GQM4Q66RUWIFGFGI45OCIJGH2XNDJCF4L2N6UPYFCKJJHYFA, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.101995144 (195h26m16.117664757s) [noise=true , address=145.79.227.21:4485 , piece_id=HPET4TTOR3KPMYP6UVXMMIRZIVHWHEAXW3KJCLVF7LDEL4PJHBRQ, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.102105850 (195h26m16.117775458s) [noise=true , address=77.90.31.14:4687 , piece_id=2SNQMF43X42PS4DFJ4A4Q4PJQZ4W2P7V6INFCIFQJ2DOAWP7RV2Q, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.102538807 (195h26m16.118208418s) [noise=true , address=193.149.164.69:4095 , piece_id=OKKGKGL3OPLUUTNAML7KS7Z355OQTQADDU7ZH4VEJGNHDMWE6ZAA, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.102937431 (195h26m16.118607039s) [noise=true , address=145.79.227.43:4175 , piece_id=TQRTZBPACZZ3SU4CZ4C2IWS7EOFBL5ES4H3SAHSZXSWOZHTLMJTA, seg_pos={0 0}, upload=187737]: Uploading piece... Mar 18 19:37:13.654721924 (195h26m16.670391534s) [upload=187737]: Next returned nil segment Mar 18 19:37:13.654739810 (195h26m16.670409421s) [upload=187737]: Waiting for error group managing segments... Mar 18 19:37:13.794878973 (195h26m16.810548577s) [noise=true , address=77.90.31.12:4609 , piece_id=VWCJAFHE5CQV6QKNCSXP4JG3YFWKOCC7LRGQGI3YNNGBLWKOECYQ, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:13.822282354 (195h26m16.837951958s) [noise=true , address=77.90.31.11:4567 , piece_id=HUJKKLGH32QQAFHNPKPX4Q6ZTOT6PE2U3LGU2YGR3ZO53EOVZZDQ, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:13.824574466 (195h26m16.84024407s) [noise=true , address=79.127.219.45:4883 , piece_id=7NB4IV6P2KFYO62WORCI3OICZWN2QD4LVTXMP2AAXO2Q5Y65JGKQ, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:13.824736917 (195h26m16.840406527s) [noise=true , address=45.91.116.12:4237 , piece_id=XYQGCI2ET6MHS3YHU7NX5BAZKS2KNTOWRS5YTH24KPUOJGZMG3EA, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:13.836649120 (195h26m16.852318721s) [noise=true , address=79.127.219.46:4907 , piece_id=F552ZCGYCFVHQAVBL2UMTAR4JVFRX7HHBLUS7ZFL6XAQZVT7UPIQ, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:13.882465904 (195h26m16.898135505s) [noise=true , address=77.90.31.19:4859 , piece_id=DSDO54RPJDJL74QLHBW5CWHNMQEUOWQXCPEX2KTA3YJXCJRH7Z2Q, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:13.899738045 (195h26m16.915407652s) [noise=true , address=77.90.31.14:4687 , piece_id=2SNQMF43X42PS4DFJ4A4Q4PJQZ4W2P7V6INFCIFQJ2DOAWP7RV2Q, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:14.163170463 (195h26m17.178840082s) [noise=true , address=167.17.72.71:4808 , piece_id=YOSBTFC3H6LTHYDW2I2KGJ3GH4ITNXJF6EUMMIWG3XJ2T4W4IJZA, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:14.313991205 (195h26m17.329660806s) [noise=true , address=193.149.164.71:4298 , piece_id=JJ7AR3N4XQHCDCPGNWRU5VH5UBSBBLBPVLVPGR7UZGFB6WBF2DTA, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:14.317475737 (195h26m17.333145341s) [noise=true , address=193.149.164.67:4010 , piece_id=6DIUZIS2OU3C55O52CQDWCAJVRHNYMQN4SXZXDXDYVK2B6LVDXLQ, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:14.318408375 (195h26m17.334077977s) [noise=true , address=193.149.164.66:4977 , piece_id=FYH4GQM4Q66RUWIFGFGI45OCIJGH2XNDJCF4L2N6UPYFCKJJHYFA, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:14.321516393 (195h26m17.337185997s) [noise=true , address=193.149.164.69:4095 , piece_id=OKKGKGL3OPLUUTNAML7KS7Z355OQTQADDU7ZH4VEJGNHDMWE6ZAA, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:37:14.346324910 (195h26m17.361994521s) [noise=true , address=193.149.164.68:4052 , piece_id=DJN4FAGSUCIPGE4TP2CH2LANKMAQXLR3JYR3KONIEFWA5S3JGMTA, seg_pos={0 0}, upload=187737]: Done uploading piece. err: <nil> Mar 18 19:38:12.698035371 (195h27m15.713704979s) [noise=true , address=145.79.227.21:4485 , piece_id=HPET4TTOR3KPMYP6UVXMMIRZIVHWHEAXW3KJCLVF7LDEL4PJHBRQ, seg_pos={0 0}, upload=187737]: Done uploading piece. err: ecclient: upload failed (node:1wmohWyFaMkNm8KBAZprmZ6RuiH6UcB8KRNght2HGpV3Ucnnq9, address:145.79.227.21:4485): protocol: rpc: dial tcp 145.79.227.21:4485: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:38:12.698112463 (195h27m15.713782073s) [noise=true , address=145.79.227.17:4977 , piece_id=EGN563W6WKAQ5XEMCVOWG62HDX6LRGDA4SNGFNIINCKS6IGDCYKQ, seg_pos={0 0}, upload=187737]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:145.79.227.17:4977): protocol: rpc: dial tcp 145.79.227.17:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:38:12.698176988 (195h27m15.713846593s) [noise=true , address=145.79.227.38:4986 , piece_id=POHAECJMSBQTPCZSBCUN3OTVKWTYFJHWTJ6KG3TQ4NJXLT4OYDMA, seg_pos={0 0}, upload=187737]: Done uploading piece. err: ecclient: upload failed (node:1xiyNnuK7RcA3pYJRz2QStzUVfaU6gVUovxX2gV86xAQ2K9np4, address:145.79.227.38:4986): protocol: rpc: dial tcp 145.79.227.38:4986: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:38:12.698193160 (195h27m15.713862769s) [noise=true , address=145.79.227.43:4175 , piece_id=TQRTZBPACZZ3SU4CZ4C2IWS7EOFBL5ES4H3SAHSZXSWOZHTLMJTA, seg_pos={0 0}, upload=187737]: Done uploading piece. err: ecclient: upload failed (node:12ou7jwFyJP3mNoALddniUTDT9duVyQKZjeq4xHCyx6sRWpwEPJ, address:145.79.227.43:4175): protocol: rpc: dial tcp 145.79.227.43:4175: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:38:12.698493447 (195h27m15.714163057s) [noise=true , address=145.79.227.4:4203 , piece_id=JQ7NCR6S3UVYLZ27OGYRIC75O2EGDRMS7VCZX5SADZAHAKR2IXIA, seg_pos={0 0}, upload=187737]: Done uploading piece. err: ecclient: upload failed (node:18cNSsgqn3n6z5LxRcVCRzDzfFDpvptV1MKh6NdNRLW4bg9fWM, address:145.79.227.4:4203): protocol: rpc: dial tcp 145.79.227.4:4203: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:38:12.698532860 (195h27m15.714202463s) [seg_pos={0 0}, upload=187737]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Mar 18 19:38:12.801009852 (195h27m15.816679453s) [upload=187737]: Done uploading segments... Mar 18 19:38:12.801045568 (195h27m15.81671517s) [upload=187737]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 3.251258683s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [....................................................XXXXXXXX] duration: 30.116886964s trace: https://jaeger.dev.storj.io/trace/30de46d41ef34c32 FAIL (2026-03-18 19:39:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== WS3Q, seg_pos={0 0}, upload=187758]: Uploading piece... Mar 18 19:38:42.829887842 (195h27m45.845557447s) [noise=true , address=77.90.31.14:4666 , piece_id=6PB7ZGNPOI54U6GJC4STBZVW2UHABI3F6XTOI3LDKBWOBF3KQXLQ, seg_pos={0 0}, upload=187758]: Uploading piece... Mar 18 19:38:42.830194632 (195h27m45.845864241s) [noise=true , address=145.79.227.8:4333 , piece_id=HSZRKVQPRJHQSWWNPQCQ2OU2UCF4VYPUPCWZ5XCLMOWCAGYRJZGA, seg_pos={0 0}, upload=187758]: Uploading piece... Mar 18 19:38:42.830466083 (195h27m45.846135684s) [noise=true , address=145.79.227.24:4585 , piece_id=6LYC64R4P7XRG52GKQJA66JS3VU5G67EV3BOLKPBKH6CEULKLS5A, seg_pos={0 0}, upload=187758]: Uploading piece... Mar 18 19:38:42.830564187 (195h27m45.84623379s) [noise=true , address=145.79.227.48:4301 , piece_id=TPUKCYEBPSSDKRKWE4C6D4FXP45ZKOLHCAMUIGWRY2Z6CIF3IAOQ, seg_pos={0 0}, upload=187758]: Uploading piece... Mar 18 19:38:42.830952600 (195h27m45.846622202s) [noise=true , address=145.79.227.46:4601 , piece_id=OAUU3BLGYN24DXZGB45LDWKK2RHUBJACYWZKCWOXJDE7MYMNUXVQ, seg_pos={0 0}, upload=187758]: Uploading piece... Mar 18 19:38:42.831379290 (195h27m45.847048902s) [noise=true , address=145.79.227.50:4377 , piece_id=LJ2YNNF7ZSG3E5M47HCCEGLD2TQX5HKBLDESO2KAATERTELEOMOA, seg_pos={0 0}, upload=187758]: Uploading piece... Mar 18 19:38:42.831415957 (195h27m45.847085571s) [noise=true , address=79.127.219.38:4817 , piece_id=VOFDTDTQC223KGDRV4ESWIJ4TMHDCTTDJUSMW2RNVLLMISOIG2UQ, seg_pos={0 0}, upload=187758]: Uploading piece... Mar 18 19:38:42.831763693 (195h27m45.847433293s) [noise=true , address=145.79.227.44:4198 , piece_id=OC2ZBXSU5H4MA2T3GUR4YZLPM7IJTHMCGESIW5OAXBWYQF5ABYCA, seg_pos={0 0}, upload=187758]: Uploading piece... Mar 18 19:38:42.831911333 (195h27m45.847580943s) [noise=true , address=145.79.227.42:4124 , piece_id=NUOMBSSUW3QUAHLMNAPQGX3TWFM32IS6CWRYSP5OIRMBSVJGJHJA, seg_pos={0 0}, upload=187758]: Uploading piece... Mar 18 19:38:42.832426282 (195h27m45.848095892s) [noise=true , address=145.79.227.22:4512 , piece_id=BEIM3QFB72BA5JYLFFQY7DNT6FZMP5E3DBH64QDMY2ZE2WLV6OXQ, seg_pos={0 0}, upload=187758]: Uploading piece... Mar 18 19:38:42.832979371 (195h27m45.848648975s) [noise=true , address=45.91.116.7:4009 , piece_id=W6QXHF5SSQXQEK6BRY2I6FKE5ERJUK6KMDSWHAMR5P7LFRQOTAMQ, seg_pos={0 0}, upload=187758]: Uploading piece... Mar 18 19:38:42.835116162 (195h27m45.850785764s) [noise=true , address=145.79.227.3:4152 , piece_id=TY3RIGGAURPFSQ6BNO7IYHAQ3HQ6HAUCYI2F5GII22CSQI5OPH5A, seg_pos={0 0}, upload=187758]: Uploading piece... Mar 18 19:38:42.835961415 (195h27m45.85163102s) [noise=true , address=145.79.227.45:4233 , piece_id=RNCEVV2C42IBOO6QAILJ4OV37R22TXLHXVQTD4NFILVWD5ISJM3A, seg_pos={0 0}, upload=187758]: Uploading piece... Mar 18 19:38:42.837604324 (195h27m45.853273933s) [noise=true , address=145.79.227.10:4404 , piece_id=RGW6TG5MKRJX3FWOR6MTCH2R4BGBW575G5UMUMSH7IIFMRPAOEFA, seg_pos={0 0}, upload=187758]: Uploading piece... Mar 18 19:38:43.432064198 (195h27m46.447733788s) [upload=187758]: Next returned nil segment Mar 18 19:38:43.432078995 (195h27m46.447748602s) [upload=187758]: Waiting for error group managing segments... Mar 18 19:38:43.578588252 (195h27m46.59425786s) [noise=true , address=77.90.31.14:4666 , piece_id=6PB7ZGNPOI54U6GJC4STBZVW2UHABI3F6XTOI3LDKBWOBF3KQXLQ, seg_pos={0 0}, upload=187758]: Done uploading piece. err: <nil> Mar 18 19:38:43.624899643 (195h27m46.640569244s) [noise=true , address=45.91.116.7:4009 , piece_id=W6QXHF5SSQXQEK6BRY2I6FKE5ERJUK6KMDSWHAMR5P7LFRQOTAMQ, seg_pos={0 0}, upload=187758]: Done uploading piece. err: <nil> Mar 18 19:38:43.676101599 (195h27m46.691771199s) [noise=true , address=79.127.219.38:4817 , piece_id=VOFDTDTQC223KGDRV4ESWIJ4TMHDCTTDJUSMW2RNVLLMISOIG2UQ, seg_pos={0 0}, upload=187758]: Done uploading piece. err: <nil> Mar 18 19:39:12.434772381 (195h28m15.450441983s) [noise=true , address=145.79.227.22:4512 , piece_id=BEIM3QFB72BA5JYLFFQY7DNT6FZMP5E3DBH64QDMY2ZE2WLV6OXQ, seg_pos={0 0}, upload=187758]: Done uploading piece. err: ecclient: upload failed (node:12cnVqYYfo8b6RvRtMykzh8mU7UAqxcqYvWe7H8G7k8DqtNLAR4, address:145.79.227.22:4512): protocol: rpc: dial tcp 145.79.227.22:4512: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:39:12.434795920 (195h28m15.450465513s) [noise=true , address=145.79.227.46:4601 , piece_id=OAUU3BLGYN24DXZGB45LDWKK2RHUBJACYWZKCWOXJDE7MYMNUXVQ, seg_pos={0 0}, upload=187758]: Done uploading piece. err: ecclient: upload failed (node:125GxLsL5UtXkuJ1LFvF42N9iZkKtRK3rq63xdR25JQbtwoT1L5, address:145.79.227.46:4601): protocol: rpc: dial tcp 145.79.227.46:4601: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:39:12.435048427 (195h28m15.45071803s) [noise=true , address=145.79.227.24:4585 , piece_id=6LYC64R4P7XRG52GKQJA66JS3VU5G67EV3BOLKPBKH6CEULKLS5A, seg_pos={0 0}, upload=187758]: Done uploading piece. err: ecclient: upload failed (node:1TyK6onzVds1kucaD6Y4aPssPDM5FzzQ91dTrez5B7ttfwvXm, address:145.79.227.24:4585): protocol: rpc: dial tcp 145.79.227.24:4585: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:39:12.435224217 (195h28m15.450893828s) [noise=true , address=145.79.227.10:4404 , piece_id=RGW6TG5MKRJX3FWOR6MTCH2R4BGBW575G5UMUMSH7IIFMRPAOEFA, seg_pos={0 0}, upload=187758]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: rpc: dial tcp 145.79.227.10:4404: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:39:12.435312729 (195h28m15.450982337s) [noise=true , address=145.79.227.48:4301 , piece_id=TPUKCYEBPSSDKRKWE4C6D4FXP45ZKOLHCAMUIGWRY2Z6CIF3IAOQ, seg_pos={0 0}, upload=187758]: Done uploading piece. err: ecclient: upload failed (node:18HkaSrdHHnvmeNz1ormfcSBZH8bMUgkX7X3JmGuqkmtBoak29, address:145.79.227.48:4301): protocol: rpc: dial tcp 145.79.227.48:4301: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:39:12.435407034 (195h28m15.451076645s) [noise=true , address=145.79.227.45:4233 , piece_id=RNCEVV2C42IBOO6QAILJ4OV37R22TXLHXVQTD4NFILVWD5ISJM3A, seg_pos={0 0}, upload=187758]: Done uploading piece. err: ecclient: upload failed (node:1DmQeoQWTnFhQeJFseJoNYzFyo7vZjYKwDtQVJ6pomksWExsTb, address:145.79.227.45:4233): protocol: rpc: dial tcp 145.79.227.45:4233: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:39:12.435547490 (195h28m15.4512171s) [noise=true , address=145.79.227.44:4198 , piece_id=OC2ZBXSU5H4MA2T3GUR4YZLPM7IJTHMCGESIW5OAXBWYQF5ABYCA, seg_pos={0 0}, upload=187758]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: rpc: dial tcp 145.79.227.44:4198: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:39:12.435634826 (195h28m15.451304438s) [noise=true , address=145.79.227.42:4124 , piece_id=NUOMBSSUW3QUAHLMNAPQGX3TWFM32IS6CWRYSP5OIRMBSVJGJHJA, seg_pos={0 0}, upload=187758]: Done uploading piece. err: ecclient: upload failed (node:12kz79BpzkFLTxbgi2Tjvz4dStT7MtrQ4GmX6bFnYw8v45jBaSy, address:145.79.227.42:4124): protocol: rpc: dial tcp 145.79.227.42:4124: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:39:12.435876651 (195h28m15.451546255s) [noise=true , address=145.79.227.49:4341 , piece_id=VHG76O2FCPT6JTZOQ6CS2Y6WRMC3V6KI2BC3HW7CYTLIGZVJWS3Q, seg_pos={0 0}, upload=187758]: Done uploading piece. err: ecclient: upload failed (node:1okrDyM4TfzYN3p8zUjTqbw2661rKQQqZdP1ptAKzzTQ82C7cN, address:145.79.227.49:4341): protocol: rpc: dial tcp 145.79.227.49:4341: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:39:12.435905753 (195h28m15.451575363s) [noise=true , address=145.79.227.3:4152 , piece_id=TY3RIGGAURPFSQ6BNO7IYHAQ3HQ6HAUCYI2F5GII22CSQI5OPH5A, seg_pos={0 0}, upload=187758]: Done uploading piece. err: ecclient: upload failed (node:122rsZGFcSywYKfw4DPU7duA9TGtj1iXmxS4ewGGGSXGRf7Q7mX, address:145.79.227.3:4152): protocol: rpc: dial tcp 145.79.227.3:4152: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:39:12.436093378 (195h28m15.451762984s) [noise=true , address=145.79.227.8:4333 , piece_id=HSZRKVQPRJHQSWWNPQCQ2OU2UCF4VYPUPCWZ5XCLMOWCAGYRJZGA, seg_pos={0 0}, upload=187758]: Done uploading piece. err: ecclient: upload failed (node:1xRdTZoEk3NsnS445GcdEvwAdDzNLwUYSfgjB7bBSGRx9L9mrU, address:145.79.227.8:4333): protocol: rpc: dial tcp 145.79.227.8:4333: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:39:12.436191150 (195h28m15.451860759s) [noise=true , address=145.79.227.2:4115 , piece_id=FFRVHW2JD5KDYDD4CBLYVAO7MVP7KAKQBGVQEIWZ2SE3AYHV3TKA, seg_pos={0 0}, upload=187758]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: rpc: dial tcp 145.79.227.2:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:39:12.436468272 (195h28m15.45213788s) [noise=true , address=145.79.227.50:4377 , piece_id=LJ2YNNF7ZSG3E5M47HCCEGLD2TQX5HKBLDESO2KAATERTELEOMOA, seg_pos={0 0}, upload=187758]: Done uploading piece. err: ecclient: upload failed (node:126HKpC7sciWFTxNaH5Hjs6BBuh8NRgu8fA4cu4qcK29zfJWg7R, address:145.79.227.50:4377): protocol: rpc: dial tcp 145.79.227.50:4377: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Mar 18 19:39:12.436495192 (195h28m15.452164783s) [seg_pos={0 0}, upload=187758]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Mar 18 19:39:12.549898766 (195h28m15.565568373s) [upload=187758]: Done uploading segments... Mar 18 19:39:12.549963035 (195h28m15.565632636s) [upload=187758]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 30.116915196s core uplink us-select-1

1
...
575
576
577
578
579
...
7090