Orbiter History

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 46 out of 60 is failed (threshold=3) [..............XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 177.362907ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 16:23:11 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 177.411721ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 42 out of 60 is failed (threshold=3) [..................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 178.249777ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 16:22:57 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 178.275782ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 45 out of 60 is failed (threshold=3) [...............XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 166.750153ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 16:22:35 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 166.781089ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 41 out of 60 is failed (threshold=3) [...................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 179.350729ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 16:22:16 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 179.38064ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 44 out of 60 is failed (threshold=3) [................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 167.640606ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 16:21:56 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 167.690016ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 40 out of 60 is failed (threshold=3) [....................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 172.665303ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 16:21:19 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 172.690445ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 43 out of 60 is failed (threshold=3) [.................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 170.904678ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 16:21:04 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 170.932888ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 39 out of 60 is failed (threshold=3) [.....................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 178.390675ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 16:20:59 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 178.416121ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 42 out of 60 is failed (threshold=3) [..................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 179.739774ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 16:20:41 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 179.764659ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 38 out of 60 is failed (threshold=3) [......................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 175.442621ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 16:20:11 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 175.465664ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 41 out of 60 is failed (threshold=3) [...................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 175.171307ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 16:19:45 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 175.193801ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.414985205s FAIL (2025-07-18 16:19:27 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== BUMKAHO3ZMVDNWVSRRI3GTLM4MO3MP6MNNGUUZKBLBWQ, seg_pos={0 0}, upload=484783]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: rpc: dial tcp 77.90.31.13:4628: operation was canceled; context canceled Jul 18 15:31:52.267100372 (606h20m52.150513985s) [seg_pos={0 0}, upload=484783]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:31:52.267178520 (606h20m52.150592177s) [upload=484783]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:31:52.267184778 (606h20m52.150598408s) [upload=484783]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:31:52.565015623 (606h20m52.448429239s) [upload=484783]: Done uploading segments... Jul 18 15:31:52.565033307 (606h20m52.448446919s) [upload=484783]: Upload finished. err: <nil> Jul 18 15:33:00.528847457 (606h22m0.412261094s) [upload=484797]: Starting upload Jul 18 15:33:00.528934444 (606h22m0.412348056s) [upload=484797]: Done starting upload Jul 18 15:33:00.529017539 (606h22m0.412431168s) [upload=484797]: Uploading segments... Jul 18 15:33:00.529303798 (606h22m0.412717427s) [upload=484797]: Got next segment. Inline: false Jul 18 15:33:00.529348408 (606h22m0.41276202s) [upload=484797]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:33:00.529353973 (606h22m0.412767599s) [upload=484797]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:33:00.640500227 (606h22m0.523913861s) [seg_pos={0 0}, upload=484797]: Begin upload segment... Jul 18 15:33:00.640588551 (606h22m0.524002189s) [seg_pos={0 0}, upload=484797]: Done begin upload segment. Jul 18 15:33:00.640649191 (606h22m0.524062829s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.640953078 (606h22m0.524366715s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641531127 (606h22m0.524944758s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641980147 (606h22m0.52539378s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642678070 (606h22m0.526091699s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642961859 (606h22m0.526375491s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643490795 (606h22m0.526904425s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643733424 (606h22m0.52714705s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644374317 (606h22m0.527787948s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644585121 (606h22m0.527998748s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644900409 (606h22m0.528314035s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646038897 (606h22m0.529452526s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646375740 (606h22m0.529789369s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.648944423 (606h22m0.53235805s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.650463350 (606h22m0.533876978s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.651369647 (606h22m0.534783267s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:01.262589342 (606h22m1.146002974s) [upload=484797]: Next returned nil segment Jul 18 15:33:01.262648256 (606h22m1.146061891s) [upload=484797]: Waiting for error group managing segments... Jul 18 15:33:01.398379699 (606h22m1.281793326s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404946640 (606h22m1.288360283s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404992987 (606h22m1.28840661s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.405551748 (606h22m1.288965369s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.407987418 (606h22m1.291401032s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.410553731 (606h22m1.293967362s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.416771944 (606h22m1.300185579s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.425807204 (606h22m1.309220819s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.430635732 (606h22m1.314049361s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.433605056 (606h22m1.317018687s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.440793839 (606h22m1.32420747s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.446368593 (606h22m1.329782225s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:30.258196584 (606h22m30.141610202s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258423161 (606h22m30.141836789s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:40968->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258491071 (606h22m30.1419047s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258527956 (606h22m30.141941579s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258553948 (606h22m30.141967578s) [seg_pos={0 0}, upload=484797]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 15:33:30.369475355 (606h22m30.252888992s) [upload=484797]: Done uploading segments... Jul 18 15:33:30.369509072 (606h22m30.252922716s) [upload=484797]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.415012932s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 37 out of 60 is failed (threshold=3) [.......................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 172.831627ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 16:19:16 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 172.856101ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 2.301200371s FAIL (2025-07-18 16:18:56 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== BUMKAHO3ZMVDNWVSRRI3GTLM4MO3MP6MNNGUUZKBLBWQ, seg_pos={0 0}, upload=484783]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: rpc: dial tcp 77.90.31.13:4628: operation was canceled; context canceled Jul 18 15:31:52.267100372 (606h20m52.150513985s) [seg_pos={0 0}, upload=484783]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:31:52.267178520 (606h20m52.150592177s) [upload=484783]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:31:52.267184778 (606h20m52.150598408s) [upload=484783]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:31:52.565015623 (606h20m52.448429239s) [upload=484783]: Done uploading segments... Jul 18 15:31:52.565033307 (606h20m52.448446919s) [upload=484783]: Upload finished. err: <nil> Jul 18 15:33:00.528847457 (606h22m0.412261094s) [upload=484797]: Starting upload Jul 18 15:33:00.528934444 (606h22m0.412348056s) [upload=484797]: Done starting upload Jul 18 15:33:00.529017539 (606h22m0.412431168s) [upload=484797]: Uploading segments... Jul 18 15:33:00.529303798 (606h22m0.412717427s) [upload=484797]: Got next segment. Inline: false Jul 18 15:33:00.529348408 (606h22m0.41276202s) [upload=484797]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:33:00.529353973 (606h22m0.412767599s) [upload=484797]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:33:00.640500227 (606h22m0.523913861s) [seg_pos={0 0}, upload=484797]: Begin upload segment... Jul 18 15:33:00.640588551 (606h22m0.524002189s) [seg_pos={0 0}, upload=484797]: Done begin upload segment. Jul 18 15:33:00.640649191 (606h22m0.524062829s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.640953078 (606h22m0.524366715s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641531127 (606h22m0.524944758s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641980147 (606h22m0.52539378s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642678070 (606h22m0.526091699s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642961859 (606h22m0.526375491s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643490795 (606h22m0.526904425s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643733424 (606h22m0.52714705s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644374317 (606h22m0.527787948s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644585121 (606h22m0.527998748s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644900409 (606h22m0.528314035s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646038897 (606h22m0.529452526s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646375740 (606h22m0.529789369s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.648944423 (606h22m0.53235805s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.650463350 (606h22m0.533876978s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.651369647 (606h22m0.534783267s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:01.262589342 (606h22m1.146002974s) [upload=484797]: Next returned nil segment Jul 18 15:33:01.262648256 (606h22m1.146061891s) [upload=484797]: Waiting for error group managing segments... Jul 18 15:33:01.398379699 (606h22m1.281793326s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404946640 (606h22m1.288360283s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404992987 (606h22m1.28840661s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.405551748 (606h22m1.288965369s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.407987418 (606h22m1.291401032s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.410553731 (606h22m1.293967362s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.416771944 (606h22m1.300185579s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.425807204 (606h22m1.309220819s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.430635732 (606h22m1.314049361s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.433605056 (606h22m1.317018687s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.440793839 (606h22m1.32420747s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.446368593 (606h22m1.329782225s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:30.258196584 (606h22m30.141610202s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258423161 (606h22m30.141836789s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:40968->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258491071 (606h22m30.1419047s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258527956 (606h22m30.141941579s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258553948 (606h22m30.141967578s) [seg_pos={0 0}, upload=484797]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 15:33:30.369475355 (606h22m30.252888992s) [upload=484797]: Done uploading segments... Jul 18 15:33:30.369509072 (606h22m30.252922716s) [upload=484797]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.301229768s core uplink us-select-1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 40 out of 60 is failed (threshold=3) [....................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 177.895025ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 16:18:47 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 177.920318ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 36 out of 60 is failed (threshold=3) [........................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 205.355713ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 16:18:22 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 205.452415ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 2.718475719s FAIL (2025-07-18 16:18:18 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== BUMKAHO3ZMVDNWVSRRI3GTLM4MO3MP6MNNGUUZKBLBWQ, seg_pos={0 0}, upload=484783]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: rpc: dial tcp 77.90.31.13:4628: operation was canceled; context canceled Jul 18 15:31:52.267100372 (606h20m52.150513985s) [seg_pos={0 0}, upload=484783]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:31:52.267178520 (606h20m52.150592177s) [upload=484783]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:31:52.267184778 (606h20m52.150598408s) [upload=484783]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:31:52.565015623 (606h20m52.448429239s) [upload=484783]: Done uploading segments... Jul 18 15:31:52.565033307 (606h20m52.448446919s) [upload=484783]: Upload finished. err: <nil> Jul 18 15:33:00.528847457 (606h22m0.412261094s) [upload=484797]: Starting upload Jul 18 15:33:00.528934444 (606h22m0.412348056s) [upload=484797]: Done starting upload Jul 18 15:33:00.529017539 (606h22m0.412431168s) [upload=484797]: Uploading segments... Jul 18 15:33:00.529303798 (606h22m0.412717427s) [upload=484797]: Got next segment. Inline: false Jul 18 15:33:00.529348408 (606h22m0.41276202s) [upload=484797]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:33:00.529353973 (606h22m0.412767599s) [upload=484797]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:33:00.640500227 (606h22m0.523913861s) [seg_pos={0 0}, upload=484797]: Begin upload segment... Jul 18 15:33:00.640588551 (606h22m0.524002189s) [seg_pos={0 0}, upload=484797]: Done begin upload segment. Jul 18 15:33:00.640649191 (606h22m0.524062829s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.640953078 (606h22m0.524366715s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641531127 (606h22m0.524944758s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641980147 (606h22m0.52539378s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642678070 (606h22m0.526091699s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642961859 (606h22m0.526375491s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643490795 (606h22m0.526904425s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643733424 (606h22m0.52714705s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644374317 (606h22m0.527787948s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644585121 (606h22m0.527998748s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644900409 (606h22m0.528314035s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646038897 (606h22m0.529452526s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646375740 (606h22m0.529789369s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.648944423 (606h22m0.53235805s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.650463350 (606h22m0.533876978s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.651369647 (606h22m0.534783267s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:01.262589342 (606h22m1.146002974s) [upload=484797]: Next returned nil segment Jul 18 15:33:01.262648256 (606h22m1.146061891s) [upload=484797]: Waiting for error group managing segments... Jul 18 15:33:01.398379699 (606h22m1.281793326s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404946640 (606h22m1.288360283s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404992987 (606h22m1.28840661s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.405551748 (606h22m1.288965369s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.407987418 (606h22m1.291401032s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.410553731 (606h22m1.293967362s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.416771944 (606h22m1.300185579s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.425807204 (606h22m1.309220819s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.430635732 (606h22m1.314049361s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.433605056 (606h22m1.317018687s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.440793839 (606h22m1.32420747s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.446368593 (606h22m1.329782225s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:30.258196584 (606h22m30.141610202s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258423161 (606h22m30.141836789s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:40968->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258491071 (606h22m30.1419047s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258527956 (606h22m30.141941579s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258553948 (606h22m30.141967578s) [seg_pos={0 0}, upload=484797]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 15:33:30.369475355 (606h22m30.252888992s) [upload=484797]: Done uploading segments... Jul 18 15:33:30.369509072 (606h22m30.252922716s) [upload=484797]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.718502934s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 35 out of 60 is failed (threshold=3) [.........................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 176.971064ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 16:17:43 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 177.000741ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 39 out of 60 is failed (threshold=3) [.....................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 177.29919ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 16:17:37 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 177.319952ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [X..X........................................................] duration: 2.357431635s FAIL (2025-07-18 16:17:09 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== BUMKAHO3ZMVDNWVSRRI3GTLM4MO3MP6MNNGUUZKBLBWQ, seg_pos={0 0}, upload=484783]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: rpc: dial tcp 77.90.31.13:4628: operation was canceled; context canceled Jul 18 15:31:52.267100372 (606h20m52.150513985s) [seg_pos={0 0}, upload=484783]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:31:52.267178520 (606h20m52.150592177s) [upload=484783]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:31:52.267184778 (606h20m52.150598408s) [upload=484783]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:31:52.565015623 (606h20m52.448429239s) [upload=484783]: Done uploading segments... Jul 18 15:31:52.565033307 (606h20m52.448446919s) [upload=484783]: Upload finished. err: <nil> Jul 18 15:33:00.528847457 (606h22m0.412261094s) [upload=484797]: Starting upload Jul 18 15:33:00.528934444 (606h22m0.412348056s) [upload=484797]: Done starting upload Jul 18 15:33:00.529017539 (606h22m0.412431168s) [upload=484797]: Uploading segments... Jul 18 15:33:00.529303798 (606h22m0.412717427s) [upload=484797]: Got next segment. Inline: false Jul 18 15:33:00.529348408 (606h22m0.41276202s) [upload=484797]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:33:00.529353973 (606h22m0.412767599s) [upload=484797]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:33:00.640500227 (606h22m0.523913861s) [seg_pos={0 0}, upload=484797]: Begin upload segment... Jul 18 15:33:00.640588551 (606h22m0.524002189s) [seg_pos={0 0}, upload=484797]: Done begin upload segment. Jul 18 15:33:00.640649191 (606h22m0.524062829s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.640953078 (606h22m0.524366715s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641531127 (606h22m0.524944758s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641980147 (606h22m0.52539378s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642678070 (606h22m0.526091699s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642961859 (606h22m0.526375491s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643490795 (606h22m0.526904425s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643733424 (606h22m0.52714705s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644374317 (606h22m0.527787948s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644585121 (606h22m0.527998748s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644900409 (606h22m0.528314035s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646038897 (606h22m0.529452526s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646375740 (606h22m0.529789369s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.648944423 (606h22m0.53235805s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.650463350 (606h22m0.533876978s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.651369647 (606h22m0.534783267s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:01.262589342 (606h22m1.146002974s) [upload=484797]: Next returned nil segment Jul 18 15:33:01.262648256 (606h22m1.146061891s) [upload=484797]: Waiting for error group managing segments... Jul 18 15:33:01.398379699 (606h22m1.281793326s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404946640 (606h22m1.288360283s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404992987 (606h22m1.28840661s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.405551748 (606h22m1.288965369s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.407987418 (606h22m1.291401032s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.410553731 (606h22m1.293967362s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.416771944 (606h22m1.300185579s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.425807204 (606h22m1.309220819s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.430635732 (606h22m1.314049361s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.433605056 (606h22m1.317018687s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.440793839 (606h22m1.32420747s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.446368593 (606h22m1.329782225s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:30.258196584 (606h22m30.141610202s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258423161 (606h22m30.141836789s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:40968->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258491071 (606h22m30.1419047s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258527956 (606h22m30.141941579s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258553948 (606h22m30.141967578s) [seg_pos={0 0}, upload=484797]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 15:33:30.369475355 (606h22m30.252888992s) [upload=484797]: Done uploading segments... Jul 18 15:33:30.369509072 (606h22m30.252922716s) [upload=484797]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.357460545s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 34 out of 60 is failed (threshold=3) [..........................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 177.996739ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 16:16:44 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 178.040643ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 38 out of 60 is failed (threshold=3) [......................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 172.87626ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 16:16:39 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 172.901789ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [XX..X.......................................................] duration: 2.377585958s FAIL (2025-07-18 16:16:27 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== BUMKAHO3ZMVDNWVSRRI3GTLM4MO3MP6MNNGUUZKBLBWQ, seg_pos={0 0}, upload=484783]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: rpc: dial tcp 77.90.31.13:4628: operation was canceled; context canceled Jul 18 15:31:52.267100372 (606h20m52.150513985s) [seg_pos={0 0}, upload=484783]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:31:52.267178520 (606h20m52.150592177s) [upload=484783]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:31:52.267184778 (606h20m52.150598408s) [upload=484783]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:31:52.565015623 (606h20m52.448429239s) [upload=484783]: Done uploading segments... Jul 18 15:31:52.565033307 (606h20m52.448446919s) [upload=484783]: Upload finished. err: <nil> Jul 18 15:33:00.528847457 (606h22m0.412261094s) [upload=484797]: Starting upload Jul 18 15:33:00.528934444 (606h22m0.412348056s) [upload=484797]: Done starting upload Jul 18 15:33:00.529017539 (606h22m0.412431168s) [upload=484797]: Uploading segments... Jul 18 15:33:00.529303798 (606h22m0.412717427s) [upload=484797]: Got next segment. Inline: false Jul 18 15:33:00.529348408 (606h22m0.41276202s) [upload=484797]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:33:00.529353973 (606h22m0.412767599s) [upload=484797]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:33:00.640500227 (606h22m0.523913861s) [seg_pos={0 0}, upload=484797]: Begin upload segment... Jul 18 15:33:00.640588551 (606h22m0.524002189s) [seg_pos={0 0}, upload=484797]: Done begin upload segment. Jul 18 15:33:00.640649191 (606h22m0.524062829s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.640953078 (606h22m0.524366715s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641531127 (606h22m0.524944758s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641980147 (606h22m0.52539378s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642678070 (606h22m0.526091699s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642961859 (606h22m0.526375491s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643490795 (606h22m0.526904425s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643733424 (606h22m0.52714705s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644374317 (606h22m0.527787948s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644585121 (606h22m0.527998748s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644900409 (606h22m0.528314035s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646038897 (606h22m0.529452526s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646375740 (606h22m0.529789369s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.648944423 (606h22m0.53235805s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.650463350 (606h22m0.533876978s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.651369647 (606h22m0.534783267s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:01.262589342 (606h22m1.146002974s) [upload=484797]: Next returned nil segment Jul 18 15:33:01.262648256 (606h22m1.146061891s) [upload=484797]: Waiting for error group managing segments... Jul 18 15:33:01.398379699 (606h22m1.281793326s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404946640 (606h22m1.288360283s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404992987 (606h22m1.28840661s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.405551748 (606h22m1.288965369s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.407987418 (606h22m1.291401032s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.410553731 (606h22m1.293967362s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.416771944 (606h22m1.300185579s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.425807204 (606h22m1.309220819s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.430635732 (606h22m1.314049361s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.433605056 (606h22m1.317018687s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.440793839 (606h22m1.32420747s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.446368593 (606h22m1.329782225s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:30.258196584 (606h22m30.141610202s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258423161 (606h22m30.141836789s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:40968->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258491071 (606h22m30.1419047s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258527956 (606h22m30.141941579s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258553948 (606h22m30.141967578s) [seg_pos={0 0}, upload=484797]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 15:33:30.369475355 (606h22m30.252888992s) [upload=484797]: Done uploading segments... Jul 18 15:33:30.369509072 (606h22m30.252922716s) [upload=484797]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.377622605s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 33 out of 60 is failed (threshold=3) [...........................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 172.740024ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 16:16:06 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 172.772538ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 37 out of 60 is failed (threshold=3) [.......................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 175.919196ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 16:15:59 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 176.003704ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.XX..X......................................................] duration: 2.380005589s FAIL (2025-07-18 16:15:15 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== BUMKAHO3ZMVDNWVSRRI3GTLM4MO3MP6MNNGUUZKBLBWQ, seg_pos={0 0}, upload=484783]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: rpc: dial tcp 77.90.31.13:4628: operation was canceled; context canceled Jul 18 15:31:52.267100372 (606h20m52.150513985s) [seg_pos={0 0}, upload=484783]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:31:52.267178520 (606h20m52.150592177s) [upload=484783]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:31:52.267184778 (606h20m52.150598408s) [upload=484783]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:31:52.565015623 (606h20m52.448429239s) [upload=484783]: Done uploading segments... Jul 18 15:31:52.565033307 (606h20m52.448446919s) [upload=484783]: Upload finished. err: <nil> Jul 18 15:33:00.528847457 (606h22m0.412261094s) [upload=484797]: Starting upload Jul 18 15:33:00.528934444 (606h22m0.412348056s) [upload=484797]: Done starting upload Jul 18 15:33:00.529017539 (606h22m0.412431168s) [upload=484797]: Uploading segments... Jul 18 15:33:00.529303798 (606h22m0.412717427s) [upload=484797]: Got next segment. Inline: false Jul 18 15:33:00.529348408 (606h22m0.41276202s) [upload=484797]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:33:00.529353973 (606h22m0.412767599s) [upload=484797]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:33:00.640500227 (606h22m0.523913861s) [seg_pos={0 0}, upload=484797]: Begin upload segment... Jul 18 15:33:00.640588551 (606h22m0.524002189s) [seg_pos={0 0}, upload=484797]: Done begin upload segment. Jul 18 15:33:00.640649191 (606h22m0.524062829s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.640953078 (606h22m0.524366715s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641531127 (606h22m0.524944758s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641980147 (606h22m0.52539378s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642678070 (606h22m0.526091699s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642961859 (606h22m0.526375491s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643490795 (606h22m0.526904425s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643733424 (606h22m0.52714705s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644374317 (606h22m0.527787948s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644585121 (606h22m0.527998748s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644900409 (606h22m0.528314035s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646038897 (606h22m0.529452526s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646375740 (606h22m0.529789369s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.648944423 (606h22m0.53235805s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.650463350 (606h22m0.533876978s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.651369647 (606h22m0.534783267s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:01.262589342 (606h22m1.146002974s) [upload=484797]: Next returned nil segment Jul 18 15:33:01.262648256 (606h22m1.146061891s) [upload=484797]: Waiting for error group managing segments... Jul 18 15:33:01.398379699 (606h22m1.281793326s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404946640 (606h22m1.288360283s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404992987 (606h22m1.28840661s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.405551748 (606h22m1.288965369s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.407987418 (606h22m1.291401032s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.410553731 (606h22m1.293967362s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.416771944 (606h22m1.300185579s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.425807204 (606h22m1.309220819s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.430635732 (606h22m1.314049361s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.433605056 (606h22m1.317018687s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.440793839 (606h22m1.32420747s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.446368593 (606h22m1.329782225s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:30.258196584 (606h22m30.141610202s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258423161 (606h22m30.141836789s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:40968->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258491071 (606h22m30.1419047s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258527956 (606h22m30.141941579s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258553948 (606h22m30.141967578s) [seg_pos={0 0}, upload=484797]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 15:33:30.369475355 (606h22m30.252888992s) [upload=484797]: Done uploading segments... Jul 18 15:33:30.369509072 (606h22m30.252922716s) [upload=484797]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.380038716s core uplink us-select-1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 36 out of 60 is failed (threshold=3) [........................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 170.896979ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 16:15:04 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 171.08044ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 32 out of 60 is failed (threshold=3) [............................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 179.457196ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 16:14:54 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 179.483616ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 35 out of 60 is failed (threshold=3) [.........................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 177.574537ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 16:14:42 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 177.60098ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..XX..X.....................................................] duration: 2.467698171s FAIL (2025-07-18 16:14:34 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== BUMKAHO3ZMVDNWVSRRI3GTLM4MO3MP6MNNGUUZKBLBWQ, seg_pos={0 0}, upload=484783]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: rpc: dial tcp 77.90.31.13:4628: operation was canceled; context canceled Jul 18 15:31:52.267100372 (606h20m52.150513985s) [seg_pos={0 0}, upload=484783]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:31:52.267178520 (606h20m52.150592177s) [upload=484783]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:31:52.267184778 (606h20m52.150598408s) [upload=484783]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:31:52.565015623 (606h20m52.448429239s) [upload=484783]: Done uploading segments... Jul 18 15:31:52.565033307 (606h20m52.448446919s) [upload=484783]: Upload finished. err: <nil> Jul 18 15:33:00.528847457 (606h22m0.412261094s) [upload=484797]: Starting upload Jul 18 15:33:00.528934444 (606h22m0.412348056s) [upload=484797]: Done starting upload Jul 18 15:33:00.529017539 (606h22m0.412431168s) [upload=484797]: Uploading segments... Jul 18 15:33:00.529303798 (606h22m0.412717427s) [upload=484797]: Got next segment. Inline: false Jul 18 15:33:00.529348408 (606h22m0.41276202s) [upload=484797]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:33:00.529353973 (606h22m0.412767599s) [upload=484797]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:33:00.640500227 (606h22m0.523913861s) [seg_pos={0 0}, upload=484797]: Begin upload segment... Jul 18 15:33:00.640588551 (606h22m0.524002189s) [seg_pos={0 0}, upload=484797]: Done begin upload segment. Jul 18 15:33:00.640649191 (606h22m0.524062829s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.640953078 (606h22m0.524366715s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641531127 (606h22m0.524944758s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641980147 (606h22m0.52539378s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642678070 (606h22m0.526091699s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642961859 (606h22m0.526375491s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643490795 (606h22m0.526904425s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643733424 (606h22m0.52714705s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644374317 (606h22m0.527787948s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644585121 (606h22m0.527998748s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644900409 (606h22m0.528314035s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646038897 (606h22m0.529452526s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646375740 (606h22m0.529789369s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.648944423 (606h22m0.53235805s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.650463350 (606h22m0.533876978s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.651369647 (606h22m0.534783267s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:01.262589342 (606h22m1.146002974s) [upload=484797]: Next returned nil segment Jul 18 15:33:01.262648256 (606h22m1.146061891s) [upload=484797]: Waiting for error group managing segments... Jul 18 15:33:01.398379699 (606h22m1.281793326s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404946640 (606h22m1.288360283s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404992987 (606h22m1.28840661s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.405551748 (606h22m1.288965369s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.407987418 (606h22m1.291401032s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.410553731 (606h22m1.293967362s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.416771944 (606h22m1.300185579s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.425807204 (606h22m1.309220819s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.430635732 (606h22m1.314049361s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.433605056 (606h22m1.317018687s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.440793839 (606h22m1.32420747s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.446368593 (606h22m1.329782225s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:30.258196584 (606h22m30.141610202s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258423161 (606h22m30.141836789s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:40968->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258491071 (606h22m30.1419047s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258527956 (606h22m30.141941579s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258553948 (606h22m30.141967578s) [seg_pos={0 0}, upload=484797]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 15:33:30.369475355 (606h22m30.252888992s) [upload=484797]: Done uploading segments... Jul 18 15:33:30.369509072 (606h22m30.252922716s) [upload=484797]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.467730577s core uplink us-select-1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 34 out of 60 is failed (threshold=3) [..........................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 174.5426ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 16:14:26 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 174.562766ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...XX..X....................................................] duration: 2.436694993s FAIL (2025-07-18 16:14:02 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== BUMKAHO3ZMVDNWVSRRI3GTLM4MO3MP6MNNGUUZKBLBWQ, seg_pos={0 0}, upload=484783]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: rpc: dial tcp 77.90.31.13:4628: operation was canceled; context canceled Jul 18 15:31:52.267100372 (606h20m52.150513985s) [seg_pos={0 0}, upload=484783]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:31:52.267178520 (606h20m52.150592177s) [upload=484783]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:31:52.267184778 (606h20m52.150598408s) [upload=484783]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:31:52.565015623 (606h20m52.448429239s) [upload=484783]: Done uploading segments... Jul 18 15:31:52.565033307 (606h20m52.448446919s) [upload=484783]: Upload finished. err: <nil> Jul 18 15:33:00.528847457 (606h22m0.412261094s) [upload=484797]: Starting upload Jul 18 15:33:00.528934444 (606h22m0.412348056s) [upload=484797]: Done starting upload Jul 18 15:33:00.529017539 (606h22m0.412431168s) [upload=484797]: Uploading segments... Jul 18 15:33:00.529303798 (606h22m0.412717427s) [upload=484797]: Got next segment. Inline: false Jul 18 15:33:00.529348408 (606h22m0.41276202s) [upload=484797]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:33:00.529353973 (606h22m0.412767599s) [upload=484797]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:33:00.640500227 (606h22m0.523913861s) [seg_pos={0 0}, upload=484797]: Begin upload segment... Jul 18 15:33:00.640588551 (606h22m0.524002189s) [seg_pos={0 0}, upload=484797]: Done begin upload segment. Jul 18 15:33:00.640649191 (606h22m0.524062829s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.640953078 (606h22m0.524366715s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641531127 (606h22m0.524944758s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641980147 (606h22m0.52539378s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642678070 (606h22m0.526091699s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642961859 (606h22m0.526375491s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643490795 (606h22m0.526904425s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643733424 (606h22m0.52714705s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644374317 (606h22m0.527787948s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644585121 (606h22m0.527998748s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644900409 (606h22m0.528314035s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646038897 (606h22m0.529452526s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646375740 (606h22m0.529789369s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.648944423 (606h22m0.53235805s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.650463350 (606h22m0.533876978s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.651369647 (606h22m0.534783267s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:01.262589342 (606h22m1.146002974s) [upload=484797]: Next returned nil segment Jul 18 15:33:01.262648256 (606h22m1.146061891s) [upload=484797]: Waiting for error group managing segments... Jul 18 15:33:01.398379699 (606h22m1.281793326s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404946640 (606h22m1.288360283s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404992987 (606h22m1.28840661s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.405551748 (606h22m1.288965369s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.407987418 (606h22m1.291401032s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.410553731 (606h22m1.293967362s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.416771944 (606h22m1.300185579s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.425807204 (606h22m1.309220819s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.430635732 (606h22m1.314049361s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.433605056 (606h22m1.317018687s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.440793839 (606h22m1.32420747s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.446368593 (606h22m1.329782225s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:30.258196584 (606h22m30.141610202s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258423161 (606h22m30.141836789s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:40968->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258491071 (606h22m30.1419047s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258527956 (606h22m30.141941579s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258553948 (606h22m30.141967578s) [seg_pos={0 0}, upload=484797]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 15:33:30.369475355 (606h22m30.252888992s) [upload=484797]: Done uploading segments... Jul 18 15:33:30.369509072 (606h22m30.252922716s) [upload=484797]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.436772198s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 31 out of 60 is failed (threshold=3) [.............................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 170.933611ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 16:13:51 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 170.9548ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 33 out of 60 is failed (threshold=3) [...........................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 221.379483ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 16:13:20 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 221.421956ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 30 out of 60 is failed (threshold=3) [..............................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 171.619625ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 16:13:08 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 171.714083ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....XX..X...................................................] duration: 2.392101041s FAIL (2025-07-18 16:12:56 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== BUMKAHO3ZMVDNWVSRRI3GTLM4MO3MP6MNNGUUZKBLBWQ, seg_pos={0 0}, upload=484783]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: rpc: dial tcp 77.90.31.13:4628: operation was canceled; context canceled Jul 18 15:31:52.267100372 (606h20m52.150513985s) [seg_pos={0 0}, upload=484783]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:31:52.267178520 (606h20m52.150592177s) [upload=484783]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:31:52.267184778 (606h20m52.150598408s) [upload=484783]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:31:52.565015623 (606h20m52.448429239s) [upload=484783]: Done uploading segments... Jul 18 15:31:52.565033307 (606h20m52.448446919s) [upload=484783]: Upload finished. err: <nil> Jul 18 15:33:00.528847457 (606h22m0.412261094s) [upload=484797]: Starting upload Jul 18 15:33:00.528934444 (606h22m0.412348056s) [upload=484797]: Done starting upload Jul 18 15:33:00.529017539 (606h22m0.412431168s) [upload=484797]: Uploading segments... Jul 18 15:33:00.529303798 (606h22m0.412717427s) [upload=484797]: Got next segment. Inline: false Jul 18 15:33:00.529348408 (606h22m0.41276202s) [upload=484797]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:33:00.529353973 (606h22m0.412767599s) [upload=484797]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:33:00.640500227 (606h22m0.523913861s) [seg_pos={0 0}, upload=484797]: Begin upload segment... Jul 18 15:33:00.640588551 (606h22m0.524002189s) [seg_pos={0 0}, upload=484797]: Done begin upload segment. Jul 18 15:33:00.640649191 (606h22m0.524062829s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.640953078 (606h22m0.524366715s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641531127 (606h22m0.524944758s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641980147 (606h22m0.52539378s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642678070 (606h22m0.526091699s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642961859 (606h22m0.526375491s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643490795 (606h22m0.526904425s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643733424 (606h22m0.52714705s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644374317 (606h22m0.527787948s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644585121 (606h22m0.527998748s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644900409 (606h22m0.528314035s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646038897 (606h22m0.529452526s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646375740 (606h22m0.529789369s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.648944423 (606h22m0.53235805s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.650463350 (606h22m0.533876978s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.651369647 (606h22m0.534783267s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:01.262589342 (606h22m1.146002974s) [upload=484797]: Next returned nil segment Jul 18 15:33:01.262648256 (606h22m1.146061891s) [upload=484797]: Waiting for error group managing segments... Jul 18 15:33:01.398379699 (606h22m1.281793326s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404946640 (606h22m1.288360283s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404992987 (606h22m1.28840661s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.405551748 (606h22m1.288965369s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.407987418 (606h22m1.291401032s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.410553731 (606h22m1.293967362s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.416771944 (606h22m1.300185579s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.425807204 (606h22m1.309220819s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.430635732 (606h22m1.314049361s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.433605056 (606h22m1.317018687s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.440793839 (606h22m1.32420747s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.446368593 (606h22m1.329782225s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:30.258196584 (606h22m30.141610202s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258423161 (606h22m30.141836789s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:40968->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258491071 (606h22m30.1419047s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258527956 (606h22m30.141941579s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258553948 (606h22m30.141967578s) [seg_pos={0 0}, upload=484797]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 15:33:30.369475355 (606h22m30.252888992s) [upload=484797]: Done uploading segments... Jul 18 15:33:30.369509072 (606h22m30.252922716s) [upload=484797]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.392137235s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 29 out of 60 is failed (threshold=3) [...............................XXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 179.502837ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 16:12:21 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 179.530629ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 32 out of 60 is failed (threshold=3) [............................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 172.990588ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 16:12:07 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 173.094945ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 31 out of 60 is failed (threshold=3) [.............................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 174.148382ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 16:11:51 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 174.183339ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....XX..X..................................................] duration: 2.545963676s FAIL (2025-07-18 16:11:43 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== BUMKAHO3ZMVDNWVSRRI3GTLM4MO3MP6MNNGUUZKBLBWQ, seg_pos={0 0}, upload=484783]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: rpc: dial tcp 77.90.31.13:4628: operation was canceled; context canceled Jul 18 15:31:52.267100372 (606h20m52.150513985s) [seg_pos={0 0}, upload=484783]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:31:52.267178520 (606h20m52.150592177s) [upload=484783]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:31:52.267184778 (606h20m52.150598408s) [upload=484783]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:31:52.565015623 (606h20m52.448429239s) [upload=484783]: Done uploading segments... Jul 18 15:31:52.565033307 (606h20m52.448446919s) [upload=484783]: Upload finished. err: <nil> Jul 18 15:33:00.528847457 (606h22m0.412261094s) [upload=484797]: Starting upload Jul 18 15:33:00.528934444 (606h22m0.412348056s) [upload=484797]: Done starting upload Jul 18 15:33:00.529017539 (606h22m0.412431168s) [upload=484797]: Uploading segments... Jul 18 15:33:00.529303798 (606h22m0.412717427s) [upload=484797]: Got next segment. Inline: false Jul 18 15:33:00.529348408 (606h22m0.41276202s) [upload=484797]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:33:00.529353973 (606h22m0.412767599s) [upload=484797]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:33:00.640500227 (606h22m0.523913861s) [seg_pos={0 0}, upload=484797]: Begin upload segment... Jul 18 15:33:00.640588551 (606h22m0.524002189s) [seg_pos={0 0}, upload=484797]: Done begin upload segment. Jul 18 15:33:00.640649191 (606h22m0.524062829s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.640953078 (606h22m0.524366715s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641531127 (606h22m0.524944758s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641980147 (606h22m0.52539378s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642678070 (606h22m0.526091699s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642961859 (606h22m0.526375491s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643490795 (606h22m0.526904425s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643733424 (606h22m0.52714705s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644374317 (606h22m0.527787948s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644585121 (606h22m0.527998748s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644900409 (606h22m0.528314035s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646038897 (606h22m0.529452526s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646375740 (606h22m0.529789369s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.648944423 (606h22m0.53235805s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.650463350 (606h22m0.533876978s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.651369647 (606h22m0.534783267s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:01.262589342 (606h22m1.146002974s) [upload=484797]: Next returned nil segment Jul 18 15:33:01.262648256 (606h22m1.146061891s) [upload=484797]: Waiting for error group managing segments... Jul 18 15:33:01.398379699 (606h22m1.281793326s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404946640 (606h22m1.288360283s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404992987 (606h22m1.28840661s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.405551748 (606h22m1.288965369s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.407987418 (606h22m1.291401032s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.410553731 (606h22m1.293967362s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.416771944 (606h22m1.300185579s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.425807204 (606h22m1.309220819s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.430635732 (606h22m1.314049361s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.433605056 (606h22m1.317018687s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.440793839 (606h22m1.32420747s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.446368593 (606h22m1.329782225s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:30.258196584 (606h22m30.141610202s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258423161 (606h22m30.141836789s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:40968->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258491071 (606h22m30.1419047s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258527956 (606h22m30.141941579s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258553948 (606h22m30.141967578s) [seg_pos={0 0}, upload=484797]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 15:33:30.369475355 (606h22m30.252888992s) [upload=484797]: Done uploading segments... Jul 18 15:33:30.369509072 (606h22m30.252922716s) [upload=484797]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.546081207s core uplink us-select-1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 30 out of 60 is failed (threshold=3) [..............................XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 177.883937ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 16:11:35 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 177.925524ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 28 out of 60 is failed (threshold=3) [................................XXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 173.046211ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 16:11:27 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 173.142045ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 29 out of 60 is failed (threshold=3) [...............................XXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 180.176516ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 16:11:15 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 180.207349ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......XX..X.................................................] duration: 2.437500354s FAIL (2025-07-18 16:10:45 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== BUMKAHO3ZMVDNWVSRRI3GTLM4MO3MP6MNNGUUZKBLBWQ, seg_pos={0 0}, upload=484783]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: rpc: dial tcp 77.90.31.13:4628: operation was canceled; context canceled Jul 18 15:31:52.267100372 (606h20m52.150513985s) [seg_pos={0 0}, upload=484783]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:31:52.267178520 (606h20m52.150592177s) [upload=484783]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:31:52.267184778 (606h20m52.150598408s) [upload=484783]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:31:52.565015623 (606h20m52.448429239s) [upload=484783]: Done uploading segments... Jul 18 15:31:52.565033307 (606h20m52.448446919s) [upload=484783]: Upload finished. err: <nil> Jul 18 15:33:00.528847457 (606h22m0.412261094s) [upload=484797]: Starting upload Jul 18 15:33:00.528934444 (606h22m0.412348056s) [upload=484797]: Done starting upload Jul 18 15:33:00.529017539 (606h22m0.412431168s) [upload=484797]: Uploading segments... Jul 18 15:33:00.529303798 (606h22m0.412717427s) [upload=484797]: Got next segment. Inline: false Jul 18 15:33:00.529348408 (606h22m0.41276202s) [upload=484797]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:33:00.529353973 (606h22m0.412767599s) [upload=484797]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:33:00.640500227 (606h22m0.523913861s) [seg_pos={0 0}, upload=484797]: Begin upload segment... Jul 18 15:33:00.640588551 (606h22m0.524002189s) [seg_pos={0 0}, upload=484797]: Done begin upload segment. Jul 18 15:33:00.640649191 (606h22m0.524062829s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.640953078 (606h22m0.524366715s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641531127 (606h22m0.524944758s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641980147 (606h22m0.52539378s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642678070 (606h22m0.526091699s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642961859 (606h22m0.526375491s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643490795 (606h22m0.526904425s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643733424 (606h22m0.52714705s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644374317 (606h22m0.527787948s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644585121 (606h22m0.527998748s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644900409 (606h22m0.528314035s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646038897 (606h22m0.529452526s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646375740 (606h22m0.529789369s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.648944423 (606h22m0.53235805s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.650463350 (606h22m0.533876978s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.651369647 (606h22m0.534783267s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:01.262589342 (606h22m1.146002974s) [upload=484797]: Next returned nil segment Jul 18 15:33:01.262648256 (606h22m1.146061891s) [upload=484797]: Waiting for error group managing segments... Jul 18 15:33:01.398379699 (606h22m1.281793326s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404946640 (606h22m1.288360283s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404992987 (606h22m1.28840661s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.405551748 (606h22m1.288965369s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.407987418 (606h22m1.291401032s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.410553731 (606h22m1.293967362s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.416771944 (606h22m1.300185579s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.425807204 (606h22m1.309220819s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.430635732 (606h22m1.314049361s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.433605056 (606h22m1.317018687s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.440793839 (606h22m1.32420747s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.446368593 (606h22m1.329782225s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:30.258196584 (606h22m30.141610202s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258423161 (606h22m30.141836789s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:40968->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258491071 (606h22m30.1419047s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258527956 (606h22m30.141941579s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258553948 (606h22m30.141967578s) [seg_pos={0 0}, upload=484797]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 15:33:30.369475355 (606h22m30.252888992s) [upload=484797]: Done uploading segments... Jul 18 15:33:30.369509072 (606h22m30.252922716s) [upload=484797]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.437563348s core uplink us-select-1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 28 out of 60 is failed (threshold=3) [................................XXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 176.483342ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 16:10:32 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 176.505165ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 27 out of 60 is failed (threshold=3) [.................................XXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 177.326066ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 16:10:31 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 177.396742ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 26 out of 60 is failed (threshold=3) [..................................XXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 172.581569ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 16:10:00 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 172.605658ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......XX..X................................................] duration: 2.461407269s FAIL (2025-07-18 16:09:49 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== BUMKAHO3ZMVDNWVSRRI3GTLM4MO3MP6MNNGUUZKBLBWQ, seg_pos={0 0}, upload=484783]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: rpc: dial tcp 77.90.31.13:4628: operation was canceled; context canceled Jul 18 15:31:52.267100372 (606h20m52.150513985s) [seg_pos={0 0}, upload=484783]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:31:52.267178520 (606h20m52.150592177s) [upload=484783]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:31:52.267184778 (606h20m52.150598408s) [upload=484783]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:31:52.565015623 (606h20m52.448429239s) [upload=484783]: Done uploading segments... Jul 18 15:31:52.565033307 (606h20m52.448446919s) [upload=484783]: Upload finished. err: <nil> Jul 18 15:33:00.528847457 (606h22m0.412261094s) [upload=484797]: Starting upload Jul 18 15:33:00.528934444 (606h22m0.412348056s) [upload=484797]: Done starting upload Jul 18 15:33:00.529017539 (606h22m0.412431168s) [upload=484797]: Uploading segments... Jul 18 15:33:00.529303798 (606h22m0.412717427s) [upload=484797]: Got next segment. Inline: false Jul 18 15:33:00.529348408 (606h22m0.41276202s) [upload=484797]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:33:00.529353973 (606h22m0.412767599s) [upload=484797]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:33:00.640500227 (606h22m0.523913861s) [seg_pos={0 0}, upload=484797]: Begin upload segment... Jul 18 15:33:00.640588551 (606h22m0.524002189s) [seg_pos={0 0}, upload=484797]: Done begin upload segment. Jul 18 15:33:00.640649191 (606h22m0.524062829s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.640953078 (606h22m0.524366715s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641531127 (606h22m0.524944758s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641980147 (606h22m0.52539378s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642678070 (606h22m0.526091699s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642961859 (606h22m0.526375491s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643490795 (606h22m0.526904425s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643733424 (606h22m0.52714705s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644374317 (606h22m0.527787948s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644585121 (606h22m0.527998748s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644900409 (606h22m0.528314035s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646038897 (606h22m0.529452526s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646375740 (606h22m0.529789369s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.648944423 (606h22m0.53235805s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.650463350 (606h22m0.533876978s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.651369647 (606h22m0.534783267s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:01.262589342 (606h22m1.146002974s) [upload=484797]: Next returned nil segment Jul 18 15:33:01.262648256 (606h22m1.146061891s) [upload=484797]: Waiting for error group managing segments... Jul 18 15:33:01.398379699 (606h22m1.281793326s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404946640 (606h22m1.288360283s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404992987 (606h22m1.28840661s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.405551748 (606h22m1.288965369s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.407987418 (606h22m1.291401032s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.410553731 (606h22m1.293967362s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.416771944 (606h22m1.300185579s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.425807204 (606h22m1.309220819s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.430635732 (606h22m1.314049361s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.433605056 (606h22m1.317018687s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.440793839 (606h22m1.32420747s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.446368593 (606h22m1.329782225s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:30.258196584 (606h22m30.141610202s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258423161 (606h22m30.141836789s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:40968->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258491071 (606h22m30.1419047s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258527956 (606h22m30.141941579s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258553948 (606h22m30.141967578s) [seg_pos={0 0}, upload=484797]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 15:33:30.369475355 (606h22m30.252888992s) [upload=484797]: Done uploading segments... Jul 18 15:33:30.369509072 (606h22m30.252922716s) [upload=484797]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.46144143s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........XX..X...............................................] duration: 2.39572295s FAIL (2025-07-18 16:09:32 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== BUMKAHO3ZMVDNWVSRRI3GTLM4MO3MP6MNNGUUZKBLBWQ, seg_pos={0 0}, upload=484783]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: rpc: dial tcp 77.90.31.13:4628: operation was canceled; context canceled Jul 18 15:31:52.267100372 (606h20m52.150513985s) [seg_pos={0 0}, upload=484783]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:31:52.267178520 (606h20m52.150592177s) [upload=484783]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:31:52.267184778 (606h20m52.150598408s) [upload=484783]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:31:52.565015623 (606h20m52.448429239s) [upload=484783]: Done uploading segments... Jul 18 15:31:52.565033307 (606h20m52.448446919s) [upload=484783]: Upload finished. err: <nil> Jul 18 15:33:00.528847457 (606h22m0.412261094s) [upload=484797]: Starting upload Jul 18 15:33:00.528934444 (606h22m0.412348056s) [upload=484797]: Done starting upload Jul 18 15:33:00.529017539 (606h22m0.412431168s) [upload=484797]: Uploading segments... Jul 18 15:33:00.529303798 (606h22m0.412717427s) [upload=484797]: Got next segment. Inline: false Jul 18 15:33:00.529348408 (606h22m0.41276202s) [upload=484797]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:33:00.529353973 (606h22m0.412767599s) [upload=484797]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:33:00.640500227 (606h22m0.523913861s) [seg_pos={0 0}, upload=484797]: Begin upload segment... Jul 18 15:33:00.640588551 (606h22m0.524002189s) [seg_pos={0 0}, upload=484797]: Done begin upload segment. Jul 18 15:33:00.640649191 (606h22m0.524062829s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.640953078 (606h22m0.524366715s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641531127 (606h22m0.524944758s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641980147 (606h22m0.52539378s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642678070 (606h22m0.526091699s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642961859 (606h22m0.526375491s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643490795 (606h22m0.526904425s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643733424 (606h22m0.52714705s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644374317 (606h22m0.527787948s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644585121 (606h22m0.527998748s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644900409 (606h22m0.528314035s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646038897 (606h22m0.529452526s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646375740 (606h22m0.529789369s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.648944423 (606h22m0.53235805s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.650463350 (606h22m0.533876978s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.651369647 (606h22m0.534783267s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:01.262589342 (606h22m1.146002974s) [upload=484797]: Next returned nil segment Jul 18 15:33:01.262648256 (606h22m1.146061891s) [upload=484797]: Waiting for error group managing segments... Jul 18 15:33:01.398379699 (606h22m1.281793326s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404946640 (606h22m1.288360283s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404992987 (606h22m1.28840661s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.405551748 (606h22m1.288965369s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.407987418 (606h22m1.291401032s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.410553731 (606h22m1.293967362s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.416771944 (606h22m1.300185579s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.425807204 (606h22m1.309220819s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.430635732 (606h22m1.314049361s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.433605056 (606h22m1.317018687s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.440793839 (606h22m1.32420747s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.446368593 (606h22m1.329782225s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:30.258196584 (606h22m30.141610202s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258423161 (606h22m30.141836789s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:40968->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258491071 (606h22m30.1419047s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258527956 (606h22m30.141941579s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258553948 (606h22m30.141967578s) [seg_pos={0 0}, upload=484797]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 15:33:30.369475355 (606h22m30.252888992s) [upload=484797]: Done uploading segments... Jul 18 15:33:30.369509072 (606h22m30.252922716s) [upload=484797]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.395756019s core uplink us-select-1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 27 out of 60 is failed (threshold=3) [.................................XXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 173.599592ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 16:09:19 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 173.625058ms edge ipfs us1

Orbiter Check History

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)