Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......XX..X.................................................] duration: 2.437500354s FAIL (2025-07-18 16:10:45 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== BUMKAHO3ZMVDNWVSRRI3GTLM4MO3MP6MNNGUUZKBLBWQ, seg_pos={0 0}, upload=484783]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: rpc: dial tcp 77.90.31.13:4628: operation was canceled; context canceled Jul 18 15:31:52.267100372 (606h20m52.150513985s) [seg_pos={0 0}, upload=484783]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:31:52.267178520 (606h20m52.150592177s) [upload=484783]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:31:52.267184778 (606h20m52.150598408s) [upload=484783]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:31:52.565015623 (606h20m52.448429239s) [upload=484783]: Done uploading segments... Jul 18 15:31:52.565033307 (606h20m52.448446919s) [upload=484783]: Upload finished. err: <nil> Jul 18 15:33:00.528847457 (606h22m0.412261094s) [upload=484797]: Starting upload Jul 18 15:33:00.528934444 (606h22m0.412348056s) [upload=484797]: Done starting upload Jul 18 15:33:00.529017539 (606h22m0.412431168s) [upload=484797]: Uploading segments... Jul 18 15:33:00.529303798 (606h22m0.412717427s) [upload=484797]: Got next segment. Inline: false Jul 18 15:33:00.529348408 (606h22m0.41276202s) [upload=484797]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:33:00.529353973 (606h22m0.412767599s) [upload=484797]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:33:00.640500227 (606h22m0.523913861s) [seg_pos={0 0}, upload=484797]: Begin upload segment... Jul 18 15:33:00.640588551 (606h22m0.524002189s) [seg_pos={0 0}, upload=484797]: Done begin upload segment. Jul 18 15:33:00.640649191 (606h22m0.524062829s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.640953078 (606h22m0.524366715s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641531127 (606h22m0.524944758s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641980147 (606h22m0.52539378s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642678070 (606h22m0.526091699s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642961859 (606h22m0.526375491s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643490795 (606h22m0.526904425s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643733424 (606h22m0.52714705s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644374317 (606h22m0.527787948s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644585121 (606h22m0.527998748s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644900409 (606h22m0.528314035s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646038897 (606h22m0.529452526s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646375740 (606h22m0.529789369s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.648944423 (606h22m0.53235805s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.650463350 (606h22m0.533876978s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.651369647 (606h22m0.534783267s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:01.262589342 (606h22m1.146002974s) [upload=484797]: Next returned nil segment Jul 18 15:33:01.262648256 (606h22m1.146061891s) [upload=484797]: Waiting for error group managing segments... Jul 18 15:33:01.398379699 (606h22m1.281793326s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404946640 (606h22m1.288360283s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404992987 (606h22m1.28840661s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.405551748 (606h22m1.288965369s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.407987418 (606h22m1.291401032s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.410553731 (606h22m1.293967362s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.416771944 (606h22m1.300185579s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.425807204 (606h22m1.309220819s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.430635732 (606h22m1.314049361s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.433605056 (606h22m1.317018687s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.440793839 (606h22m1.32420747s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.446368593 (606h22m1.329782225s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:30.258196584 (606h22m30.141610202s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258423161 (606h22m30.141836789s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:40968->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258491071 (606h22m30.1419047s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258527956 (606h22m30.141941579s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258553948 (606h22m30.141967578s) [seg_pos={0 0}, upload=484797]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 15:33:30.369475355 (606h22m30.252888992s) [upload=484797]: Done uploading segments... Jul 18 15:33:30.369509072 (606h22m30.252922716s) [upload=484797]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.437563348s core uplink us-select-1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 28 out of 60 is failed (threshold=3) [................................XXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 176.483342ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 16:10:32 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 176.505165ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 27 out of 60 is failed (threshold=3) [.................................XXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 177.326066ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 16:10:31 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 177.396742ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 26 out of 60 is failed (threshold=3) [..................................XXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 172.581569ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 16:10:00 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 172.605658ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......XX..X................................................] duration: 2.461407269s FAIL (2025-07-18 16:09:49 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== BUMKAHO3ZMVDNWVSRRI3GTLM4MO3MP6MNNGUUZKBLBWQ, seg_pos={0 0}, upload=484783]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: rpc: dial tcp 77.90.31.13:4628: operation was canceled; context canceled Jul 18 15:31:52.267100372 (606h20m52.150513985s) [seg_pos={0 0}, upload=484783]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:31:52.267178520 (606h20m52.150592177s) [upload=484783]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:31:52.267184778 (606h20m52.150598408s) [upload=484783]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:31:52.565015623 (606h20m52.448429239s) [upload=484783]: Done uploading segments... Jul 18 15:31:52.565033307 (606h20m52.448446919s) [upload=484783]: Upload finished. err: <nil> Jul 18 15:33:00.528847457 (606h22m0.412261094s) [upload=484797]: Starting upload Jul 18 15:33:00.528934444 (606h22m0.412348056s) [upload=484797]: Done starting upload Jul 18 15:33:00.529017539 (606h22m0.412431168s) [upload=484797]: Uploading segments... Jul 18 15:33:00.529303798 (606h22m0.412717427s) [upload=484797]: Got next segment. Inline: false Jul 18 15:33:00.529348408 (606h22m0.41276202s) [upload=484797]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:33:00.529353973 (606h22m0.412767599s) [upload=484797]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:33:00.640500227 (606h22m0.523913861s) [seg_pos={0 0}, upload=484797]: Begin upload segment... Jul 18 15:33:00.640588551 (606h22m0.524002189s) [seg_pos={0 0}, upload=484797]: Done begin upload segment. Jul 18 15:33:00.640649191 (606h22m0.524062829s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.640953078 (606h22m0.524366715s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641531127 (606h22m0.524944758s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641980147 (606h22m0.52539378s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642678070 (606h22m0.526091699s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642961859 (606h22m0.526375491s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643490795 (606h22m0.526904425s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643733424 (606h22m0.52714705s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644374317 (606h22m0.527787948s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644585121 (606h22m0.527998748s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644900409 (606h22m0.528314035s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646038897 (606h22m0.529452526s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646375740 (606h22m0.529789369s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.648944423 (606h22m0.53235805s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.650463350 (606h22m0.533876978s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.651369647 (606h22m0.534783267s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:01.262589342 (606h22m1.146002974s) [upload=484797]: Next returned nil segment Jul 18 15:33:01.262648256 (606h22m1.146061891s) [upload=484797]: Waiting for error group managing segments... Jul 18 15:33:01.398379699 (606h22m1.281793326s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404946640 (606h22m1.288360283s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404992987 (606h22m1.28840661s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.405551748 (606h22m1.288965369s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.407987418 (606h22m1.291401032s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.410553731 (606h22m1.293967362s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.416771944 (606h22m1.300185579s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.425807204 (606h22m1.309220819s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.430635732 (606h22m1.314049361s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.433605056 (606h22m1.317018687s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.440793839 (606h22m1.32420747s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.446368593 (606h22m1.329782225s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:30.258196584 (606h22m30.141610202s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258423161 (606h22m30.141836789s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:40968->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258491071 (606h22m30.1419047s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258527956 (606h22m30.141941579s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258553948 (606h22m30.141967578s) [seg_pos={0 0}, upload=484797]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 15:33:30.369475355 (606h22m30.252888992s) [upload=484797]: Done uploading segments... Jul 18 15:33:30.369509072 (606h22m30.252922716s) [upload=484797]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.46144143s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........XX..X...............................................] duration: 2.39572295s FAIL (2025-07-18 16:09:32 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== BUMKAHO3ZMVDNWVSRRI3GTLM4MO3MP6MNNGUUZKBLBWQ, seg_pos={0 0}, upload=484783]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: rpc: dial tcp 77.90.31.13:4628: operation was canceled; context canceled Jul 18 15:31:52.267100372 (606h20m52.150513985s) [seg_pos={0 0}, upload=484783]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:31:52.267178520 (606h20m52.150592177s) [upload=484783]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:31:52.267184778 (606h20m52.150598408s) [upload=484783]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:31:52.565015623 (606h20m52.448429239s) [upload=484783]: Done uploading segments... Jul 18 15:31:52.565033307 (606h20m52.448446919s) [upload=484783]: Upload finished. err: <nil> Jul 18 15:33:00.528847457 (606h22m0.412261094s) [upload=484797]: Starting upload Jul 18 15:33:00.528934444 (606h22m0.412348056s) [upload=484797]: Done starting upload Jul 18 15:33:00.529017539 (606h22m0.412431168s) [upload=484797]: Uploading segments... Jul 18 15:33:00.529303798 (606h22m0.412717427s) [upload=484797]: Got next segment. Inline: false Jul 18 15:33:00.529348408 (606h22m0.41276202s) [upload=484797]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:33:00.529353973 (606h22m0.412767599s) [upload=484797]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:33:00.640500227 (606h22m0.523913861s) [seg_pos={0 0}, upload=484797]: Begin upload segment... Jul 18 15:33:00.640588551 (606h22m0.524002189s) [seg_pos={0 0}, upload=484797]: Done begin upload segment. Jul 18 15:33:00.640649191 (606h22m0.524062829s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.640953078 (606h22m0.524366715s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641531127 (606h22m0.524944758s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641980147 (606h22m0.52539378s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642678070 (606h22m0.526091699s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642961859 (606h22m0.526375491s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643490795 (606h22m0.526904425s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643733424 (606h22m0.52714705s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644374317 (606h22m0.527787948s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644585121 (606h22m0.527998748s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644900409 (606h22m0.528314035s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646038897 (606h22m0.529452526s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646375740 (606h22m0.529789369s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.648944423 (606h22m0.53235805s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.650463350 (606h22m0.533876978s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.651369647 (606h22m0.534783267s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:01.262589342 (606h22m1.146002974s) [upload=484797]: Next returned nil segment Jul 18 15:33:01.262648256 (606h22m1.146061891s) [upload=484797]: Waiting for error group managing segments... Jul 18 15:33:01.398379699 (606h22m1.281793326s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404946640 (606h22m1.288360283s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404992987 (606h22m1.28840661s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.405551748 (606h22m1.288965369s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.407987418 (606h22m1.291401032s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.410553731 (606h22m1.293967362s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.416771944 (606h22m1.300185579s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.425807204 (606h22m1.309220819s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.430635732 (606h22m1.314049361s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.433605056 (606h22m1.317018687s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.440793839 (606h22m1.32420747s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.446368593 (606h22m1.329782225s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:30.258196584 (606h22m30.141610202s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258423161 (606h22m30.141836789s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:40968->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258491071 (606h22m30.1419047s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258527956 (606h22m30.141941579s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258553948 (606h22m30.141967578s) [seg_pos={0 0}, upload=484797]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 15:33:30.369475355 (606h22m30.252888992s) [upload=484797]: Done uploading segments... Jul 18 15:33:30.369509072 (606h22m30.252922716s) [upload=484797]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.395756019s core uplink us-select-1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 27 out of 60 is failed (threshold=3) [.................................XXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 173.599592ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 16:09:19 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 173.625058ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........XX..X..............................................] duration: 2.483514363s FAIL (2025-07-18 16:09:14 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== BUMKAHO3ZMVDNWVSRRI3GTLM4MO3MP6MNNGUUZKBLBWQ, seg_pos={0 0}, upload=484783]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: rpc: dial tcp 77.90.31.13:4628: operation was canceled; context canceled Jul 18 15:31:52.267100372 (606h20m52.150513985s) [seg_pos={0 0}, upload=484783]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:31:52.267178520 (606h20m52.150592177s) [upload=484783]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:31:52.267184778 (606h20m52.150598408s) [upload=484783]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:31:52.565015623 (606h20m52.448429239s) [upload=484783]: Done uploading segments... Jul 18 15:31:52.565033307 (606h20m52.448446919s) [upload=484783]: Upload finished. err: <nil> Jul 18 15:33:00.528847457 (606h22m0.412261094s) [upload=484797]: Starting upload Jul 18 15:33:00.528934444 (606h22m0.412348056s) [upload=484797]: Done starting upload Jul 18 15:33:00.529017539 (606h22m0.412431168s) [upload=484797]: Uploading segments... Jul 18 15:33:00.529303798 (606h22m0.412717427s) [upload=484797]: Got next segment. Inline: false Jul 18 15:33:00.529348408 (606h22m0.41276202s) [upload=484797]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:33:00.529353973 (606h22m0.412767599s) [upload=484797]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:33:00.640500227 (606h22m0.523913861s) [seg_pos={0 0}, upload=484797]: Begin upload segment... Jul 18 15:33:00.640588551 (606h22m0.524002189s) [seg_pos={0 0}, upload=484797]: Done begin upload segment. Jul 18 15:33:00.640649191 (606h22m0.524062829s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.640953078 (606h22m0.524366715s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641531127 (606h22m0.524944758s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641980147 (606h22m0.52539378s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642678070 (606h22m0.526091699s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642961859 (606h22m0.526375491s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643490795 (606h22m0.526904425s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643733424 (606h22m0.52714705s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644374317 (606h22m0.527787948s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644585121 (606h22m0.527998748s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644900409 (606h22m0.528314035s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646038897 (606h22m0.529452526s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646375740 (606h22m0.529789369s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.648944423 (606h22m0.53235805s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.650463350 (606h22m0.533876978s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.651369647 (606h22m0.534783267s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:01.262589342 (606h22m1.146002974s) [upload=484797]: Next returned nil segment Jul 18 15:33:01.262648256 (606h22m1.146061891s) [upload=484797]: Waiting for error group managing segments... Jul 18 15:33:01.398379699 (606h22m1.281793326s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404946640 (606h22m1.288360283s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404992987 (606h22m1.28840661s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.405551748 (606h22m1.288965369s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.407987418 (606h22m1.291401032s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.410553731 (606h22m1.293967362s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.416771944 (606h22m1.300185579s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.425807204 (606h22m1.309220819s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.430635732 (606h22m1.314049361s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.433605056 (606h22m1.317018687s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.440793839 (606h22m1.32420747s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.446368593 (606h22m1.329782225s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:30.258196584 (606h22m30.141610202s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258423161 (606h22m30.141836789s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:40968->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258491071 (606h22m30.1419047s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258527956 (606h22m30.141941579s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258553948 (606h22m30.141967578s) [seg_pos={0 0}, upload=484797]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 15:33:30.369475355 (606h22m30.252888992s) [upload=484797]: Done uploading segments... Jul 18 15:33:30.369509072 (606h22m30.252922716s) [upload=484797]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.48354563s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 25 out of 60 is failed (threshold=3) [...................................XXXXXXXXXXXXXXXXXXXXXXXXX] duration: 186.306413ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 16:09:03 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 186.346526ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........XX..X.............................................] duration: 2.4044072s FAIL (2025-07-18 16:08:39 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== BUMKAHO3ZMVDNWVSRRI3GTLM4MO3MP6MNNGUUZKBLBWQ, seg_pos={0 0}, upload=484783]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: rpc: dial tcp 77.90.31.13:4628: operation was canceled; context canceled Jul 18 15:31:52.267100372 (606h20m52.150513985s) [seg_pos={0 0}, upload=484783]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:31:52.267178520 (606h20m52.150592177s) [upload=484783]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:31:52.267184778 (606h20m52.150598408s) [upload=484783]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:31:52.565015623 (606h20m52.448429239s) [upload=484783]: Done uploading segments... Jul 18 15:31:52.565033307 (606h20m52.448446919s) [upload=484783]: Upload finished. err: <nil> Jul 18 15:33:00.528847457 (606h22m0.412261094s) [upload=484797]: Starting upload Jul 18 15:33:00.528934444 (606h22m0.412348056s) [upload=484797]: Done starting upload Jul 18 15:33:00.529017539 (606h22m0.412431168s) [upload=484797]: Uploading segments... Jul 18 15:33:00.529303798 (606h22m0.412717427s) [upload=484797]: Got next segment. Inline: false Jul 18 15:33:00.529348408 (606h22m0.41276202s) [upload=484797]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:33:00.529353973 (606h22m0.412767599s) [upload=484797]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:33:00.640500227 (606h22m0.523913861s) [seg_pos={0 0}, upload=484797]: Begin upload segment... Jul 18 15:33:00.640588551 (606h22m0.524002189s) [seg_pos={0 0}, upload=484797]: Done begin upload segment. Jul 18 15:33:00.640649191 (606h22m0.524062829s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.640953078 (606h22m0.524366715s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641531127 (606h22m0.524944758s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641980147 (606h22m0.52539378s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642678070 (606h22m0.526091699s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642961859 (606h22m0.526375491s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643490795 (606h22m0.526904425s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643733424 (606h22m0.52714705s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644374317 (606h22m0.527787948s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644585121 (606h22m0.527998748s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644900409 (606h22m0.528314035s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646038897 (606h22m0.529452526s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646375740 (606h22m0.529789369s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.648944423 (606h22m0.53235805s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.650463350 (606h22m0.533876978s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.651369647 (606h22m0.534783267s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:01.262589342 (606h22m1.146002974s) [upload=484797]: Next returned nil segment Jul 18 15:33:01.262648256 (606h22m1.146061891s) [upload=484797]: Waiting for error group managing segments... Jul 18 15:33:01.398379699 (606h22m1.281793326s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404946640 (606h22m1.288360283s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404992987 (606h22m1.28840661s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.405551748 (606h22m1.288965369s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.407987418 (606h22m1.291401032s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.410553731 (606h22m1.293967362s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.416771944 (606h22m1.300185579s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.425807204 (606h22m1.309220819s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.430635732 (606h22m1.314049361s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.433605056 (606h22m1.317018687s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.440793839 (606h22m1.32420747s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.446368593 (606h22m1.329782225s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:30.258196584 (606h22m30.141610202s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258423161 (606h22m30.141836789s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:40968->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258491071 (606h22m30.1419047s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258527956 (606h22m30.141941579s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258553948 (606h22m30.141967578s) [seg_pos={0 0}, upload=484797]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 15:33:30.369475355 (606h22m30.252888992s) [upload=484797]: Done uploading segments... Jul 18 15:33:30.369509072 (606h22m30.252922716s) [upload=484797]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.404437098s core uplink us-select-1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 26 out of 60 is failed (threshold=3) [..................................XXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 174.873705ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 16:08:36 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 174.898535ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 25 out of 60 is failed (threshold=3) [...................................XXXXXXXXXXXXXXXXXXXXXXXXX] duration: 206.358085ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 16:08:13 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 206.418392ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 24 out of 60 is failed (threshold=3) [....................................XXXXXXXXXXXXXXXXXXXXXXXX] duration: 176.179344ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 16:07:56 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 176.205532ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 24 out of 60 is failed (threshold=3) [....................................XXXXXXXXXXXXXXXXXXXXXXXX] duration: 178.934138ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 16:07:54 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 178.958277ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........XX..X............................................] duration: 2.38212239s FAIL (2025-07-18 16:07:27 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== BUMKAHO3ZMVDNWVSRRI3GTLM4MO3MP6MNNGUUZKBLBWQ, seg_pos={0 0}, upload=484783]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: rpc: dial tcp 77.90.31.13:4628: operation was canceled; context canceled Jul 18 15:31:52.267100372 (606h20m52.150513985s) [seg_pos={0 0}, upload=484783]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:31:52.267178520 (606h20m52.150592177s) [upload=484783]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:31:52.267184778 (606h20m52.150598408s) [upload=484783]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:31:52.565015623 (606h20m52.448429239s) [upload=484783]: Done uploading segments... Jul 18 15:31:52.565033307 (606h20m52.448446919s) [upload=484783]: Upload finished. err: <nil> Jul 18 15:33:00.528847457 (606h22m0.412261094s) [upload=484797]: Starting upload Jul 18 15:33:00.528934444 (606h22m0.412348056s) [upload=484797]: Done starting upload Jul 18 15:33:00.529017539 (606h22m0.412431168s) [upload=484797]: Uploading segments... Jul 18 15:33:00.529303798 (606h22m0.412717427s) [upload=484797]: Got next segment. Inline: false Jul 18 15:33:00.529348408 (606h22m0.41276202s) [upload=484797]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:33:00.529353973 (606h22m0.412767599s) [upload=484797]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:33:00.640500227 (606h22m0.523913861s) [seg_pos={0 0}, upload=484797]: Begin upload segment... Jul 18 15:33:00.640588551 (606h22m0.524002189s) [seg_pos={0 0}, upload=484797]: Done begin upload segment. Jul 18 15:33:00.640649191 (606h22m0.524062829s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.640953078 (606h22m0.524366715s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641531127 (606h22m0.524944758s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641980147 (606h22m0.52539378s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642678070 (606h22m0.526091699s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642961859 (606h22m0.526375491s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643490795 (606h22m0.526904425s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643733424 (606h22m0.52714705s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644374317 (606h22m0.527787948s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644585121 (606h22m0.527998748s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644900409 (606h22m0.528314035s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646038897 (606h22m0.529452526s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646375740 (606h22m0.529789369s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.648944423 (606h22m0.53235805s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.650463350 (606h22m0.533876978s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.651369647 (606h22m0.534783267s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:01.262589342 (606h22m1.146002974s) [upload=484797]: Next returned nil segment Jul 18 15:33:01.262648256 (606h22m1.146061891s) [upload=484797]: Waiting for error group managing segments... Jul 18 15:33:01.398379699 (606h22m1.281793326s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404946640 (606h22m1.288360283s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404992987 (606h22m1.28840661s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.405551748 (606h22m1.288965369s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.407987418 (606h22m1.291401032s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.410553731 (606h22m1.293967362s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.416771944 (606h22m1.300185579s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.425807204 (606h22m1.309220819s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.430635732 (606h22m1.314049361s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.433605056 (606h22m1.317018687s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.440793839 (606h22m1.32420747s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.446368593 (606h22m1.329782225s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:30.258196584 (606h22m30.141610202s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258423161 (606h22m30.141836789s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:40968->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258491071 (606h22m30.1419047s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258527956 (606h22m30.141941579s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258553948 (606h22m30.141967578s) [seg_pos={0 0}, upload=484797]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 15:33:30.369475355 (606h22m30.252888992s) [upload=484797]: Done uploading segments... Jul 18 15:33:30.369509072 (606h22m30.252922716s) [upload=484797]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.382147914s core uplink us-select-1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 23 out of 60 is failed (threshold=3) [.....................................XXXXXXXXXXXXXXXXXXXXXXX] duration: 177.279484ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 16:06:56 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 177.305932ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [............XX..X...........................................] duration: 2.371668631s FAIL (2025-07-18 16:06:55 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== BUMKAHO3ZMVDNWVSRRI3GTLM4MO3MP6MNNGUUZKBLBWQ, seg_pos={0 0}, upload=484783]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: rpc: dial tcp 77.90.31.13:4628: operation was canceled; context canceled Jul 18 15:31:52.267100372 (606h20m52.150513985s) [seg_pos={0 0}, upload=484783]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:31:52.267178520 (606h20m52.150592177s) [upload=484783]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:31:52.267184778 (606h20m52.150598408s) [upload=484783]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:31:52.565015623 (606h20m52.448429239s) [upload=484783]: Done uploading segments... Jul 18 15:31:52.565033307 (606h20m52.448446919s) [upload=484783]: Upload finished. err: <nil> Jul 18 15:33:00.528847457 (606h22m0.412261094s) [upload=484797]: Starting upload Jul 18 15:33:00.528934444 (606h22m0.412348056s) [upload=484797]: Done starting upload Jul 18 15:33:00.529017539 (606h22m0.412431168s) [upload=484797]: Uploading segments... Jul 18 15:33:00.529303798 (606h22m0.412717427s) [upload=484797]: Got next segment. Inline: false Jul 18 15:33:00.529348408 (606h22m0.41276202s) [upload=484797]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:33:00.529353973 (606h22m0.412767599s) [upload=484797]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:33:00.640500227 (606h22m0.523913861s) [seg_pos={0 0}, upload=484797]: Begin upload segment... Jul 18 15:33:00.640588551 (606h22m0.524002189s) [seg_pos={0 0}, upload=484797]: Done begin upload segment. Jul 18 15:33:00.640649191 (606h22m0.524062829s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.640953078 (606h22m0.524366715s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641531127 (606h22m0.524944758s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641980147 (606h22m0.52539378s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642678070 (606h22m0.526091699s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642961859 (606h22m0.526375491s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643490795 (606h22m0.526904425s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643733424 (606h22m0.52714705s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644374317 (606h22m0.527787948s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644585121 (606h22m0.527998748s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644900409 (606h22m0.528314035s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646038897 (606h22m0.529452526s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646375740 (606h22m0.529789369s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.648944423 (606h22m0.53235805s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.650463350 (606h22m0.533876978s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.651369647 (606h22m0.534783267s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:01.262589342 (606h22m1.146002974s) [upload=484797]: Next returned nil segment Jul 18 15:33:01.262648256 (606h22m1.146061891s) [upload=484797]: Waiting for error group managing segments... Jul 18 15:33:01.398379699 (606h22m1.281793326s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404946640 (606h22m1.288360283s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404992987 (606h22m1.28840661s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.405551748 (606h22m1.288965369s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.407987418 (606h22m1.291401032s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.410553731 (606h22m1.293967362s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.416771944 (606h22m1.300185579s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.425807204 (606h22m1.309220819s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.430635732 (606h22m1.314049361s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.433605056 (606h22m1.317018687s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.440793839 (606h22m1.32420747s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.446368593 (606h22m1.329782225s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:30.258196584 (606h22m30.141610202s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258423161 (606h22m30.141836789s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:40968->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258491071 (606h22m30.1419047s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258527956 (606h22m30.141941579s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258553948 (606h22m30.141967578s) [seg_pos={0 0}, upload=484797]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 15:33:30.369475355 (606h22m30.252888992s) [upload=484797]: Done uploading segments... Jul 18 15:33:30.369509072 (606h22m30.252922716s) [upload=484797]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.371721402s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 23 out of 60 is failed (threshold=3) [.....................................XXXXXXXXXXXXXXXXXXXXXXX] duration: 171.832617ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 16:06:43 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 171.853812ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 22 out of 60 is failed (threshold=3) [......................................XXXXXXXXXXXXXXXXXXXXXX] duration: 167.310846ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 16:06:10 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 167.353653ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............XX..X..........................................] duration: 2.435868757s FAIL (2025-07-18 16:06:04 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== BUMKAHO3ZMVDNWVSRRI3GTLM4MO3MP6MNNGUUZKBLBWQ, seg_pos={0 0}, upload=484783]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: rpc: dial tcp 77.90.31.13:4628: operation was canceled; context canceled Jul 18 15:31:52.267100372 (606h20m52.150513985s) [seg_pos={0 0}, upload=484783]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:31:52.267178520 (606h20m52.150592177s) [upload=484783]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:31:52.267184778 (606h20m52.150598408s) [upload=484783]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:31:52.565015623 (606h20m52.448429239s) [upload=484783]: Done uploading segments... Jul 18 15:31:52.565033307 (606h20m52.448446919s) [upload=484783]: Upload finished. err: <nil> Jul 18 15:33:00.528847457 (606h22m0.412261094s) [upload=484797]: Starting upload Jul 18 15:33:00.528934444 (606h22m0.412348056s) [upload=484797]: Done starting upload Jul 18 15:33:00.529017539 (606h22m0.412431168s) [upload=484797]: Uploading segments... Jul 18 15:33:00.529303798 (606h22m0.412717427s) [upload=484797]: Got next segment. Inline: false Jul 18 15:33:00.529348408 (606h22m0.41276202s) [upload=484797]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:33:00.529353973 (606h22m0.412767599s) [upload=484797]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:33:00.640500227 (606h22m0.523913861s) [seg_pos={0 0}, upload=484797]: Begin upload segment... Jul 18 15:33:00.640588551 (606h22m0.524002189s) [seg_pos={0 0}, upload=484797]: Done begin upload segment. Jul 18 15:33:00.640649191 (606h22m0.524062829s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.640953078 (606h22m0.524366715s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641531127 (606h22m0.524944758s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641980147 (606h22m0.52539378s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642678070 (606h22m0.526091699s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642961859 (606h22m0.526375491s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643490795 (606h22m0.526904425s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643733424 (606h22m0.52714705s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644374317 (606h22m0.527787948s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644585121 (606h22m0.527998748s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644900409 (606h22m0.528314035s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646038897 (606h22m0.529452526s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646375740 (606h22m0.529789369s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.648944423 (606h22m0.53235805s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.650463350 (606h22m0.533876978s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.651369647 (606h22m0.534783267s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:01.262589342 (606h22m1.146002974s) [upload=484797]: Next returned nil segment Jul 18 15:33:01.262648256 (606h22m1.146061891s) [upload=484797]: Waiting for error group managing segments... Jul 18 15:33:01.398379699 (606h22m1.281793326s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404946640 (606h22m1.288360283s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404992987 (606h22m1.28840661s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.405551748 (606h22m1.288965369s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.407987418 (606h22m1.291401032s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.410553731 (606h22m1.293967362s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.416771944 (606h22m1.300185579s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.425807204 (606h22m1.309220819s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.430635732 (606h22m1.314049361s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.433605056 (606h22m1.317018687s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.440793839 (606h22m1.32420747s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.446368593 (606h22m1.329782225s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:30.258196584 (606h22m30.141610202s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258423161 (606h22m30.141836789s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:40968->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258491071 (606h22m30.1419047s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258527956 (606h22m30.141941579s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258553948 (606h22m30.141967578s) [seg_pos={0 0}, upload=484797]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 15:33:30.369475355 (606h22m30.252888992s) [upload=484797]: Done uploading segments... Jul 18 15:33:30.369509072 (606h22m30.252922716s) [upload=484797]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.43593245s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 22 out of 60 is failed (threshold=3) [......................................XXXXXXXXXXXXXXXXXXXXXX] duration: 176.975724ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 16:05:58 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 177.012142ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 21 out of 60 is failed (threshold=3) [.......................................XXXXXXXXXXXXXXXXXXXXX] duration: 175.023679ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 16:05:40 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 175.052657ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............XX..X.........................................] duration: 2.418700544s FAIL (2025-07-18 16:05:24 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== BUMKAHO3ZMVDNWVSRRI3GTLM4MO3MP6MNNGUUZKBLBWQ, seg_pos={0 0}, upload=484783]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: rpc: dial tcp 77.90.31.13:4628: operation was canceled; context canceled Jul 18 15:31:52.267100372 (606h20m52.150513985s) [seg_pos={0 0}, upload=484783]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:31:52.267178520 (606h20m52.150592177s) [upload=484783]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:31:52.267184778 (606h20m52.150598408s) [upload=484783]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:31:52.565015623 (606h20m52.448429239s) [upload=484783]: Done uploading segments... Jul 18 15:31:52.565033307 (606h20m52.448446919s) [upload=484783]: Upload finished. err: <nil> Jul 18 15:33:00.528847457 (606h22m0.412261094s) [upload=484797]: Starting upload Jul 18 15:33:00.528934444 (606h22m0.412348056s) [upload=484797]: Done starting upload Jul 18 15:33:00.529017539 (606h22m0.412431168s) [upload=484797]: Uploading segments... Jul 18 15:33:00.529303798 (606h22m0.412717427s) [upload=484797]: Got next segment. Inline: false Jul 18 15:33:00.529348408 (606h22m0.41276202s) [upload=484797]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:33:00.529353973 (606h22m0.412767599s) [upload=484797]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:33:00.640500227 (606h22m0.523913861s) [seg_pos={0 0}, upload=484797]: Begin upload segment... Jul 18 15:33:00.640588551 (606h22m0.524002189s) [seg_pos={0 0}, upload=484797]: Done begin upload segment. Jul 18 15:33:00.640649191 (606h22m0.524062829s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.640953078 (606h22m0.524366715s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641531127 (606h22m0.524944758s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641980147 (606h22m0.52539378s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642678070 (606h22m0.526091699s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642961859 (606h22m0.526375491s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643490795 (606h22m0.526904425s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643733424 (606h22m0.52714705s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644374317 (606h22m0.527787948s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644585121 (606h22m0.527998748s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644900409 (606h22m0.528314035s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646038897 (606h22m0.529452526s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646375740 (606h22m0.529789369s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.648944423 (606h22m0.53235805s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.650463350 (606h22m0.533876978s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.651369647 (606h22m0.534783267s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:01.262589342 (606h22m1.146002974s) [upload=484797]: Next returned nil segment Jul 18 15:33:01.262648256 (606h22m1.146061891s) [upload=484797]: Waiting for error group managing segments... Jul 18 15:33:01.398379699 (606h22m1.281793326s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404946640 (606h22m1.288360283s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404992987 (606h22m1.28840661s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.405551748 (606h22m1.288965369s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.407987418 (606h22m1.291401032s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.410553731 (606h22m1.293967362s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.416771944 (606h22m1.300185579s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.425807204 (606h22m1.309220819s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.430635732 (606h22m1.314049361s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.433605056 (606h22m1.317018687s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.440793839 (606h22m1.32420747s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.446368593 (606h22m1.329782225s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:30.258196584 (606h22m30.141610202s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258423161 (606h22m30.141836789s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:40968->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258491071 (606h22m30.1419047s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258527956 (606h22m30.141941579s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258553948 (606h22m30.141967578s) [seg_pos={0 0}, upload=484797]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 15:33:30.369475355 (606h22m30.252888992s) [upload=484797]: Done uploading segments... Jul 18 15:33:30.369509072 (606h22m30.252922716s) [upload=484797]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.41873594s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 20 out of 60 is failed (threshold=3) [........................................XXXXXXXXXXXXXXXXXXXX] duration: 179.213264ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 16:05:23 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 179.242789ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............XX..X........................................] duration: 2.437682352s FAIL (2025-07-18 16:05:02 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== BUMKAHO3ZMVDNWVSRRI3GTLM4MO3MP6MNNGUUZKBLBWQ, seg_pos={0 0}, upload=484783]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: rpc: dial tcp 77.90.31.13:4628: operation was canceled; context canceled Jul 18 15:31:52.267100372 (606h20m52.150513985s) [seg_pos={0 0}, upload=484783]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:31:52.267178520 (606h20m52.150592177s) [upload=484783]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:31:52.267184778 (606h20m52.150598408s) [upload=484783]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:31:52.565015623 (606h20m52.448429239s) [upload=484783]: Done uploading segments... Jul 18 15:31:52.565033307 (606h20m52.448446919s) [upload=484783]: Upload finished. err: <nil> Jul 18 15:33:00.528847457 (606h22m0.412261094s) [upload=484797]: Starting upload Jul 18 15:33:00.528934444 (606h22m0.412348056s) [upload=484797]: Done starting upload Jul 18 15:33:00.529017539 (606h22m0.412431168s) [upload=484797]: Uploading segments... Jul 18 15:33:00.529303798 (606h22m0.412717427s) [upload=484797]: Got next segment. Inline: false Jul 18 15:33:00.529348408 (606h22m0.41276202s) [upload=484797]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:33:00.529353973 (606h22m0.412767599s) [upload=484797]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:33:00.640500227 (606h22m0.523913861s) [seg_pos={0 0}, upload=484797]: Begin upload segment... Jul 18 15:33:00.640588551 (606h22m0.524002189s) [seg_pos={0 0}, upload=484797]: Done begin upload segment. Jul 18 15:33:00.640649191 (606h22m0.524062829s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.640953078 (606h22m0.524366715s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641531127 (606h22m0.524944758s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641980147 (606h22m0.52539378s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642678070 (606h22m0.526091699s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642961859 (606h22m0.526375491s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643490795 (606h22m0.526904425s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643733424 (606h22m0.52714705s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644374317 (606h22m0.527787948s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644585121 (606h22m0.527998748s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644900409 (606h22m0.528314035s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646038897 (606h22m0.529452526s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646375740 (606h22m0.529789369s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.648944423 (606h22m0.53235805s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.650463350 (606h22m0.533876978s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.651369647 (606h22m0.534783267s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:01.262589342 (606h22m1.146002974s) [upload=484797]: Next returned nil segment Jul 18 15:33:01.262648256 (606h22m1.146061891s) [upload=484797]: Waiting for error group managing segments... Jul 18 15:33:01.398379699 (606h22m1.281793326s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404946640 (606h22m1.288360283s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404992987 (606h22m1.28840661s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.405551748 (606h22m1.288965369s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.407987418 (606h22m1.291401032s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.410553731 (606h22m1.293967362s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.416771944 (606h22m1.300185579s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.425807204 (606h22m1.309220819s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.430635732 (606h22m1.314049361s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.433605056 (606h22m1.317018687s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.440793839 (606h22m1.32420747s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.446368593 (606h22m1.329782225s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:30.258196584 (606h22m30.141610202s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258423161 (606h22m30.141836789s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:40968->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258491071 (606h22m30.1419047s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258527956 (606h22m30.141941579s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258553948 (606h22m30.141967578s) [seg_pos={0 0}, upload=484797]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 15:33:30.369475355 (606h22m30.252888992s) [upload=484797]: Done uploading segments... Jul 18 15:33:30.369509072 (606h22m30.252922716s) [upload=484797]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.437721193s core uplink us-select-1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 21 out of 60 is failed (threshold=3) [.......................................XXXXXXXXXXXXXXXXXXXXX] duration: 179.628593ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 16:05:00 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 179.651735ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 19 out of 60 is failed (threshold=3) [.........................................XXXXXXXXXXXXXXXXXXX] duration: 170.610716ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 16:04:39 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 170.637139ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 20 out of 60 is failed (threshold=3) [........................................XXXXXXXXXXXXXXXXXXXX] duration: 173.009002ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 16:04:37 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 173.042647ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 19 out of 60 is failed (threshold=3) [.........................................XXXXXXXXXXXXXXXXXXX] duration: 179.148831ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 16:04:00 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 179.260435ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [................XX..X.......................................] duration: 2.380075535s FAIL (2025-07-18 16:03:57 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== BUMKAHO3ZMVDNWVSRRI3GTLM4MO3MP6MNNGUUZKBLBWQ, seg_pos={0 0}, upload=484783]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: rpc: dial tcp 77.90.31.13:4628: operation was canceled; context canceled Jul 18 15:31:52.267100372 (606h20m52.150513985s) [seg_pos={0 0}, upload=484783]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:31:52.267178520 (606h20m52.150592177s) [upload=484783]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:31:52.267184778 (606h20m52.150598408s) [upload=484783]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:31:52.565015623 (606h20m52.448429239s) [upload=484783]: Done uploading segments... Jul 18 15:31:52.565033307 (606h20m52.448446919s) [upload=484783]: Upload finished. err: <nil> Jul 18 15:33:00.528847457 (606h22m0.412261094s) [upload=484797]: Starting upload Jul 18 15:33:00.528934444 (606h22m0.412348056s) [upload=484797]: Done starting upload Jul 18 15:33:00.529017539 (606h22m0.412431168s) [upload=484797]: Uploading segments... Jul 18 15:33:00.529303798 (606h22m0.412717427s) [upload=484797]: Got next segment. Inline: false Jul 18 15:33:00.529348408 (606h22m0.41276202s) [upload=484797]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:33:00.529353973 (606h22m0.412767599s) [upload=484797]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:33:00.640500227 (606h22m0.523913861s) [seg_pos={0 0}, upload=484797]: Begin upload segment... Jul 18 15:33:00.640588551 (606h22m0.524002189s) [seg_pos={0 0}, upload=484797]: Done begin upload segment. Jul 18 15:33:00.640649191 (606h22m0.524062829s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.640953078 (606h22m0.524366715s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641531127 (606h22m0.524944758s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641980147 (606h22m0.52539378s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642678070 (606h22m0.526091699s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642961859 (606h22m0.526375491s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643490795 (606h22m0.526904425s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643733424 (606h22m0.52714705s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644374317 (606h22m0.527787948s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644585121 (606h22m0.527998748s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644900409 (606h22m0.528314035s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646038897 (606h22m0.529452526s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646375740 (606h22m0.529789369s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.648944423 (606h22m0.53235805s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.650463350 (606h22m0.533876978s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.651369647 (606h22m0.534783267s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:01.262589342 (606h22m1.146002974s) [upload=484797]: Next returned nil segment Jul 18 15:33:01.262648256 (606h22m1.146061891s) [upload=484797]: Waiting for error group managing segments... Jul 18 15:33:01.398379699 (606h22m1.281793326s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404946640 (606h22m1.288360283s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404992987 (606h22m1.28840661s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.405551748 (606h22m1.288965369s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.407987418 (606h22m1.291401032s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.410553731 (606h22m1.293967362s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.416771944 (606h22m1.300185579s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.425807204 (606h22m1.309220819s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.430635732 (606h22m1.314049361s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.433605056 (606h22m1.317018687s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.440793839 (606h22m1.32420747s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.446368593 (606h22m1.329782225s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:30.258196584 (606h22m30.141610202s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258423161 (606h22m30.141836789s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:40968->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258491071 (606h22m30.1419047s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258527956 (606h22m30.141941579s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258553948 (606h22m30.141967578s) [seg_pos={0 0}, upload=484797]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 15:33:30.369475355 (606h22m30.252888992s) [upload=484797]: Done uploading segments... Jul 18 15:33:30.369509072 (606h22m30.252922716s) [upload=484797]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.380115727s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 18 out of 60 is failed (threshold=3) [..........................................XXXXXXXXXXXXXXXXXX] duration: 177.081871ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 16:03:52 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 177.108496ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 18 out of 60 is failed (threshold=3) [..........................................XXXXXXXXXXXXXXXXXX] duration: 171.076663ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 16:03:31 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 171.104584ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 17 out of 60 is failed (threshold=3) [...........................................XXXXXXXXXXXXXXXXX] duration: 207.955345ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 16:03:13 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 207.981351ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................XX..X......................................] duration: 2.379701589s FAIL (2025-07-18 16:02:47 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== BUMKAHO3ZMVDNWVSRRI3GTLM4MO3MP6MNNGUUZKBLBWQ, seg_pos={0 0}, upload=484783]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: rpc: dial tcp 77.90.31.13:4628: operation was canceled; context canceled Jul 18 15:31:52.267100372 (606h20m52.150513985s) [seg_pos={0 0}, upload=484783]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:31:52.267178520 (606h20m52.150592177s) [upload=484783]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:31:52.267184778 (606h20m52.150598408s) [upload=484783]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:31:52.565015623 (606h20m52.448429239s) [upload=484783]: Done uploading segments... Jul 18 15:31:52.565033307 (606h20m52.448446919s) [upload=484783]: Upload finished. err: <nil> Jul 18 15:33:00.528847457 (606h22m0.412261094s) [upload=484797]: Starting upload Jul 18 15:33:00.528934444 (606h22m0.412348056s) [upload=484797]: Done starting upload Jul 18 15:33:00.529017539 (606h22m0.412431168s) [upload=484797]: Uploading segments... Jul 18 15:33:00.529303798 (606h22m0.412717427s) [upload=484797]: Got next segment. Inline: false Jul 18 15:33:00.529348408 (606h22m0.41276202s) [upload=484797]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:33:00.529353973 (606h22m0.412767599s) [upload=484797]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:33:00.640500227 (606h22m0.523913861s) [seg_pos={0 0}, upload=484797]: Begin upload segment... Jul 18 15:33:00.640588551 (606h22m0.524002189s) [seg_pos={0 0}, upload=484797]: Done begin upload segment. Jul 18 15:33:00.640649191 (606h22m0.524062829s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.640953078 (606h22m0.524366715s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641531127 (606h22m0.524944758s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641980147 (606h22m0.52539378s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642678070 (606h22m0.526091699s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642961859 (606h22m0.526375491s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643490795 (606h22m0.526904425s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643733424 (606h22m0.52714705s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644374317 (606h22m0.527787948s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644585121 (606h22m0.527998748s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644900409 (606h22m0.528314035s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646038897 (606h22m0.529452526s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646375740 (606h22m0.529789369s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.648944423 (606h22m0.53235805s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.650463350 (606h22m0.533876978s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.651369647 (606h22m0.534783267s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:01.262589342 (606h22m1.146002974s) [upload=484797]: Next returned nil segment Jul 18 15:33:01.262648256 (606h22m1.146061891s) [upload=484797]: Waiting for error group managing segments... Jul 18 15:33:01.398379699 (606h22m1.281793326s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404946640 (606h22m1.288360283s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404992987 (606h22m1.28840661s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.405551748 (606h22m1.288965369s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.407987418 (606h22m1.291401032s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.410553731 (606h22m1.293967362s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.416771944 (606h22m1.300185579s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.425807204 (606h22m1.309220819s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.430635732 (606h22m1.314049361s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.433605056 (606h22m1.317018687s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.440793839 (606h22m1.32420747s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.446368593 (606h22m1.329782225s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:30.258196584 (606h22m30.141610202s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258423161 (606h22m30.141836789s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:40968->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258491071 (606h22m30.1419047s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258527956 (606h22m30.141941579s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258553948 (606h22m30.141967578s) [seg_pos={0 0}, upload=484797]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 15:33:30.369475355 (606h22m30.252888992s) [upload=484797]: Done uploading segments... Jul 18 15:33:30.369509072 (606h22m30.252922716s) [upload=484797]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.37973429s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 17 out of 60 is failed (threshold=3) [...........................................XXXXXXXXXXXXXXXXX] duration: 174.636977ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 16:02:39 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 174.669137ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 16 out of 60 is failed (threshold=3) [............................................XXXXXXXXXXXXXXXX] duration: 176.170728ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 16:02:36 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 176.205411ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 16 out of 60 is failed (threshold=3) [............................................XXXXXXXXXXXXXXXX] duration: 172.081941ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 16:02:06 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 172.109621ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 15 out of 60 is failed (threshold=3) [.............................................XXXXXXXXXXXXXXX] duration: 179.639645ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 16:01:47 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 179.664235ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................XX..X.....................................] duration: 2.679154608s FAIL (2025-07-18 16:01:38 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== BUMKAHO3ZMVDNWVSRRI3GTLM4MO3MP6MNNGUUZKBLBWQ, seg_pos={0 0}, upload=484783]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: rpc: dial tcp 77.90.31.13:4628: operation was canceled; context canceled Jul 18 15:31:52.267100372 (606h20m52.150513985s) [seg_pos={0 0}, upload=484783]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:31:52.267178520 (606h20m52.150592177s) [upload=484783]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:31:52.267184778 (606h20m52.150598408s) [upload=484783]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:31:52.565015623 (606h20m52.448429239s) [upload=484783]: Done uploading segments... Jul 18 15:31:52.565033307 (606h20m52.448446919s) [upload=484783]: Upload finished. err: <nil> Jul 18 15:33:00.528847457 (606h22m0.412261094s) [upload=484797]: Starting upload Jul 18 15:33:00.528934444 (606h22m0.412348056s) [upload=484797]: Done starting upload Jul 18 15:33:00.529017539 (606h22m0.412431168s) [upload=484797]: Uploading segments... Jul 18 15:33:00.529303798 (606h22m0.412717427s) [upload=484797]: Got next segment. Inline: false Jul 18 15:33:00.529348408 (606h22m0.41276202s) [upload=484797]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:33:00.529353973 (606h22m0.412767599s) [upload=484797]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:33:00.640500227 (606h22m0.523913861s) [seg_pos={0 0}, upload=484797]: Begin upload segment... Jul 18 15:33:00.640588551 (606h22m0.524002189s) [seg_pos={0 0}, upload=484797]: Done begin upload segment. Jul 18 15:33:00.640649191 (606h22m0.524062829s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.640953078 (606h22m0.524366715s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641531127 (606h22m0.524944758s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641980147 (606h22m0.52539378s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642678070 (606h22m0.526091699s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642961859 (606h22m0.526375491s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643490795 (606h22m0.526904425s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643733424 (606h22m0.52714705s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644374317 (606h22m0.527787948s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644585121 (606h22m0.527998748s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644900409 (606h22m0.528314035s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646038897 (606h22m0.529452526s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646375740 (606h22m0.529789369s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.648944423 (606h22m0.53235805s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.650463350 (606h22m0.533876978s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.651369647 (606h22m0.534783267s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:01.262589342 (606h22m1.146002974s) [upload=484797]: Next returned nil segment Jul 18 15:33:01.262648256 (606h22m1.146061891s) [upload=484797]: Waiting for error group managing segments... Jul 18 15:33:01.398379699 (606h22m1.281793326s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404946640 (606h22m1.288360283s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404992987 (606h22m1.28840661s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.405551748 (606h22m1.288965369s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.407987418 (606h22m1.291401032s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.410553731 (606h22m1.293967362s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.416771944 (606h22m1.300185579s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.425807204 (606h22m1.309220819s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.430635732 (606h22m1.314049361s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.433605056 (606h22m1.317018687s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.440793839 (606h22m1.32420747s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.446368593 (606h22m1.329782225s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:30.258196584 (606h22m30.141610202s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258423161 (606h22m30.141836789s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:40968->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258491071 (606h22m30.1419047s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258527956 (606h22m30.141941579s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258553948 (606h22m30.141967578s) [seg_pos={0 0}, upload=484797]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 15:33:30.369475355 (606h22m30.252888992s) [upload=484797]: Done uploading segments... Jul 18 15:33:30.369509072 (606h22m30.252922716s) [upload=484797]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.679181403s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 15 out of 60 is failed (threshold=3) [.............................................XXXXXXXXXXXXXXX] duration: 172.564416ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 16:01:29 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 172.595695ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 14 out of 60 is failed (threshold=3) [..............................................XXXXXXXXXXXXXX] duration: 173.952385ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 16:01:21 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 173.980389ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 13 out of 60 is failed (threshold=3) [...............................................XXXXXXXXXXXXX] duration: 172.752381ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 16:01:06 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 172.780394ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................XX..X....................................] duration: 2.372705535s FAIL (2025-07-18 16:00:24 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== BUMKAHO3ZMVDNWVSRRI3GTLM4MO3MP6MNNGUUZKBLBWQ, seg_pos={0 0}, upload=484783]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: rpc: dial tcp 77.90.31.13:4628: operation was canceled; context canceled Jul 18 15:31:52.267100372 (606h20m52.150513985s) [seg_pos={0 0}, upload=484783]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:31:52.267178520 (606h20m52.150592177s) [upload=484783]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:31:52.267184778 (606h20m52.150598408s) [upload=484783]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:31:52.565015623 (606h20m52.448429239s) [upload=484783]: Done uploading segments... Jul 18 15:31:52.565033307 (606h20m52.448446919s) [upload=484783]: Upload finished. err: <nil> Jul 18 15:33:00.528847457 (606h22m0.412261094s) [upload=484797]: Starting upload Jul 18 15:33:00.528934444 (606h22m0.412348056s) [upload=484797]: Done starting upload Jul 18 15:33:00.529017539 (606h22m0.412431168s) [upload=484797]: Uploading segments... Jul 18 15:33:00.529303798 (606h22m0.412717427s) [upload=484797]: Got next segment. Inline: false Jul 18 15:33:00.529348408 (606h22m0.41276202s) [upload=484797]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:33:00.529353973 (606h22m0.412767599s) [upload=484797]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:33:00.640500227 (606h22m0.523913861s) [seg_pos={0 0}, upload=484797]: Begin upload segment... Jul 18 15:33:00.640588551 (606h22m0.524002189s) [seg_pos={0 0}, upload=484797]: Done begin upload segment. Jul 18 15:33:00.640649191 (606h22m0.524062829s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.640953078 (606h22m0.524366715s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641531127 (606h22m0.524944758s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641980147 (606h22m0.52539378s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642678070 (606h22m0.526091699s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642961859 (606h22m0.526375491s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643490795 (606h22m0.526904425s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643733424 (606h22m0.52714705s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644374317 (606h22m0.527787948s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644585121 (606h22m0.527998748s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644900409 (606h22m0.528314035s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646038897 (606h22m0.529452526s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646375740 (606h22m0.529789369s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.648944423 (606h22m0.53235805s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.650463350 (606h22m0.533876978s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.651369647 (606h22m0.534783267s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:01.262589342 (606h22m1.146002974s) [upload=484797]: Next returned nil segment Jul 18 15:33:01.262648256 (606h22m1.146061891s) [upload=484797]: Waiting for error group managing segments... Jul 18 15:33:01.398379699 (606h22m1.281793326s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404946640 (606h22m1.288360283s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404992987 (606h22m1.28840661s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.405551748 (606h22m1.288965369s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.407987418 (606h22m1.291401032s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.410553731 (606h22m1.293967362s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.416771944 (606h22m1.300185579s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.425807204 (606h22m1.309220819s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.430635732 (606h22m1.314049361s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.433605056 (606h22m1.317018687s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.440793839 (606h22m1.32420747s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.446368593 (606h22m1.329782225s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:30.258196584 (606h22m30.141610202s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258423161 (606h22m30.141836789s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:40968->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258491071 (606h22m30.1419047s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258527956 (606h22m30.141941579s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258553948 (606h22m30.141967578s) [seg_pos={0 0}, upload=484797]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 15:33:30.369475355 (606h22m30.252888992s) [upload=484797]: Done uploading segments... Jul 18 15:33:30.369509072 (606h22m30.252922716s) [upload=484797]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.37273788s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 14 out of 60 is failed (threshold=3) [..............................................XXXXXXXXXXXXXX] duration: 172.393517ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 16:00:16 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 172.424327ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 12 out of 60 is failed (threshold=3) [................................................XXXXXXXXXXXX] duration: 174.263428ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 15:59:54 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 174.297611ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....................XX..X...................................] duration: 2.347585718s FAIL (2025-07-18 15:59:39 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== BUMKAHO3ZMVDNWVSRRI3GTLM4MO3MP6MNNGUUZKBLBWQ, seg_pos={0 0}, upload=484783]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: rpc: dial tcp 77.90.31.13:4628: operation was canceled; context canceled Jul 18 15:31:52.267100372 (606h20m52.150513985s) [seg_pos={0 0}, upload=484783]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:31:52.267178520 (606h20m52.150592177s) [upload=484783]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:31:52.267184778 (606h20m52.150598408s) [upload=484783]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:31:52.565015623 (606h20m52.448429239s) [upload=484783]: Done uploading segments... Jul 18 15:31:52.565033307 (606h20m52.448446919s) [upload=484783]: Upload finished. err: <nil> Jul 18 15:33:00.528847457 (606h22m0.412261094s) [upload=484797]: Starting upload Jul 18 15:33:00.528934444 (606h22m0.412348056s) [upload=484797]: Done starting upload Jul 18 15:33:00.529017539 (606h22m0.412431168s) [upload=484797]: Uploading segments... Jul 18 15:33:00.529303798 (606h22m0.412717427s) [upload=484797]: Got next segment. Inline: false Jul 18 15:33:00.529348408 (606h22m0.41276202s) [upload=484797]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:33:00.529353973 (606h22m0.412767599s) [upload=484797]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:33:00.640500227 (606h22m0.523913861s) [seg_pos={0 0}, upload=484797]: Begin upload segment... Jul 18 15:33:00.640588551 (606h22m0.524002189s) [seg_pos={0 0}, upload=484797]: Done begin upload segment. Jul 18 15:33:00.640649191 (606h22m0.524062829s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.640953078 (606h22m0.524366715s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641531127 (606h22m0.524944758s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641980147 (606h22m0.52539378s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642678070 (606h22m0.526091699s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642961859 (606h22m0.526375491s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643490795 (606h22m0.526904425s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643733424 (606h22m0.52714705s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644374317 (606h22m0.527787948s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644585121 (606h22m0.527998748s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644900409 (606h22m0.528314035s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646038897 (606h22m0.529452526s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646375740 (606h22m0.529789369s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.648944423 (606h22m0.53235805s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.650463350 (606h22m0.533876978s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.651369647 (606h22m0.534783267s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:01.262589342 (606h22m1.146002974s) [upload=484797]: Next returned nil segment Jul 18 15:33:01.262648256 (606h22m1.146061891s) [upload=484797]: Waiting for error group managing segments... Jul 18 15:33:01.398379699 (606h22m1.281793326s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404946640 (606h22m1.288360283s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404992987 (606h22m1.28840661s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.405551748 (606h22m1.288965369s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.407987418 (606h22m1.291401032s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.410553731 (606h22m1.293967362s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.416771944 (606h22m1.300185579s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.425807204 (606h22m1.309220819s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.430635732 (606h22m1.314049361s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.433605056 (606h22m1.317018687s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.440793839 (606h22m1.32420747s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.446368593 (606h22m1.329782225s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:30.258196584 (606h22m30.141610202s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258423161 (606h22m30.141836789s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:40968->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258491071 (606h22m30.1419047s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258527956 (606h22m30.141941579s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258553948 (606h22m30.141967578s) [seg_pos={0 0}, upload=484797]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 15:33:30.369475355 (606h22m30.252888992s) [upload=484797]: Done uploading segments... Jul 18 15:33:30.369509072 (606h22m30.252922716s) [upload=484797]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.347612861s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 13 out of 60 is failed (threshold=3) [...............................................XXXXXXXXXXXXX] duration: 179.211644ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 15:59:11 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 179.235977ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 11 out of 60 is failed (threshold=3) [.................................................XXXXXXXXXXX] duration: 173.850405ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 15:58:46 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 173.874749ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 12 out of 60 is failed (threshold=3) [................................................XXXXXXXXXXXX] duration: 177.552117ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 15:58:39 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 177.577126ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....................XX..X..................................] duration: 2.431572009s FAIL (2025-07-18 15:58:35 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== BUMKAHO3ZMVDNWVSRRI3GTLM4MO3MP6MNNGUUZKBLBWQ, seg_pos={0 0}, upload=484783]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: rpc: dial tcp 77.90.31.13:4628: operation was canceled; context canceled Jul 18 15:31:52.267100372 (606h20m52.150513985s) [seg_pos={0 0}, upload=484783]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 15:31:52.267178520 (606h20m52.150592177s) [upload=484783]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 15:31:52.267184778 (606h20m52.150598408s) [upload=484783]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 15:31:52.565015623 (606h20m52.448429239s) [upload=484783]: Done uploading segments... Jul 18 15:31:52.565033307 (606h20m52.448446919s) [upload=484783]: Upload finished. err: <nil> Jul 18 15:33:00.528847457 (606h22m0.412261094s) [upload=484797]: Starting upload Jul 18 15:33:00.528934444 (606h22m0.412348056s) [upload=484797]: Done starting upload Jul 18 15:33:00.529017539 (606h22m0.412431168s) [upload=484797]: Uploading segments... Jul 18 15:33:00.529303798 (606h22m0.412717427s) [upload=484797]: Got next segment. Inline: false Jul 18 15:33:00.529348408 (606h22m0.41276202s) [upload=484797]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 15:33:00.529353973 (606h22m0.412767599s) [upload=484797]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 15:33:00.640500227 (606h22m0.523913861s) [seg_pos={0 0}, upload=484797]: Begin upload segment... Jul 18 15:33:00.640588551 (606h22m0.524002189s) [seg_pos={0 0}, upload=484797]: Done begin upload segment. Jul 18 15:33:00.640649191 (606h22m0.524062829s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.640953078 (606h22m0.524366715s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641531127 (606h22m0.524944758s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.641980147 (606h22m0.52539378s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642678070 (606h22m0.526091699s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.642961859 (606h22m0.526375491s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643490795 (606h22m0.526904425s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.643733424 (606h22m0.52714705s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644374317 (606h22m0.527787948s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644585121 (606h22m0.527998748s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.644900409 (606h22m0.528314035s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646038897 (606h22m0.529452526s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.646375740 (606h22m0.529789369s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.648944423 (606h22m0.53235805s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.650463350 (606h22m0.533876978s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:00.651369647 (606h22m0.534783267s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Uploading piece... Jul 18 15:33:01.262589342 (606h22m1.146002974s) [upload=484797]: Next returned nil segment Jul 18 15:33:01.262648256 (606h22m1.146061891s) [upload=484797]: Waiting for error group managing segments... Jul 18 15:33:01.398379699 (606h22m1.281793326s) [noise=true , address=77.90.31.14:4661 , piece_id=O3MO7X5TB4UAPQBWVBNPVSI27TUM6VCURHLJG5HEYO4DVVYACIUA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404946640 (606h22m1.288360283s) [noise=true , address=77.90.31.15:4698 , piece_id=2SIXSZXA6GTBEZPTHMVXS6YEXADFV3OH7PCREDLSUEG24POB55NQ, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.404992987 (606h22m1.28840661s) [noise=true , address=45.91.116.4:4939 , piece_id=HJ67GVSKZTLLNZKKNS6INR63O7TQE53H6J57BX5H4GBJQV4NO27Q, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.405551748 (606h22m1.288965369s) [noise=true , address=77.90.31.14:4662 , piece_id=676S3KDOVDPLEXZT5VQUTFLUCESA66V7GK4UQ4DFKIA6MGWMA2EA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.407987418 (606h22m1.291401032s) [noise=true , address=45.91.116.10:4083 , piece_id=GPIYDMSSIX2QCPORQMIJOS7J7RBX2QOZ5HEAUZ5MJ427NLB4XENA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.410553731 (606h22m1.293967362s) [noise=true , address=45.91.116.13:4154 , piece_id=IILXTHASZ3NMX2UVSZ6CXZGSU5WCRNSZVHSAKAANAGKMPK5FPVTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.416771944 (606h22m1.300185579s) [noise=true , address=104.36.82.12:4150 , piece_id=VSQZLUMKIPHBGB43MSZLTAIFXMOIBY6LKJCHFCD437GAGCNXC7OA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.425807204 (606h22m1.309220819s) [noise=true , address=104.36.82.5:4838 , piece_id=FWYXJ44GBR42MQ2EEHOMU5SQD3WDYWD46EILRDAOKA5C6WBYGU5A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.430635732 (606h22m1.314049361s) [noise=true , address=104.36.82.10:4077 , piece_id=Q3YUPZ6EHSM3RQ5MLR6EV33SJEFCCLXJBRIG7BW73K5LCEIS7CTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.433605056 (606h22m1.317018687s) [noise=true , address=104.36.82.14:4221 , piece_id=TMLB47IV22NKLLQNUWT27NG3G5P7IIU3N77AN65UNSYHPFVKWP2A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.440793839 (606h22m1.32420747s) [noise=true , address=45.91.116.12:4227 , piece_id=CJFSTMTP5S2CFJVL3LGHTM6O6CEXSJSE5TQX5EBFZMEZ3VBYMV3A, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:01.446368593 (606h22m1.329782225s) [noise=true , address=45.91.116.7:4009 , piece_id=HTL7WBDRUW7WUB5G64LYFKYSC4ZUX5IJIZK225O5MOE6ZMSH5LYA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: <nil> Jul 18 15:33:30.258196584 (606h22m30.141610202s) [noise=true , address=77.90.31.19:4841 , piece_id=7HWGFOURSOC6JLKAKVYVRJEYJW3WH6FM6NGK5TSERO4ERP5NIHPA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258423161 (606h22m30.141836789s) [noise=true , address=77.90.31.19:4842 , piece_id=PFMYCQ5QFJ4YSGYY2QVD56MKIPOJG6VW7EO3QHLKTRB2QJZ7EUTA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:40968->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258491071 (606h22m30.1419047s) [noise=true , address=77.90.31.21:4879 , piece_id=ZQMN2UKPJ7D4PJKBVWANUUAHKGTFH4QF5RZZPPWZHDY6MDS3WDFA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258527956 (606h22m30.141941579s) [noise=true , address=77.90.31.10:4519 , piece_id=Y6IRD6PE6O4IMHBLBGVVEY2H36M7DQ3RL2WCDIUMJRH2WFTCCPZA, seg_pos={0 0}, upload=484797]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 15:33:30.258553948 (606h22m30.141967578s) [seg_pos={0 0}, upload=484797]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 15:33:30.369475355 (606h22m30.252888992s) [upload=484797]: Done uploading segments... Jul 18 15:33:30.369509072 (606h22m30.252922716s) [upload=484797]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.431599532s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)