Orbiter History

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 178.535099ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 17:10:49 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 178.568309ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 167.70911ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 17:10:43 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 167.736139ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 2.408488838s FAIL (2025-07-18 17:10:35 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 9]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: context canceled; context canceled Jul 18 16:28:06.573617088 (607h17m6.457030715s) [seg_pos={0 0}, upload=485529]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 16:28:06.573709829 (607h17m6.457123445s) [upload=485529]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 16:28:06.573716041 (607h17m6.457129658s) [upload=485529]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 16:28:06.724230331 (607h17m6.607643945s) [upload=485529]: Done uploading segments... Jul 18 16:28:06.724250141 (607h17m6.607663755s) [upload=485529]: Upload finished. err: <nil> Jul 18 16:29:19.727355583 (607h18m19.610769209s) [upload=485547]: Starting upload Jul 18 16:29:19.727427333 (607h18m19.61084097s) [upload=485547]: Done starting upload Jul 18 16:29:19.727775544 (607h18m19.611189161s) [upload=485547]: Uploading segments... Jul 18 16:29:19.727840340 (607h18m19.611253975s) [upload=485547]: Got next segment. Inline: false Jul 18 16:29:19.727875280 (607h18m19.611288906s) [upload=485547]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 16:29:19.727880962 (607h18m19.611294582s) [upload=485547]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 16:29:19.840919734 (607h18m19.724333363s) [seg_pos={0 0}, upload=485547]: Begin upload segment... Jul 18 16:29:19.840972492 (607h18m19.72438611s) [seg_pos={0 0}, upload=485547]: Done begin upload segment. Jul 18 16:29:19.841004794 (607h18m19.724418434s) [noise=true , address=77.90.31.19:4841 , piece_id=IERSGG2GKGB5UD3QH3ETCKSJZJT5A6ELFHDLFLJFJ6VB3Y6QIFHA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841022981 (607h18m19.724436614s) [noise=true , address=77.90.31.11:4553 , piece_id=KYYATSWOYX5R7SKM67PJBA2QZC7PRRNJJ7GKPYLZBOTOZ2J6WFYQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841446916 (607h18m19.724860553s) [noise=true , address=45.91.116.9:4262 , piece_id=5MJOYGUIGLNE24MZCQ2PLX75TT4X4XSSVVVBN3WJH352MT5W64GQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841470680 (607h18m19.724884316s) [noise=true , address=104.36.82.12:4151 , piece_id=4I5GEGRSL7PVJE55J5EC3MPDL2ZII74ZFO6KFOCBUZ4WS7KU3F5A, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841825001 (607h18m19.725238633s) [noise=true , address=104.36.82.14:4222 , piece_id=2TAGHXI3NADLHJQXCRIRY5CM6VMI2BHRPT7C6S7Y7TSPC2MGTXSQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841863759 (607h18m19.725277377s) [noise=true , address=104.36.82.7:4911 , piece_id=INO4Q54WEULMMEAEG73QKVN4A6LLN4Z7CRJZTSLRBUDZ4NR4YHZA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842376953 (607h18m19.725790586s) [noise=true , address=45.91.116.14:4189 , piece_id=EDZFMKYC63ABE7V5AE6E4FELMGIWPPPANRNSOFN3GAYE6X6UNUMQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842942175 (607h18m19.726355788s) [noise=true , address=104.36.82.5:4839 , piece_id=NWXJQEAAM2TZKJPIJMRB73K6OCI5A4SLHKY74XLPWXFFMW4RO3CQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842951799 (607h18m19.726365432s) [noise=true , address=77.90.31.14:4661 , piece_id=SU2WXUKNOBMJ5HBPOVU44MD2T4PPMUDHTUSHFBQR6PXQNV5KCCIQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.843362766 (607h18m19.726776393s) [noise=true , address=104.36.82.14:4223 , piece_id=34IMOFVRIMVWGSZ2LJE5PYORVNQD4H3BQQNL5I2QTHD6LNB2MEPQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.844357136 (607h18m19.727770773s) [noise=true , address=45.91.116.10:4081 , piece_id=ZP3GV7W7B2L2M45V4Y7OOWX6Z4Q4MI5PEX2MHMMY7D4L2PJWR3SQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.845591955 (607h18m19.729005589s) [noise=true , address=45.91.116.5:4297 , piece_id=XFZUDFYDRRBBVSW6SS75DZHJHT7GS2DKPXJZCLIF7IDSYZWLSYVQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.846972534 (607h18m19.730386169s) [noise=true , address=77.90.31.19:4843 , piece_id=YYCC2RDU5LFZV5HE2BCQGPOMUAPK3BPFLU4KR65D652IQEENQTFA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.848301465 (607h18m19.731715101s) [noise=true , address=77.90.31.18:4806 , piece_id=6SFVUK4MQG27S35PSQXZHPWUVEYXQG2QHI7TIVSKVL7M4BGO5KSA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.849163101 (607h18m19.732576738s) [noise=true , address=77.90.31.10:4519 , piece_id=3QM3LBGNHO5FUVQH7DNXHMVQON7DOYEABGIXWVICTTPSPUIYJQYA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.849839742 (607h18m19.733253378s) [noise=true , address=104.36.82.3:4767 , piece_id=XPYEQ25OR24YS2LO7PUSHL2KFYYRAONBRJOZBYJP77HMM7ONWJ2Q, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:20.472725878 (607h18m20.356139509s) [upload=485547]: Next returned nil segment Jul 18 16:29:20.472752679 (607h18m20.35616632s) [upload=485547]: Waiting for error group managing segments... Jul 18 16:29:20.596980167 (607h18m20.480393796s) [noise=true , address=45.91.116.10:4081 , piece_id=ZP3GV7W7B2L2M45V4Y7OOWX6Z4Q4MI5PEX2MHMMY7D4L2PJWR3SQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.600142338 (607h18m20.483555961s) [noise=true , address=45.91.116.14:4189 , piece_id=EDZFMKYC63ABE7V5AE6E4FELMGIWPPPANRNSOFN3GAYE6X6UNUMQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.602409866 (607h18m20.485823483s) [noise=true , address=77.90.31.14:4661 , piece_id=SU2WXUKNOBMJ5HBPOVU44MD2T4PPMUDHTUSHFBQR6PXQNV5KCCIQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.605640089 (607h18m20.489053709s) [noise=true , address=45.91.116.9:4262 , piece_id=5MJOYGUIGLNE24MZCQ2PLX75TT4X4XSSVVVBN3WJH352MT5W64GQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.613715192 (607h18m20.497128805s) [noise=true , address=77.90.31.19:4843 , piece_id=YYCC2RDU5LFZV5HE2BCQGPOMUAPK3BPFLU4KR65D652IQEENQTFA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.653906544 (607h18m20.537320177s) [noise=true , address=104.36.82.5:4839 , piece_id=NWXJQEAAM2TZKJPIJMRB73K6OCI5A4SLHKY74XLPWXFFMW4RO3CQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654343289 (607h18m20.537756909s) [noise=true , address=104.36.82.3:4767 , piece_id=XPYEQ25OR24YS2LO7PUSHL2KFYYRAONBRJOZBYJP77HMM7ONWJ2Q, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654633188 (607h18m20.538046806s) [noise=true , address=104.36.82.14:4222 , piece_id=2TAGHXI3NADLHJQXCRIRY5CM6VMI2BHRPT7C6S7Y7TSPC2MGTXSQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654929539 (607h18m20.538343156s) [noise=true , address=104.36.82.14:4223 , piece_id=34IMOFVRIMVWGSZ2LJE5PYORVNQD4H3BQQNL5I2QTHD6LNB2MEPQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655173635 (607h18m20.538587262s) [noise=true , address=104.36.82.7:4911 , piece_id=INO4Q54WEULMMEAEG73QKVN4A6LLN4Z7CRJZTSLRBUDZ4NR4YHZA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655659265 (607h18m20.539072881s) [noise=true , address=45.91.116.5:4297 , piece_id=XFZUDFYDRRBBVSW6SS75DZHJHT7GS2DKPXJZCLIF7IDSYZWLSYVQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655717213 (607h18m20.53913084s) [noise=true , address=104.36.82.12:4151 , piece_id=4I5GEGRSL7PVJE55J5EC3MPDL2ZII74ZFO6KFOCBUZ4WS7KU3F5A, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:49.451525389 (607h18m49.334939016s) [noise=true , address=77.90.31.18:4806 , piece_id=6SFVUK4MQG27S35PSQXZHPWUVEYXQG2QHI7TIVSKVL7M4BGO5KSA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:35536->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451613960 (607h18m49.335027587s) [noise=true , address=77.90.31.11:4553 , piece_id=KYYATSWOYX5R7SKM67PJBA2QZC7PRRNJJ7GKPYLZBOTOZ2J6WFYQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:36256->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451690640 (607h18m49.335104266s) [noise=true , address=77.90.31.19:4841 , piece_id=IERSGG2GKGB5UD3QH3ETCKSJZJT5A6ELFHDLFLJFJ6VB3Y6QIFHA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 139.59.144.22:54400->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451746289 (607h18m49.335159915s) [noise=true , address=77.90.31.10:4519 , piece_id=3QM3LBGNHO5FUVQH7DNXHMVQON7DOYEABGIXWVICTTPSPUIYJQYA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451790375 (607h18m49.335204001s) [seg_pos={0 0}, upload=485547]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 16:29:49.569242253 (607h18m49.452655873s) [upload=485547]: Done uploading segments... Jul 18 16:29:49.569295294 (607h18m49.45270894s) [upload=485547]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.408519742s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [X.......X...................................................] duration: 2.424292851s FAIL (2025-07-18 17:10:10 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 9]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: context canceled; context canceled Jul 18 16:28:06.573617088 (607h17m6.457030715s) [seg_pos={0 0}, upload=485529]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 16:28:06.573709829 (607h17m6.457123445s) [upload=485529]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 16:28:06.573716041 (607h17m6.457129658s) [upload=485529]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 16:28:06.724230331 (607h17m6.607643945s) [upload=485529]: Done uploading segments... Jul 18 16:28:06.724250141 (607h17m6.607663755s) [upload=485529]: Upload finished. err: <nil> Jul 18 16:29:19.727355583 (607h18m19.610769209s) [upload=485547]: Starting upload Jul 18 16:29:19.727427333 (607h18m19.61084097s) [upload=485547]: Done starting upload Jul 18 16:29:19.727775544 (607h18m19.611189161s) [upload=485547]: Uploading segments... Jul 18 16:29:19.727840340 (607h18m19.611253975s) [upload=485547]: Got next segment. Inline: false Jul 18 16:29:19.727875280 (607h18m19.611288906s) [upload=485547]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 16:29:19.727880962 (607h18m19.611294582s) [upload=485547]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 16:29:19.840919734 (607h18m19.724333363s) [seg_pos={0 0}, upload=485547]: Begin upload segment... Jul 18 16:29:19.840972492 (607h18m19.72438611s) [seg_pos={0 0}, upload=485547]: Done begin upload segment. Jul 18 16:29:19.841004794 (607h18m19.724418434s) [noise=true , address=77.90.31.19:4841 , piece_id=IERSGG2GKGB5UD3QH3ETCKSJZJT5A6ELFHDLFLJFJ6VB3Y6QIFHA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841022981 (607h18m19.724436614s) [noise=true , address=77.90.31.11:4553 , piece_id=KYYATSWOYX5R7SKM67PJBA2QZC7PRRNJJ7GKPYLZBOTOZ2J6WFYQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841446916 (607h18m19.724860553s) [noise=true , address=45.91.116.9:4262 , piece_id=5MJOYGUIGLNE24MZCQ2PLX75TT4X4XSSVVVBN3WJH352MT5W64GQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841470680 (607h18m19.724884316s) [noise=true , address=104.36.82.12:4151 , piece_id=4I5GEGRSL7PVJE55J5EC3MPDL2ZII74ZFO6KFOCBUZ4WS7KU3F5A, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841825001 (607h18m19.725238633s) [noise=true , address=104.36.82.14:4222 , piece_id=2TAGHXI3NADLHJQXCRIRY5CM6VMI2BHRPT7C6S7Y7TSPC2MGTXSQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841863759 (607h18m19.725277377s) [noise=true , address=104.36.82.7:4911 , piece_id=INO4Q54WEULMMEAEG73QKVN4A6LLN4Z7CRJZTSLRBUDZ4NR4YHZA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842376953 (607h18m19.725790586s) [noise=true , address=45.91.116.14:4189 , piece_id=EDZFMKYC63ABE7V5AE6E4FELMGIWPPPANRNSOFN3GAYE6X6UNUMQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842942175 (607h18m19.726355788s) [noise=true , address=104.36.82.5:4839 , piece_id=NWXJQEAAM2TZKJPIJMRB73K6OCI5A4SLHKY74XLPWXFFMW4RO3CQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842951799 (607h18m19.726365432s) [noise=true , address=77.90.31.14:4661 , piece_id=SU2WXUKNOBMJ5HBPOVU44MD2T4PPMUDHTUSHFBQR6PXQNV5KCCIQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.843362766 (607h18m19.726776393s) [noise=true , address=104.36.82.14:4223 , piece_id=34IMOFVRIMVWGSZ2LJE5PYORVNQD4H3BQQNL5I2QTHD6LNB2MEPQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.844357136 (607h18m19.727770773s) [noise=true , address=45.91.116.10:4081 , piece_id=ZP3GV7W7B2L2M45V4Y7OOWX6Z4Q4MI5PEX2MHMMY7D4L2PJWR3SQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.845591955 (607h18m19.729005589s) [noise=true , address=45.91.116.5:4297 , piece_id=XFZUDFYDRRBBVSW6SS75DZHJHT7GS2DKPXJZCLIF7IDSYZWLSYVQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.846972534 (607h18m19.730386169s) [noise=true , address=77.90.31.19:4843 , piece_id=YYCC2RDU5LFZV5HE2BCQGPOMUAPK3BPFLU4KR65D652IQEENQTFA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.848301465 (607h18m19.731715101s) [noise=true , address=77.90.31.18:4806 , piece_id=6SFVUK4MQG27S35PSQXZHPWUVEYXQG2QHI7TIVSKVL7M4BGO5KSA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.849163101 (607h18m19.732576738s) [noise=true , address=77.90.31.10:4519 , piece_id=3QM3LBGNHO5FUVQH7DNXHMVQON7DOYEABGIXWVICTTPSPUIYJQYA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.849839742 (607h18m19.733253378s) [noise=true , address=104.36.82.3:4767 , piece_id=XPYEQ25OR24YS2LO7PUSHL2KFYYRAONBRJOZBYJP77HMM7ONWJ2Q, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:20.472725878 (607h18m20.356139509s) [upload=485547]: Next returned nil segment Jul 18 16:29:20.472752679 (607h18m20.35616632s) [upload=485547]: Waiting for error group managing segments... Jul 18 16:29:20.596980167 (607h18m20.480393796s) [noise=true , address=45.91.116.10:4081 , piece_id=ZP3GV7W7B2L2M45V4Y7OOWX6Z4Q4MI5PEX2MHMMY7D4L2PJWR3SQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.600142338 (607h18m20.483555961s) [noise=true , address=45.91.116.14:4189 , piece_id=EDZFMKYC63ABE7V5AE6E4FELMGIWPPPANRNSOFN3GAYE6X6UNUMQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.602409866 (607h18m20.485823483s) [noise=true , address=77.90.31.14:4661 , piece_id=SU2WXUKNOBMJ5HBPOVU44MD2T4PPMUDHTUSHFBQR6PXQNV5KCCIQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.605640089 (607h18m20.489053709s) [noise=true , address=45.91.116.9:4262 , piece_id=5MJOYGUIGLNE24MZCQ2PLX75TT4X4XSSVVVBN3WJH352MT5W64GQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.613715192 (607h18m20.497128805s) [noise=true , address=77.90.31.19:4843 , piece_id=YYCC2RDU5LFZV5HE2BCQGPOMUAPK3BPFLU4KR65D652IQEENQTFA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.653906544 (607h18m20.537320177s) [noise=true , address=104.36.82.5:4839 , piece_id=NWXJQEAAM2TZKJPIJMRB73K6OCI5A4SLHKY74XLPWXFFMW4RO3CQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654343289 (607h18m20.537756909s) [noise=true , address=104.36.82.3:4767 , piece_id=XPYEQ25OR24YS2LO7PUSHL2KFYYRAONBRJOZBYJP77HMM7ONWJ2Q, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654633188 (607h18m20.538046806s) [noise=true , address=104.36.82.14:4222 , piece_id=2TAGHXI3NADLHJQXCRIRY5CM6VMI2BHRPT7C6S7Y7TSPC2MGTXSQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654929539 (607h18m20.538343156s) [noise=true , address=104.36.82.14:4223 , piece_id=34IMOFVRIMVWGSZ2LJE5PYORVNQD4H3BQQNL5I2QTHD6LNB2MEPQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655173635 (607h18m20.538587262s) [noise=true , address=104.36.82.7:4911 , piece_id=INO4Q54WEULMMEAEG73QKVN4A6LLN4Z7CRJZTSLRBUDZ4NR4YHZA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655659265 (607h18m20.539072881s) [noise=true , address=45.91.116.5:4297 , piece_id=XFZUDFYDRRBBVSW6SS75DZHJHT7GS2DKPXJZCLIF7IDSYZWLSYVQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655717213 (607h18m20.53913084s) [noise=true , address=104.36.82.12:4151 , piece_id=4I5GEGRSL7PVJE55J5EC3MPDL2ZII74ZFO6KFOCBUZ4WS7KU3F5A, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:49.451525389 (607h18m49.334939016s) [noise=true , address=77.90.31.18:4806 , piece_id=6SFVUK4MQG27S35PSQXZHPWUVEYXQG2QHI7TIVSKVL7M4BGO5KSA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:35536->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451613960 (607h18m49.335027587s) [noise=true , address=77.90.31.11:4553 , piece_id=KYYATSWOYX5R7SKM67PJBA2QZC7PRRNJJ7GKPYLZBOTOZ2J6WFYQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:36256->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451690640 (607h18m49.335104266s) [noise=true , address=77.90.31.19:4841 , piece_id=IERSGG2GKGB5UD3QH3ETCKSJZJT5A6ELFHDLFLJFJ6VB3Y6QIFHA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 139.59.144.22:54400->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451746289 (607h18m49.335159915s) [noise=true , address=77.90.31.10:4519 , piece_id=3QM3LBGNHO5FUVQH7DNXHMVQON7DOYEABGIXWVICTTPSPUIYJQYA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451790375 (607h18m49.335204001s) [seg_pos={0 0}, upload=485547]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 16:29:49.569242253 (607h18m49.452655873s) [upload=485547]: Done uploading segments... Jul 18 16:29:49.569295294 (607h18m49.45270894s) [upload=485547]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.424334595s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.X.......X..................................................] duration: 2.396562177s FAIL (2025-07-18 17:09:42 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 9]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: context canceled; context canceled Jul 18 16:28:06.573617088 (607h17m6.457030715s) [seg_pos={0 0}, upload=485529]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 16:28:06.573709829 (607h17m6.457123445s) [upload=485529]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 16:28:06.573716041 (607h17m6.457129658s) [upload=485529]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 16:28:06.724230331 (607h17m6.607643945s) [upload=485529]: Done uploading segments... Jul 18 16:28:06.724250141 (607h17m6.607663755s) [upload=485529]: Upload finished. err: <nil> Jul 18 16:29:19.727355583 (607h18m19.610769209s) [upload=485547]: Starting upload Jul 18 16:29:19.727427333 (607h18m19.61084097s) [upload=485547]: Done starting upload Jul 18 16:29:19.727775544 (607h18m19.611189161s) [upload=485547]: Uploading segments... Jul 18 16:29:19.727840340 (607h18m19.611253975s) [upload=485547]: Got next segment. Inline: false Jul 18 16:29:19.727875280 (607h18m19.611288906s) [upload=485547]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 16:29:19.727880962 (607h18m19.611294582s) [upload=485547]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 16:29:19.840919734 (607h18m19.724333363s) [seg_pos={0 0}, upload=485547]: Begin upload segment... Jul 18 16:29:19.840972492 (607h18m19.72438611s) [seg_pos={0 0}, upload=485547]: Done begin upload segment. Jul 18 16:29:19.841004794 (607h18m19.724418434s) [noise=true , address=77.90.31.19:4841 , piece_id=IERSGG2GKGB5UD3QH3ETCKSJZJT5A6ELFHDLFLJFJ6VB3Y6QIFHA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841022981 (607h18m19.724436614s) [noise=true , address=77.90.31.11:4553 , piece_id=KYYATSWOYX5R7SKM67PJBA2QZC7PRRNJJ7GKPYLZBOTOZ2J6WFYQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841446916 (607h18m19.724860553s) [noise=true , address=45.91.116.9:4262 , piece_id=5MJOYGUIGLNE24MZCQ2PLX75TT4X4XSSVVVBN3WJH352MT5W64GQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841470680 (607h18m19.724884316s) [noise=true , address=104.36.82.12:4151 , piece_id=4I5GEGRSL7PVJE55J5EC3MPDL2ZII74ZFO6KFOCBUZ4WS7KU3F5A, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841825001 (607h18m19.725238633s) [noise=true , address=104.36.82.14:4222 , piece_id=2TAGHXI3NADLHJQXCRIRY5CM6VMI2BHRPT7C6S7Y7TSPC2MGTXSQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841863759 (607h18m19.725277377s) [noise=true , address=104.36.82.7:4911 , piece_id=INO4Q54WEULMMEAEG73QKVN4A6LLN4Z7CRJZTSLRBUDZ4NR4YHZA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842376953 (607h18m19.725790586s) [noise=true , address=45.91.116.14:4189 , piece_id=EDZFMKYC63ABE7V5AE6E4FELMGIWPPPANRNSOFN3GAYE6X6UNUMQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842942175 (607h18m19.726355788s) [noise=true , address=104.36.82.5:4839 , piece_id=NWXJQEAAM2TZKJPIJMRB73K6OCI5A4SLHKY74XLPWXFFMW4RO3CQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842951799 (607h18m19.726365432s) [noise=true , address=77.90.31.14:4661 , piece_id=SU2WXUKNOBMJ5HBPOVU44MD2T4PPMUDHTUSHFBQR6PXQNV5KCCIQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.843362766 (607h18m19.726776393s) [noise=true , address=104.36.82.14:4223 , piece_id=34IMOFVRIMVWGSZ2LJE5PYORVNQD4H3BQQNL5I2QTHD6LNB2MEPQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.844357136 (607h18m19.727770773s) [noise=true , address=45.91.116.10:4081 , piece_id=ZP3GV7W7B2L2M45V4Y7OOWX6Z4Q4MI5PEX2MHMMY7D4L2PJWR3SQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.845591955 (607h18m19.729005589s) [noise=true , address=45.91.116.5:4297 , piece_id=XFZUDFYDRRBBVSW6SS75DZHJHT7GS2DKPXJZCLIF7IDSYZWLSYVQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.846972534 (607h18m19.730386169s) [noise=true , address=77.90.31.19:4843 , piece_id=YYCC2RDU5LFZV5HE2BCQGPOMUAPK3BPFLU4KR65D652IQEENQTFA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.848301465 (607h18m19.731715101s) [noise=true , address=77.90.31.18:4806 , piece_id=6SFVUK4MQG27S35PSQXZHPWUVEYXQG2QHI7TIVSKVL7M4BGO5KSA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.849163101 (607h18m19.732576738s) [noise=true , address=77.90.31.10:4519 , piece_id=3QM3LBGNHO5FUVQH7DNXHMVQON7DOYEABGIXWVICTTPSPUIYJQYA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.849839742 (607h18m19.733253378s) [noise=true , address=104.36.82.3:4767 , piece_id=XPYEQ25OR24YS2LO7PUSHL2KFYYRAONBRJOZBYJP77HMM7ONWJ2Q, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:20.472725878 (607h18m20.356139509s) [upload=485547]: Next returned nil segment Jul 18 16:29:20.472752679 (607h18m20.35616632s) [upload=485547]: Waiting for error group managing segments... Jul 18 16:29:20.596980167 (607h18m20.480393796s) [noise=true , address=45.91.116.10:4081 , piece_id=ZP3GV7W7B2L2M45V4Y7OOWX6Z4Q4MI5PEX2MHMMY7D4L2PJWR3SQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.600142338 (607h18m20.483555961s) [noise=true , address=45.91.116.14:4189 , piece_id=EDZFMKYC63ABE7V5AE6E4FELMGIWPPPANRNSOFN3GAYE6X6UNUMQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.602409866 (607h18m20.485823483s) [noise=true , address=77.90.31.14:4661 , piece_id=SU2WXUKNOBMJ5HBPOVU44MD2T4PPMUDHTUSHFBQR6PXQNV5KCCIQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.605640089 (607h18m20.489053709s) [noise=true , address=45.91.116.9:4262 , piece_id=5MJOYGUIGLNE24MZCQ2PLX75TT4X4XSSVVVBN3WJH352MT5W64GQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.613715192 (607h18m20.497128805s) [noise=true , address=77.90.31.19:4843 , piece_id=YYCC2RDU5LFZV5HE2BCQGPOMUAPK3BPFLU4KR65D652IQEENQTFA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.653906544 (607h18m20.537320177s) [noise=true , address=104.36.82.5:4839 , piece_id=NWXJQEAAM2TZKJPIJMRB73K6OCI5A4SLHKY74XLPWXFFMW4RO3CQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654343289 (607h18m20.537756909s) [noise=true , address=104.36.82.3:4767 , piece_id=XPYEQ25OR24YS2LO7PUSHL2KFYYRAONBRJOZBYJP77HMM7ONWJ2Q, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654633188 (607h18m20.538046806s) [noise=true , address=104.36.82.14:4222 , piece_id=2TAGHXI3NADLHJQXCRIRY5CM6VMI2BHRPT7C6S7Y7TSPC2MGTXSQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654929539 (607h18m20.538343156s) [noise=true , address=104.36.82.14:4223 , piece_id=34IMOFVRIMVWGSZ2LJE5PYORVNQD4H3BQQNL5I2QTHD6LNB2MEPQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655173635 (607h18m20.538587262s) [noise=true , address=104.36.82.7:4911 , piece_id=INO4Q54WEULMMEAEG73QKVN4A6LLN4Z7CRJZTSLRBUDZ4NR4YHZA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655659265 (607h18m20.539072881s) [noise=true , address=45.91.116.5:4297 , piece_id=XFZUDFYDRRBBVSW6SS75DZHJHT7GS2DKPXJZCLIF7IDSYZWLSYVQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655717213 (607h18m20.53913084s) [noise=true , address=104.36.82.12:4151 , piece_id=4I5GEGRSL7PVJE55J5EC3MPDL2ZII74ZFO6KFOCBUZ4WS7KU3F5A, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:49.451525389 (607h18m49.334939016s) [noise=true , address=77.90.31.18:4806 , piece_id=6SFVUK4MQG27S35PSQXZHPWUVEYXQG2QHI7TIVSKVL7M4BGO5KSA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:35536->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451613960 (607h18m49.335027587s) [noise=true , address=77.90.31.11:4553 , piece_id=KYYATSWOYX5R7SKM67PJBA2QZC7PRRNJJ7GKPYLZBOTOZ2J6WFYQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:36256->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451690640 (607h18m49.335104266s) [noise=true , address=77.90.31.19:4841 , piece_id=IERSGG2GKGB5UD3QH3ETCKSJZJT5A6ELFHDLFLJFJ6VB3Y6QIFHA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 139.59.144.22:54400->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451746289 (607h18m49.335159915s) [noise=true , address=77.90.31.10:4519 , piece_id=3QM3LBGNHO5FUVQH7DNXHMVQON7DOYEABGIXWVICTTPSPUIYJQYA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451790375 (607h18m49.335204001s) [seg_pos={0 0}, upload=485547]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 16:29:49.569242253 (607h18m49.452655873s) [upload=485547]: Done uploading segments... Jul 18 16:29:49.569295294 (607h18m49.45270894s) [upload=485547]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.396664864s core uplink us-select-1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 176.504317ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 17:09:38 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 176.606414ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 177.656093ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 17:09:30 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 177.68355ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 176.859273ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 17:09:18 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 176.883969ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 179.537278ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 17:09:11 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 179.564547ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..X.......X.................................................] duration: 2.506302375s FAIL (2025-07-18 17:08:53 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 9]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: context canceled; context canceled Jul 18 16:28:06.573617088 (607h17m6.457030715s) [seg_pos={0 0}, upload=485529]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 16:28:06.573709829 (607h17m6.457123445s) [upload=485529]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 16:28:06.573716041 (607h17m6.457129658s) [upload=485529]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 16:28:06.724230331 (607h17m6.607643945s) [upload=485529]: Done uploading segments... Jul 18 16:28:06.724250141 (607h17m6.607663755s) [upload=485529]: Upload finished. err: <nil> Jul 18 16:29:19.727355583 (607h18m19.610769209s) [upload=485547]: Starting upload Jul 18 16:29:19.727427333 (607h18m19.61084097s) [upload=485547]: Done starting upload Jul 18 16:29:19.727775544 (607h18m19.611189161s) [upload=485547]: Uploading segments... Jul 18 16:29:19.727840340 (607h18m19.611253975s) [upload=485547]: Got next segment. Inline: false Jul 18 16:29:19.727875280 (607h18m19.611288906s) [upload=485547]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 16:29:19.727880962 (607h18m19.611294582s) [upload=485547]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 16:29:19.840919734 (607h18m19.724333363s) [seg_pos={0 0}, upload=485547]: Begin upload segment... Jul 18 16:29:19.840972492 (607h18m19.72438611s) [seg_pos={0 0}, upload=485547]: Done begin upload segment. Jul 18 16:29:19.841004794 (607h18m19.724418434s) [noise=true , address=77.90.31.19:4841 , piece_id=IERSGG2GKGB5UD3QH3ETCKSJZJT5A6ELFHDLFLJFJ6VB3Y6QIFHA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841022981 (607h18m19.724436614s) [noise=true , address=77.90.31.11:4553 , piece_id=KYYATSWOYX5R7SKM67PJBA2QZC7PRRNJJ7GKPYLZBOTOZ2J6WFYQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841446916 (607h18m19.724860553s) [noise=true , address=45.91.116.9:4262 , piece_id=5MJOYGUIGLNE24MZCQ2PLX75TT4X4XSSVVVBN3WJH352MT5W64GQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841470680 (607h18m19.724884316s) [noise=true , address=104.36.82.12:4151 , piece_id=4I5GEGRSL7PVJE55J5EC3MPDL2ZII74ZFO6KFOCBUZ4WS7KU3F5A, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841825001 (607h18m19.725238633s) [noise=true , address=104.36.82.14:4222 , piece_id=2TAGHXI3NADLHJQXCRIRY5CM6VMI2BHRPT7C6S7Y7TSPC2MGTXSQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841863759 (607h18m19.725277377s) [noise=true , address=104.36.82.7:4911 , piece_id=INO4Q54WEULMMEAEG73QKVN4A6LLN4Z7CRJZTSLRBUDZ4NR4YHZA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842376953 (607h18m19.725790586s) [noise=true , address=45.91.116.14:4189 , piece_id=EDZFMKYC63ABE7V5AE6E4FELMGIWPPPANRNSOFN3GAYE6X6UNUMQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842942175 (607h18m19.726355788s) [noise=true , address=104.36.82.5:4839 , piece_id=NWXJQEAAM2TZKJPIJMRB73K6OCI5A4SLHKY74XLPWXFFMW4RO3CQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842951799 (607h18m19.726365432s) [noise=true , address=77.90.31.14:4661 , piece_id=SU2WXUKNOBMJ5HBPOVU44MD2T4PPMUDHTUSHFBQR6PXQNV5KCCIQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.843362766 (607h18m19.726776393s) [noise=true , address=104.36.82.14:4223 , piece_id=34IMOFVRIMVWGSZ2LJE5PYORVNQD4H3BQQNL5I2QTHD6LNB2MEPQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.844357136 (607h18m19.727770773s) [noise=true , address=45.91.116.10:4081 , piece_id=ZP3GV7W7B2L2M45V4Y7OOWX6Z4Q4MI5PEX2MHMMY7D4L2PJWR3SQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.845591955 (607h18m19.729005589s) [noise=true , address=45.91.116.5:4297 , piece_id=XFZUDFYDRRBBVSW6SS75DZHJHT7GS2DKPXJZCLIF7IDSYZWLSYVQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.846972534 (607h18m19.730386169s) [noise=true , address=77.90.31.19:4843 , piece_id=YYCC2RDU5LFZV5HE2BCQGPOMUAPK3BPFLU4KR65D652IQEENQTFA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.848301465 (607h18m19.731715101s) [noise=true , address=77.90.31.18:4806 , piece_id=6SFVUK4MQG27S35PSQXZHPWUVEYXQG2QHI7TIVSKVL7M4BGO5KSA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.849163101 (607h18m19.732576738s) [noise=true , address=77.90.31.10:4519 , piece_id=3QM3LBGNHO5FUVQH7DNXHMVQON7DOYEABGIXWVICTTPSPUIYJQYA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.849839742 (607h18m19.733253378s) [noise=true , address=104.36.82.3:4767 , piece_id=XPYEQ25OR24YS2LO7PUSHL2KFYYRAONBRJOZBYJP77HMM7ONWJ2Q, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:20.472725878 (607h18m20.356139509s) [upload=485547]: Next returned nil segment Jul 18 16:29:20.472752679 (607h18m20.35616632s) [upload=485547]: Waiting for error group managing segments... Jul 18 16:29:20.596980167 (607h18m20.480393796s) [noise=true , address=45.91.116.10:4081 , piece_id=ZP3GV7W7B2L2M45V4Y7OOWX6Z4Q4MI5PEX2MHMMY7D4L2PJWR3SQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.600142338 (607h18m20.483555961s) [noise=true , address=45.91.116.14:4189 , piece_id=EDZFMKYC63ABE7V5AE6E4FELMGIWPPPANRNSOFN3GAYE6X6UNUMQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.602409866 (607h18m20.485823483s) [noise=true , address=77.90.31.14:4661 , piece_id=SU2WXUKNOBMJ5HBPOVU44MD2T4PPMUDHTUSHFBQR6PXQNV5KCCIQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.605640089 (607h18m20.489053709s) [noise=true , address=45.91.116.9:4262 , piece_id=5MJOYGUIGLNE24MZCQ2PLX75TT4X4XSSVVVBN3WJH352MT5W64GQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.613715192 (607h18m20.497128805s) [noise=true , address=77.90.31.19:4843 , piece_id=YYCC2RDU5LFZV5HE2BCQGPOMUAPK3BPFLU4KR65D652IQEENQTFA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.653906544 (607h18m20.537320177s) [noise=true , address=104.36.82.5:4839 , piece_id=NWXJQEAAM2TZKJPIJMRB73K6OCI5A4SLHKY74XLPWXFFMW4RO3CQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654343289 (607h18m20.537756909s) [noise=true , address=104.36.82.3:4767 , piece_id=XPYEQ25OR24YS2LO7PUSHL2KFYYRAONBRJOZBYJP77HMM7ONWJ2Q, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654633188 (607h18m20.538046806s) [noise=true , address=104.36.82.14:4222 , piece_id=2TAGHXI3NADLHJQXCRIRY5CM6VMI2BHRPT7C6S7Y7TSPC2MGTXSQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654929539 (607h18m20.538343156s) [noise=true , address=104.36.82.14:4223 , piece_id=34IMOFVRIMVWGSZ2LJE5PYORVNQD4H3BQQNL5I2QTHD6LNB2MEPQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655173635 (607h18m20.538587262s) [noise=true , address=104.36.82.7:4911 , piece_id=INO4Q54WEULMMEAEG73QKVN4A6LLN4Z7CRJZTSLRBUDZ4NR4YHZA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655659265 (607h18m20.539072881s) [noise=true , address=45.91.116.5:4297 , piece_id=XFZUDFYDRRBBVSW6SS75DZHJHT7GS2DKPXJZCLIF7IDSYZWLSYVQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655717213 (607h18m20.53913084s) [noise=true , address=104.36.82.12:4151 , piece_id=4I5GEGRSL7PVJE55J5EC3MPDL2ZII74ZFO6KFOCBUZ4WS7KU3F5A, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:49.451525389 (607h18m49.334939016s) [noise=true , address=77.90.31.18:4806 , piece_id=6SFVUK4MQG27S35PSQXZHPWUVEYXQG2QHI7TIVSKVL7M4BGO5KSA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:35536->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451613960 (607h18m49.335027587s) [noise=true , address=77.90.31.11:4553 , piece_id=KYYATSWOYX5R7SKM67PJBA2QZC7PRRNJJ7GKPYLZBOTOZ2J6WFYQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:36256->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451690640 (607h18m49.335104266s) [noise=true , address=77.90.31.19:4841 , piece_id=IERSGG2GKGB5UD3QH3ETCKSJZJT5A6ELFHDLFLJFJ6VB3Y6QIFHA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 139.59.144.22:54400->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451746289 (607h18m49.335159915s) [noise=true , address=77.90.31.10:4519 , piece_id=3QM3LBGNHO5FUVQH7DNXHMVQON7DOYEABGIXWVICTTPSPUIYJQYA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451790375 (607h18m49.335204001s) [seg_pos={0 0}, upload=485547]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 16:29:49.569242253 (607h18m49.452655873s) [upload=485547]: Done uploading segments... Jul 18 16:29:49.569295294 (607h18m49.45270894s) [upload=485547]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.506330658s core uplink us-select-1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 177.837508ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 17:08:07 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 177.858533ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 170.548436ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 17:08:03 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 170.574038ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...X.......X................................................] duration: 2.467808669s FAIL (2025-07-18 17:07:41 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 9]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: context canceled; context canceled Jul 18 16:28:06.573617088 (607h17m6.457030715s) [seg_pos={0 0}, upload=485529]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 16:28:06.573709829 (607h17m6.457123445s) [upload=485529]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 16:28:06.573716041 (607h17m6.457129658s) [upload=485529]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 16:28:06.724230331 (607h17m6.607643945s) [upload=485529]: Done uploading segments... Jul 18 16:28:06.724250141 (607h17m6.607663755s) [upload=485529]: Upload finished. err: <nil> Jul 18 16:29:19.727355583 (607h18m19.610769209s) [upload=485547]: Starting upload Jul 18 16:29:19.727427333 (607h18m19.61084097s) [upload=485547]: Done starting upload Jul 18 16:29:19.727775544 (607h18m19.611189161s) [upload=485547]: Uploading segments... Jul 18 16:29:19.727840340 (607h18m19.611253975s) [upload=485547]: Got next segment. Inline: false Jul 18 16:29:19.727875280 (607h18m19.611288906s) [upload=485547]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 16:29:19.727880962 (607h18m19.611294582s) [upload=485547]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 16:29:19.840919734 (607h18m19.724333363s) [seg_pos={0 0}, upload=485547]: Begin upload segment... Jul 18 16:29:19.840972492 (607h18m19.72438611s) [seg_pos={0 0}, upload=485547]: Done begin upload segment. Jul 18 16:29:19.841004794 (607h18m19.724418434s) [noise=true , address=77.90.31.19:4841 , piece_id=IERSGG2GKGB5UD3QH3ETCKSJZJT5A6ELFHDLFLJFJ6VB3Y6QIFHA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841022981 (607h18m19.724436614s) [noise=true , address=77.90.31.11:4553 , piece_id=KYYATSWOYX5R7SKM67PJBA2QZC7PRRNJJ7GKPYLZBOTOZ2J6WFYQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841446916 (607h18m19.724860553s) [noise=true , address=45.91.116.9:4262 , piece_id=5MJOYGUIGLNE24MZCQ2PLX75TT4X4XSSVVVBN3WJH352MT5W64GQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841470680 (607h18m19.724884316s) [noise=true , address=104.36.82.12:4151 , piece_id=4I5GEGRSL7PVJE55J5EC3MPDL2ZII74ZFO6KFOCBUZ4WS7KU3F5A, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841825001 (607h18m19.725238633s) [noise=true , address=104.36.82.14:4222 , piece_id=2TAGHXI3NADLHJQXCRIRY5CM6VMI2BHRPT7C6S7Y7TSPC2MGTXSQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841863759 (607h18m19.725277377s) [noise=true , address=104.36.82.7:4911 , piece_id=INO4Q54WEULMMEAEG73QKVN4A6LLN4Z7CRJZTSLRBUDZ4NR4YHZA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842376953 (607h18m19.725790586s) [noise=true , address=45.91.116.14:4189 , piece_id=EDZFMKYC63ABE7V5AE6E4FELMGIWPPPANRNSOFN3GAYE6X6UNUMQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842942175 (607h18m19.726355788s) [noise=true , address=104.36.82.5:4839 , piece_id=NWXJQEAAM2TZKJPIJMRB73K6OCI5A4SLHKY74XLPWXFFMW4RO3CQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842951799 (607h18m19.726365432s) [noise=true , address=77.90.31.14:4661 , piece_id=SU2WXUKNOBMJ5HBPOVU44MD2T4PPMUDHTUSHFBQR6PXQNV5KCCIQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.843362766 (607h18m19.726776393s) [noise=true , address=104.36.82.14:4223 , piece_id=34IMOFVRIMVWGSZ2LJE5PYORVNQD4H3BQQNL5I2QTHD6LNB2MEPQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.844357136 (607h18m19.727770773s) [noise=true , address=45.91.116.10:4081 , piece_id=ZP3GV7W7B2L2M45V4Y7OOWX6Z4Q4MI5PEX2MHMMY7D4L2PJWR3SQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.845591955 (607h18m19.729005589s) [noise=true , address=45.91.116.5:4297 , piece_id=XFZUDFYDRRBBVSW6SS75DZHJHT7GS2DKPXJZCLIF7IDSYZWLSYVQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.846972534 (607h18m19.730386169s) [noise=true , address=77.90.31.19:4843 , piece_id=YYCC2RDU5LFZV5HE2BCQGPOMUAPK3BPFLU4KR65D652IQEENQTFA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.848301465 (607h18m19.731715101s) [noise=true , address=77.90.31.18:4806 , piece_id=6SFVUK4MQG27S35PSQXZHPWUVEYXQG2QHI7TIVSKVL7M4BGO5KSA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.849163101 (607h18m19.732576738s) [noise=true , address=77.90.31.10:4519 , piece_id=3QM3LBGNHO5FUVQH7DNXHMVQON7DOYEABGIXWVICTTPSPUIYJQYA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.849839742 (607h18m19.733253378s) [noise=true , address=104.36.82.3:4767 , piece_id=XPYEQ25OR24YS2LO7PUSHL2KFYYRAONBRJOZBYJP77HMM7ONWJ2Q, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:20.472725878 (607h18m20.356139509s) [upload=485547]: Next returned nil segment Jul 18 16:29:20.472752679 (607h18m20.35616632s) [upload=485547]: Waiting for error group managing segments... Jul 18 16:29:20.596980167 (607h18m20.480393796s) [noise=true , address=45.91.116.10:4081 , piece_id=ZP3GV7W7B2L2M45V4Y7OOWX6Z4Q4MI5PEX2MHMMY7D4L2PJWR3SQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.600142338 (607h18m20.483555961s) [noise=true , address=45.91.116.14:4189 , piece_id=EDZFMKYC63ABE7V5AE6E4FELMGIWPPPANRNSOFN3GAYE6X6UNUMQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.602409866 (607h18m20.485823483s) [noise=true , address=77.90.31.14:4661 , piece_id=SU2WXUKNOBMJ5HBPOVU44MD2T4PPMUDHTUSHFBQR6PXQNV5KCCIQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.605640089 (607h18m20.489053709s) [noise=true , address=45.91.116.9:4262 , piece_id=5MJOYGUIGLNE24MZCQ2PLX75TT4X4XSSVVVBN3WJH352MT5W64GQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.613715192 (607h18m20.497128805s) [noise=true , address=77.90.31.19:4843 , piece_id=YYCC2RDU5LFZV5HE2BCQGPOMUAPK3BPFLU4KR65D652IQEENQTFA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.653906544 (607h18m20.537320177s) [noise=true , address=104.36.82.5:4839 , piece_id=NWXJQEAAM2TZKJPIJMRB73K6OCI5A4SLHKY74XLPWXFFMW4RO3CQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654343289 (607h18m20.537756909s) [noise=true , address=104.36.82.3:4767 , piece_id=XPYEQ25OR24YS2LO7PUSHL2KFYYRAONBRJOZBYJP77HMM7ONWJ2Q, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654633188 (607h18m20.538046806s) [noise=true , address=104.36.82.14:4222 , piece_id=2TAGHXI3NADLHJQXCRIRY5CM6VMI2BHRPT7C6S7Y7TSPC2MGTXSQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654929539 (607h18m20.538343156s) [noise=true , address=104.36.82.14:4223 , piece_id=34IMOFVRIMVWGSZ2LJE5PYORVNQD4H3BQQNL5I2QTHD6LNB2MEPQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655173635 (607h18m20.538587262s) [noise=true , address=104.36.82.7:4911 , piece_id=INO4Q54WEULMMEAEG73QKVN4A6LLN4Z7CRJZTSLRBUDZ4NR4YHZA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655659265 (607h18m20.539072881s) [noise=true , address=45.91.116.5:4297 , piece_id=XFZUDFYDRRBBVSW6SS75DZHJHT7GS2DKPXJZCLIF7IDSYZWLSYVQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655717213 (607h18m20.53913084s) [noise=true , address=104.36.82.12:4151 , piece_id=4I5GEGRSL7PVJE55J5EC3MPDL2ZII74ZFO6KFOCBUZ4WS7KU3F5A, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:49.451525389 (607h18m49.334939016s) [noise=true , address=77.90.31.18:4806 , piece_id=6SFVUK4MQG27S35PSQXZHPWUVEYXQG2QHI7TIVSKVL7M4BGO5KSA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:35536->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451613960 (607h18m49.335027587s) [noise=true , address=77.90.31.11:4553 , piece_id=KYYATSWOYX5R7SKM67PJBA2QZC7PRRNJJ7GKPYLZBOTOZ2J6WFYQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:36256->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451690640 (607h18m49.335104266s) [noise=true , address=77.90.31.19:4841 , piece_id=IERSGG2GKGB5UD3QH3ETCKSJZJT5A6ELFHDLFLJFJ6VB3Y6QIFHA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 139.59.144.22:54400->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451746289 (607h18m49.335159915s) [noise=true , address=77.90.31.10:4519 , piece_id=3QM3LBGNHO5FUVQH7DNXHMVQON7DOYEABGIXWVICTTPSPUIYJQYA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451790375 (607h18m49.335204001s) [seg_pos={0 0}, upload=485547]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 16:29:49.569242253 (607h18m49.452655873s) [upload=485547]: Done uploading segments... Jul 18 16:29:49.569295294 (607h18m49.45270894s) [upload=485547]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.467842258s core uplink us-select-1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 178.791389ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 17:07:32 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 178.829778ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 174.584279ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 17:07:26 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 174.610135ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....X.......X...............................................] duration: 2.396177791s FAIL (2025-07-18 17:07:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 9]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: context canceled; context canceled Jul 18 16:28:06.573617088 (607h17m6.457030715s) [seg_pos={0 0}, upload=485529]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 16:28:06.573709829 (607h17m6.457123445s) [upload=485529]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 16:28:06.573716041 (607h17m6.457129658s) [upload=485529]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 16:28:06.724230331 (607h17m6.607643945s) [upload=485529]: Done uploading segments... Jul 18 16:28:06.724250141 (607h17m6.607663755s) [upload=485529]: Upload finished. err: <nil> Jul 18 16:29:19.727355583 (607h18m19.610769209s) [upload=485547]: Starting upload Jul 18 16:29:19.727427333 (607h18m19.61084097s) [upload=485547]: Done starting upload Jul 18 16:29:19.727775544 (607h18m19.611189161s) [upload=485547]: Uploading segments... Jul 18 16:29:19.727840340 (607h18m19.611253975s) [upload=485547]: Got next segment. Inline: false Jul 18 16:29:19.727875280 (607h18m19.611288906s) [upload=485547]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 16:29:19.727880962 (607h18m19.611294582s) [upload=485547]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 16:29:19.840919734 (607h18m19.724333363s) [seg_pos={0 0}, upload=485547]: Begin upload segment... Jul 18 16:29:19.840972492 (607h18m19.72438611s) [seg_pos={0 0}, upload=485547]: Done begin upload segment. Jul 18 16:29:19.841004794 (607h18m19.724418434s) [noise=true , address=77.90.31.19:4841 , piece_id=IERSGG2GKGB5UD3QH3ETCKSJZJT5A6ELFHDLFLJFJ6VB3Y6QIFHA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841022981 (607h18m19.724436614s) [noise=true , address=77.90.31.11:4553 , piece_id=KYYATSWOYX5R7SKM67PJBA2QZC7PRRNJJ7GKPYLZBOTOZ2J6WFYQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841446916 (607h18m19.724860553s) [noise=true , address=45.91.116.9:4262 , piece_id=5MJOYGUIGLNE24MZCQ2PLX75TT4X4XSSVVVBN3WJH352MT5W64GQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841470680 (607h18m19.724884316s) [noise=true , address=104.36.82.12:4151 , piece_id=4I5GEGRSL7PVJE55J5EC3MPDL2ZII74ZFO6KFOCBUZ4WS7KU3F5A, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841825001 (607h18m19.725238633s) [noise=true , address=104.36.82.14:4222 , piece_id=2TAGHXI3NADLHJQXCRIRY5CM6VMI2BHRPT7C6S7Y7TSPC2MGTXSQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841863759 (607h18m19.725277377s) [noise=true , address=104.36.82.7:4911 , piece_id=INO4Q54WEULMMEAEG73QKVN4A6LLN4Z7CRJZTSLRBUDZ4NR4YHZA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842376953 (607h18m19.725790586s) [noise=true , address=45.91.116.14:4189 , piece_id=EDZFMKYC63ABE7V5AE6E4FELMGIWPPPANRNSOFN3GAYE6X6UNUMQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842942175 (607h18m19.726355788s) [noise=true , address=104.36.82.5:4839 , piece_id=NWXJQEAAM2TZKJPIJMRB73K6OCI5A4SLHKY74XLPWXFFMW4RO3CQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842951799 (607h18m19.726365432s) [noise=true , address=77.90.31.14:4661 , piece_id=SU2WXUKNOBMJ5HBPOVU44MD2T4PPMUDHTUSHFBQR6PXQNV5KCCIQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.843362766 (607h18m19.726776393s) [noise=true , address=104.36.82.14:4223 , piece_id=34IMOFVRIMVWGSZ2LJE5PYORVNQD4H3BQQNL5I2QTHD6LNB2MEPQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.844357136 (607h18m19.727770773s) [noise=true , address=45.91.116.10:4081 , piece_id=ZP3GV7W7B2L2M45V4Y7OOWX6Z4Q4MI5PEX2MHMMY7D4L2PJWR3SQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.845591955 (607h18m19.729005589s) [noise=true , address=45.91.116.5:4297 , piece_id=XFZUDFYDRRBBVSW6SS75DZHJHT7GS2DKPXJZCLIF7IDSYZWLSYVQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.846972534 (607h18m19.730386169s) [noise=true , address=77.90.31.19:4843 , piece_id=YYCC2RDU5LFZV5HE2BCQGPOMUAPK3BPFLU4KR65D652IQEENQTFA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.848301465 (607h18m19.731715101s) [noise=true , address=77.90.31.18:4806 , piece_id=6SFVUK4MQG27S35PSQXZHPWUVEYXQG2QHI7TIVSKVL7M4BGO5KSA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.849163101 (607h18m19.732576738s) [noise=true , address=77.90.31.10:4519 , piece_id=3QM3LBGNHO5FUVQH7DNXHMVQON7DOYEABGIXWVICTTPSPUIYJQYA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.849839742 (607h18m19.733253378s) [noise=true , address=104.36.82.3:4767 , piece_id=XPYEQ25OR24YS2LO7PUSHL2KFYYRAONBRJOZBYJP77HMM7ONWJ2Q, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:20.472725878 (607h18m20.356139509s) [upload=485547]: Next returned nil segment Jul 18 16:29:20.472752679 (607h18m20.35616632s) [upload=485547]: Waiting for error group managing segments... Jul 18 16:29:20.596980167 (607h18m20.480393796s) [noise=true , address=45.91.116.10:4081 , piece_id=ZP3GV7W7B2L2M45V4Y7OOWX6Z4Q4MI5PEX2MHMMY7D4L2PJWR3SQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.600142338 (607h18m20.483555961s) [noise=true , address=45.91.116.14:4189 , piece_id=EDZFMKYC63ABE7V5AE6E4FELMGIWPPPANRNSOFN3GAYE6X6UNUMQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.602409866 (607h18m20.485823483s) [noise=true , address=77.90.31.14:4661 , piece_id=SU2WXUKNOBMJ5HBPOVU44MD2T4PPMUDHTUSHFBQR6PXQNV5KCCIQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.605640089 (607h18m20.489053709s) [noise=true , address=45.91.116.9:4262 , piece_id=5MJOYGUIGLNE24MZCQ2PLX75TT4X4XSSVVVBN3WJH352MT5W64GQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.613715192 (607h18m20.497128805s) [noise=true , address=77.90.31.19:4843 , piece_id=YYCC2RDU5LFZV5HE2BCQGPOMUAPK3BPFLU4KR65D652IQEENQTFA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.653906544 (607h18m20.537320177s) [noise=true , address=104.36.82.5:4839 , piece_id=NWXJQEAAM2TZKJPIJMRB73K6OCI5A4SLHKY74XLPWXFFMW4RO3CQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654343289 (607h18m20.537756909s) [noise=true , address=104.36.82.3:4767 , piece_id=XPYEQ25OR24YS2LO7PUSHL2KFYYRAONBRJOZBYJP77HMM7ONWJ2Q, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654633188 (607h18m20.538046806s) [noise=true , address=104.36.82.14:4222 , piece_id=2TAGHXI3NADLHJQXCRIRY5CM6VMI2BHRPT7C6S7Y7TSPC2MGTXSQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654929539 (607h18m20.538343156s) [noise=true , address=104.36.82.14:4223 , piece_id=34IMOFVRIMVWGSZ2LJE5PYORVNQD4H3BQQNL5I2QTHD6LNB2MEPQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655173635 (607h18m20.538587262s) [noise=true , address=104.36.82.7:4911 , piece_id=INO4Q54WEULMMEAEG73QKVN4A6LLN4Z7CRJZTSLRBUDZ4NR4YHZA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655659265 (607h18m20.539072881s) [noise=true , address=45.91.116.5:4297 , piece_id=XFZUDFYDRRBBVSW6SS75DZHJHT7GS2DKPXJZCLIF7IDSYZWLSYVQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655717213 (607h18m20.53913084s) [noise=true , address=104.36.82.12:4151 , piece_id=4I5GEGRSL7PVJE55J5EC3MPDL2ZII74ZFO6KFOCBUZ4WS7KU3F5A, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:49.451525389 (607h18m49.334939016s) [noise=true , address=77.90.31.18:4806 , piece_id=6SFVUK4MQG27S35PSQXZHPWUVEYXQG2QHI7TIVSKVL7M4BGO5KSA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:35536->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451613960 (607h18m49.335027587s) [noise=true , address=77.90.31.11:4553 , piece_id=KYYATSWOYX5R7SKM67PJBA2QZC7PRRNJJ7GKPYLZBOTOZ2J6WFYQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:36256->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451690640 (607h18m49.335104266s) [noise=true , address=77.90.31.19:4841 , piece_id=IERSGG2GKGB5UD3QH3ETCKSJZJT5A6ELFHDLFLJFJ6VB3Y6QIFHA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 139.59.144.22:54400->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451746289 (607h18m49.335159915s) [noise=true , address=77.90.31.10:4519 , piece_id=3QM3LBGNHO5FUVQH7DNXHMVQON7DOYEABGIXWVICTTPSPUIYJQYA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451790375 (607h18m49.335204001s) [seg_pos={0 0}, upload=485547]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 16:29:49.569242253 (607h18m49.452655873s) [upload=485547]: Done uploading segments... Jul 18 16:29:49.569295294 (607h18m49.45270894s) [upload=485547]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.396206705s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 186.902005ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 17:06:42 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 186.927279ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....X.......X..............................................] duration: 2.375124779s FAIL (2025-07-18 17:06:33 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 9]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: context canceled; context canceled Jul 18 16:28:06.573617088 (607h17m6.457030715s) [seg_pos={0 0}, upload=485529]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 16:28:06.573709829 (607h17m6.457123445s) [upload=485529]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 16:28:06.573716041 (607h17m6.457129658s) [upload=485529]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 16:28:06.724230331 (607h17m6.607643945s) [upload=485529]: Done uploading segments... Jul 18 16:28:06.724250141 (607h17m6.607663755s) [upload=485529]: Upload finished. err: <nil> Jul 18 16:29:19.727355583 (607h18m19.610769209s) [upload=485547]: Starting upload Jul 18 16:29:19.727427333 (607h18m19.61084097s) [upload=485547]: Done starting upload Jul 18 16:29:19.727775544 (607h18m19.611189161s) [upload=485547]: Uploading segments... Jul 18 16:29:19.727840340 (607h18m19.611253975s) [upload=485547]: Got next segment. Inline: false Jul 18 16:29:19.727875280 (607h18m19.611288906s) [upload=485547]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 16:29:19.727880962 (607h18m19.611294582s) [upload=485547]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 16:29:19.840919734 (607h18m19.724333363s) [seg_pos={0 0}, upload=485547]: Begin upload segment... Jul 18 16:29:19.840972492 (607h18m19.72438611s) [seg_pos={0 0}, upload=485547]: Done begin upload segment. Jul 18 16:29:19.841004794 (607h18m19.724418434s) [noise=true , address=77.90.31.19:4841 , piece_id=IERSGG2GKGB5UD3QH3ETCKSJZJT5A6ELFHDLFLJFJ6VB3Y6QIFHA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841022981 (607h18m19.724436614s) [noise=true , address=77.90.31.11:4553 , piece_id=KYYATSWOYX5R7SKM67PJBA2QZC7PRRNJJ7GKPYLZBOTOZ2J6WFYQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841446916 (607h18m19.724860553s) [noise=true , address=45.91.116.9:4262 , piece_id=5MJOYGUIGLNE24MZCQ2PLX75TT4X4XSSVVVBN3WJH352MT5W64GQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841470680 (607h18m19.724884316s) [noise=true , address=104.36.82.12:4151 , piece_id=4I5GEGRSL7PVJE55J5EC3MPDL2ZII74ZFO6KFOCBUZ4WS7KU3F5A, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841825001 (607h18m19.725238633s) [noise=true , address=104.36.82.14:4222 , piece_id=2TAGHXI3NADLHJQXCRIRY5CM6VMI2BHRPT7C6S7Y7TSPC2MGTXSQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841863759 (607h18m19.725277377s) [noise=true , address=104.36.82.7:4911 , piece_id=INO4Q54WEULMMEAEG73QKVN4A6LLN4Z7CRJZTSLRBUDZ4NR4YHZA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842376953 (607h18m19.725790586s) [noise=true , address=45.91.116.14:4189 , piece_id=EDZFMKYC63ABE7V5AE6E4FELMGIWPPPANRNSOFN3GAYE6X6UNUMQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842942175 (607h18m19.726355788s) [noise=true , address=104.36.82.5:4839 , piece_id=NWXJQEAAM2TZKJPIJMRB73K6OCI5A4SLHKY74XLPWXFFMW4RO3CQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842951799 (607h18m19.726365432s) [noise=true , address=77.90.31.14:4661 , piece_id=SU2WXUKNOBMJ5HBPOVU44MD2T4PPMUDHTUSHFBQR6PXQNV5KCCIQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.843362766 (607h18m19.726776393s) [noise=true , address=104.36.82.14:4223 , piece_id=34IMOFVRIMVWGSZ2LJE5PYORVNQD4H3BQQNL5I2QTHD6LNB2MEPQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.844357136 (607h18m19.727770773s) [noise=true , address=45.91.116.10:4081 , piece_id=ZP3GV7W7B2L2M45V4Y7OOWX6Z4Q4MI5PEX2MHMMY7D4L2PJWR3SQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.845591955 (607h18m19.729005589s) [noise=true , address=45.91.116.5:4297 , piece_id=XFZUDFYDRRBBVSW6SS75DZHJHT7GS2DKPXJZCLIF7IDSYZWLSYVQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.846972534 (607h18m19.730386169s) [noise=true , address=77.90.31.19:4843 , piece_id=YYCC2RDU5LFZV5HE2BCQGPOMUAPK3BPFLU4KR65D652IQEENQTFA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.848301465 (607h18m19.731715101s) [noise=true , address=77.90.31.18:4806 , piece_id=6SFVUK4MQG27S35PSQXZHPWUVEYXQG2QHI7TIVSKVL7M4BGO5KSA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.849163101 (607h18m19.732576738s) [noise=true , address=77.90.31.10:4519 , piece_id=3QM3LBGNHO5FUVQH7DNXHMVQON7DOYEABGIXWVICTTPSPUIYJQYA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.849839742 (607h18m19.733253378s) [noise=true , address=104.36.82.3:4767 , piece_id=XPYEQ25OR24YS2LO7PUSHL2KFYYRAONBRJOZBYJP77HMM7ONWJ2Q, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:20.472725878 (607h18m20.356139509s) [upload=485547]: Next returned nil segment Jul 18 16:29:20.472752679 (607h18m20.35616632s) [upload=485547]: Waiting for error group managing segments... Jul 18 16:29:20.596980167 (607h18m20.480393796s) [noise=true , address=45.91.116.10:4081 , piece_id=ZP3GV7W7B2L2M45V4Y7OOWX6Z4Q4MI5PEX2MHMMY7D4L2PJWR3SQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.600142338 (607h18m20.483555961s) [noise=true , address=45.91.116.14:4189 , piece_id=EDZFMKYC63ABE7V5AE6E4FELMGIWPPPANRNSOFN3GAYE6X6UNUMQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.602409866 (607h18m20.485823483s) [noise=true , address=77.90.31.14:4661 , piece_id=SU2WXUKNOBMJ5HBPOVU44MD2T4PPMUDHTUSHFBQR6PXQNV5KCCIQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.605640089 (607h18m20.489053709s) [noise=true , address=45.91.116.9:4262 , piece_id=5MJOYGUIGLNE24MZCQ2PLX75TT4X4XSSVVVBN3WJH352MT5W64GQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.613715192 (607h18m20.497128805s) [noise=true , address=77.90.31.19:4843 , piece_id=YYCC2RDU5LFZV5HE2BCQGPOMUAPK3BPFLU4KR65D652IQEENQTFA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.653906544 (607h18m20.537320177s) [noise=true , address=104.36.82.5:4839 , piece_id=NWXJQEAAM2TZKJPIJMRB73K6OCI5A4SLHKY74XLPWXFFMW4RO3CQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654343289 (607h18m20.537756909s) [noise=true , address=104.36.82.3:4767 , piece_id=XPYEQ25OR24YS2LO7PUSHL2KFYYRAONBRJOZBYJP77HMM7ONWJ2Q, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654633188 (607h18m20.538046806s) [noise=true , address=104.36.82.14:4222 , piece_id=2TAGHXI3NADLHJQXCRIRY5CM6VMI2BHRPT7C6S7Y7TSPC2MGTXSQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654929539 (607h18m20.538343156s) [noise=true , address=104.36.82.14:4223 , piece_id=34IMOFVRIMVWGSZ2LJE5PYORVNQD4H3BQQNL5I2QTHD6LNB2MEPQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655173635 (607h18m20.538587262s) [noise=true , address=104.36.82.7:4911 , piece_id=INO4Q54WEULMMEAEG73QKVN4A6LLN4Z7CRJZTSLRBUDZ4NR4YHZA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655659265 (607h18m20.539072881s) [noise=true , address=45.91.116.5:4297 , piece_id=XFZUDFYDRRBBVSW6SS75DZHJHT7GS2DKPXJZCLIF7IDSYZWLSYVQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655717213 (607h18m20.53913084s) [noise=true , address=104.36.82.12:4151 , piece_id=4I5GEGRSL7PVJE55J5EC3MPDL2ZII74ZFO6KFOCBUZ4WS7KU3F5A, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:49.451525389 (607h18m49.334939016s) [noise=true , address=77.90.31.18:4806 , piece_id=6SFVUK4MQG27S35PSQXZHPWUVEYXQG2QHI7TIVSKVL7M4BGO5KSA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:35536->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451613960 (607h18m49.335027587s) [noise=true , address=77.90.31.11:4553 , piece_id=KYYATSWOYX5R7SKM67PJBA2QZC7PRRNJJ7GKPYLZBOTOZ2J6WFYQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:36256->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451690640 (607h18m49.335104266s) [noise=true , address=77.90.31.19:4841 , piece_id=IERSGG2GKGB5UD3QH3ETCKSJZJT5A6ELFHDLFLJFJ6VB3Y6QIFHA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 139.59.144.22:54400->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451746289 (607h18m49.335159915s) [noise=true , address=77.90.31.10:4519 , piece_id=3QM3LBGNHO5FUVQH7DNXHMVQON7DOYEABGIXWVICTTPSPUIYJQYA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451790375 (607h18m49.335204001s) [seg_pos={0 0}, upload=485547]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 16:29:49.569242253 (607h18m49.452655873s) [upload=485547]: Done uploading segments... Jul 18 16:29:49.569295294 (607h18m49.45270894s) [upload=485547]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.375197853s core uplink us-select-1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 169.770912ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 17:06:28 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 169.796173ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 179.80433ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 17:06:24 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 179.82881ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......X.......X.............................................] duration: 2.353720226s FAIL (2025-07-18 17:06:09 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 9]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: context canceled; context canceled Jul 18 16:28:06.573617088 (607h17m6.457030715s) [seg_pos={0 0}, upload=485529]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 16:28:06.573709829 (607h17m6.457123445s) [upload=485529]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 16:28:06.573716041 (607h17m6.457129658s) [upload=485529]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 16:28:06.724230331 (607h17m6.607643945s) [upload=485529]: Done uploading segments... Jul 18 16:28:06.724250141 (607h17m6.607663755s) [upload=485529]: Upload finished. err: <nil> Jul 18 16:29:19.727355583 (607h18m19.610769209s) [upload=485547]: Starting upload Jul 18 16:29:19.727427333 (607h18m19.61084097s) [upload=485547]: Done starting upload Jul 18 16:29:19.727775544 (607h18m19.611189161s) [upload=485547]: Uploading segments... Jul 18 16:29:19.727840340 (607h18m19.611253975s) [upload=485547]: Got next segment. Inline: false Jul 18 16:29:19.727875280 (607h18m19.611288906s) [upload=485547]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 16:29:19.727880962 (607h18m19.611294582s) [upload=485547]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 16:29:19.840919734 (607h18m19.724333363s) [seg_pos={0 0}, upload=485547]: Begin upload segment... Jul 18 16:29:19.840972492 (607h18m19.72438611s) [seg_pos={0 0}, upload=485547]: Done begin upload segment. Jul 18 16:29:19.841004794 (607h18m19.724418434s) [noise=true , address=77.90.31.19:4841 , piece_id=IERSGG2GKGB5UD3QH3ETCKSJZJT5A6ELFHDLFLJFJ6VB3Y6QIFHA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841022981 (607h18m19.724436614s) [noise=true , address=77.90.31.11:4553 , piece_id=KYYATSWOYX5R7SKM67PJBA2QZC7PRRNJJ7GKPYLZBOTOZ2J6WFYQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841446916 (607h18m19.724860553s) [noise=true , address=45.91.116.9:4262 , piece_id=5MJOYGUIGLNE24MZCQ2PLX75TT4X4XSSVVVBN3WJH352MT5W64GQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841470680 (607h18m19.724884316s) [noise=true , address=104.36.82.12:4151 , piece_id=4I5GEGRSL7PVJE55J5EC3MPDL2ZII74ZFO6KFOCBUZ4WS7KU3F5A, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841825001 (607h18m19.725238633s) [noise=true , address=104.36.82.14:4222 , piece_id=2TAGHXI3NADLHJQXCRIRY5CM6VMI2BHRPT7C6S7Y7TSPC2MGTXSQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841863759 (607h18m19.725277377s) [noise=true , address=104.36.82.7:4911 , piece_id=INO4Q54WEULMMEAEG73QKVN4A6LLN4Z7CRJZTSLRBUDZ4NR4YHZA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842376953 (607h18m19.725790586s) [noise=true , address=45.91.116.14:4189 , piece_id=EDZFMKYC63ABE7V5AE6E4FELMGIWPPPANRNSOFN3GAYE6X6UNUMQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842942175 (607h18m19.726355788s) [noise=true , address=104.36.82.5:4839 , piece_id=NWXJQEAAM2TZKJPIJMRB73K6OCI5A4SLHKY74XLPWXFFMW4RO3CQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842951799 (607h18m19.726365432s) [noise=true , address=77.90.31.14:4661 , piece_id=SU2WXUKNOBMJ5HBPOVU44MD2T4PPMUDHTUSHFBQR6PXQNV5KCCIQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.843362766 (607h18m19.726776393s) [noise=true , address=104.36.82.14:4223 , piece_id=34IMOFVRIMVWGSZ2LJE5PYORVNQD4H3BQQNL5I2QTHD6LNB2MEPQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.844357136 (607h18m19.727770773s) [noise=true , address=45.91.116.10:4081 , piece_id=ZP3GV7W7B2L2M45V4Y7OOWX6Z4Q4MI5PEX2MHMMY7D4L2PJWR3SQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.845591955 (607h18m19.729005589s) [noise=true , address=45.91.116.5:4297 , piece_id=XFZUDFYDRRBBVSW6SS75DZHJHT7GS2DKPXJZCLIF7IDSYZWLSYVQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.846972534 (607h18m19.730386169s) [noise=true , address=77.90.31.19:4843 , piece_id=YYCC2RDU5LFZV5HE2BCQGPOMUAPK3BPFLU4KR65D652IQEENQTFA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.848301465 (607h18m19.731715101s) [noise=true , address=77.90.31.18:4806 , piece_id=6SFVUK4MQG27S35PSQXZHPWUVEYXQG2QHI7TIVSKVL7M4BGO5KSA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.849163101 (607h18m19.732576738s) [noise=true , address=77.90.31.10:4519 , piece_id=3QM3LBGNHO5FUVQH7DNXHMVQON7DOYEABGIXWVICTTPSPUIYJQYA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.849839742 (607h18m19.733253378s) [noise=true , address=104.36.82.3:4767 , piece_id=XPYEQ25OR24YS2LO7PUSHL2KFYYRAONBRJOZBYJP77HMM7ONWJ2Q, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:20.472725878 (607h18m20.356139509s) [upload=485547]: Next returned nil segment Jul 18 16:29:20.472752679 (607h18m20.35616632s) [upload=485547]: Waiting for error group managing segments... Jul 18 16:29:20.596980167 (607h18m20.480393796s) [noise=true , address=45.91.116.10:4081 , piece_id=ZP3GV7W7B2L2M45V4Y7OOWX6Z4Q4MI5PEX2MHMMY7D4L2PJWR3SQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.600142338 (607h18m20.483555961s) [noise=true , address=45.91.116.14:4189 , piece_id=EDZFMKYC63ABE7V5AE6E4FELMGIWPPPANRNSOFN3GAYE6X6UNUMQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.602409866 (607h18m20.485823483s) [noise=true , address=77.90.31.14:4661 , piece_id=SU2WXUKNOBMJ5HBPOVU44MD2T4PPMUDHTUSHFBQR6PXQNV5KCCIQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.605640089 (607h18m20.489053709s) [noise=true , address=45.91.116.9:4262 , piece_id=5MJOYGUIGLNE24MZCQ2PLX75TT4X4XSSVVVBN3WJH352MT5W64GQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.613715192 (607h18m20.497128805s) [noise=true , address=77.90.31.19:4843 , piece_id=YYCC2RDU5LFZV5HE2BCQGPOMUAPK3BPFLU4KR65D652IQEENQTFA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.653906544 (607h18m20.537320177s) [noise=true , address=104.36.82.5:4839 , piece_id=NWXJQEAAM2TZKJPIJMRB73K6OCI5A4SLHKY74XLPWXFFMW4RO3CQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654343289 (607h18m20.537756909s) [noise=true , address=104.36.82.3:4767 , piece_id=XPYEQ25OR24YS2LO7PUSHL2KFYYRAONBRJOZBYJP77HMM7ONWJ2Q, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654633188 (607h18m20.538046806s) [noise=true , address=104.36.82.14:4222 , piece_id=2TAGHXI3NADLHJQXCRIRY5CM6VMI2BHRPT7C6S7Y7TSPC2MGTXSQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654929539 (607h18m20.538343156s) [noise=true , address=104.36.82.14:4223 , piece_id=34IMOFVRIMVWGSZ2LJE5PYORVNQD4H3BQQNL5I2QTHD6LNB2MEPQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655173635 (607h18m20.538587262s) [noise=true , address=104.36.82.7:4911 , piece_id=INO4Q54WEULMMEAEG73QKVN4A6LLN4Z7CRJZTSLRBUDZ4NR4YHZA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655659265 (607h18m20.539072881s) [noise=true , address=45.91.116.5:4297 , piece_id=XFZUDFYDRRBBVSW6SS75DZHJHT7GS2DKPXJZCLIF7IDSYZWLSYVQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655717213 (607h18m20.53913084s) [noise=true , address=104.36.82.12:4151 , piece_id=4I5GEGRSL7PVJE55J5EC3MPDL2ZII74ZFO6KFOCBUZ4WS7KU3F5A, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:49.451525389 (607h18m49.334939016s) [noise=true , address=77.90.31.18:4806 , piece_id=6SFVUK4MQG27S35PSQXZHPWUVEYXQG2QHI7TIVSKVL7M4BGO5KSA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:35536->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451613960 (607h18m49.335027587s) [noise=true , address=77.90.31.11:4553 , piece_id=KYYATSWOYX5R7SKM67PJBA2QZC7PRRNJJ7GKPYLZBOTOZ2J6WFYQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:36256->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451690640 (607h18m49.335104266s) [noise=true , address=77.90.31.19:4841 , piece_id=IERSGG2GKGB5UD3QH3ETCKSJZJT5A6ELFHDLFLJFJ6VB3Y6QIFHA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 139.59.144.22:54400->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451746289 (607h18m49.335159915s) [noise=true , address=77.90.31.10:4519 , piece_id=3QM3LBGNHO5FUVQH7DNXHMVQON7DOYEABGIXWVICTTPSPUIYJQYA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451790375 (607h18m49.335204001s) [seg_pos={0 0}, upload=485547]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 16:29:49.569242253 (607h18m49.452655873s) [upload=485547]: Done uploading segments... Jul 18 16:29:49.569295294 (607h18m49.45270894s) [upload=485547]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.353763871s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 173.437282ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 17:05:46 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 173.463288ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 173.039655ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 17:05:45 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 173.066956ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......X.......X............................................] duration: 2.425303843s FAIL (2025-07-18 17:05:23 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 9]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: context canceled; context canceled Jul 18 16:28:06.573617088 (607h17m6.457030715s) [seg_pos={0 0}, upload=485529]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 16:28:06.573709829 (607h17m6.457123445s) [upload=485529]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 16:28:06.573716041 (607h17m6.457129658s) [upload=485529]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 16:28:06.724230331 (607h17m6.607643945s) [upload=485529]: Done uploading segments... Jul 18 16:28:06.724250141 (607h17m6.607663755s) [upload=485529]: Upload finished. err: <nil> Jul 18 16:29:19.727355583 (607h18m19.610769209s) [upload=485547]: Starting upload Jul 18 16:29:19.727427333 (607h18m19.61084097s) [upload=485547]: Done starting upload Jul 18 16:29:19.727775544 (607h18m19.611189161s) [upload=485547]: Uploading segments... Jul 18 16:29:19.727840340 (607h18m19.611253975s) [upload=485547]: Got next segment. Inline: false Jul 18 16:29:19.727875280 (607h18m19.611288906s) [upload=485547]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 16:29:19.727880962 (607h18m19.611294582s) [upload=485547]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 16:29:19.840919734 (607h18m19.724333363s) [seg_pos={0 0}, upload=485547]: Begin upload segment... Jul 18 16:29:19.840972492 (607h18m19.72438611s) [seg_pos={0 0}, upload=485547]: Done begin upload segment. Jul 18 16:29:19.841004794 (607h18m19.724418434s) [noise=true , address=77.90.31.19:4841 , piece_id=IERSGG2GKGB5UD3QH3ETCKSJZJT5A6ELFHDLFLJFJ6VB3Y6QIFHA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841022981 (607h18m19.724436614s) [noise=true , address=77.90.31.11:4553 , piece_id=KYYATSWOYX5R7SKM67PJBA2QZC7PRRNJJ7GKPYLZBOTOZ2J6WFYQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841446916 (607h18m19.724860553s) [noise=true , address=45.91.116.9:4262 , piece_id=5MJOYGUIGLNE24MZCQ2PLX75TT4X4XSSVVVBN3WJH352MT5W64GQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841470680 (607h18m19.724884316s) [noise=true , address=104.36.82.12:4151 , piece_id=4I5GEGRSL7PVJE55J5EC3MPDL2ZII74ZFO6KFOCBUZ4WS7KU3F5A, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841825001 (607h18m19.725238633s) [noise=true , address=104.36.82.14:4222 , piece_id=2TAGHXI3NADLHJQXCRIRY5CM6VMI2BHRPT7C6S7Y7TSPC2MGTXSQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841863759 (607h18m19.725277377s) [noise=true , address=104.36.82.7:4911 , piece_id=INO4Q54WEULMMEAEG73QKVN4A6LLN4Z7CRJZTSLRBUDZ4NR4YHZA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842376953 (607h18m19.725790586s) [noise=true , address=45.91.116.14:4189 , piece_id=EDZFMKYC63ABE7V5AE6E4FELMGIWPPPANRNSOFN3GAYE6X6UNUMQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842942175 (607h18m19.726355788s) [noise=true , address=104.36.82.5:4839 , piece_id=NWXJQEAAM2TZKJPIJMRB73K6OCI5A4SLHKY74XLPWXFFMW4RO3CQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842951799 (607h18m19.726365432s) [noise=true , address=77.90.31.14:4661 , piece_id=SU2WXUKNOBMJ5HBPOVU44MD2T4PPMUDHTUSHFBQR6PXQNV5KCCIQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.843362766 (607h18m19.726776393s) [noise=true , address=104.36.82.14:4223 , piece_id=34IMOFVRIMVWGSZ2LJE5PYORVNQD4H3BQQNL5I2QTHD6LNB2MEPQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.844357136 (607h18m19.727770773s) [noise=true , address=45.91.116.10:4081 , piece_id=ZP3GV7W7B2L2M45V4Y7OOWX6Z4Q4MI5PEX2MHMMY7D4L2PJWR3SQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.845591955 (607h18m19.729005589s) [noise=true , address=45.91.116.5:4297 , piece_id=XFZUDFYDRRBBVSW6SS75DZHJHT7GS2DKPXJZCLIF7IDSYZWLSYVQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.846972534 (607h18m19.730386169s) [noise=true , address=77.90.31.19:4843 , piece_id=YYCC2RDU5LFZV5HE2BCQGPOMUAPK3BPFLU4KR65D652IQEENQTFA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.848301465 (607h18m19.731715101s) [noise=true , address=77.90.31.18:4806 , piece_id=6SFVUK4MQG27S35PSQXZHPWUVEYXQG2QHI7TIVSKVL7M4BGO5KSA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.849163101 (607h18m19.732576738s) [noise=true , address=77.90.31.10:4519 , piece_id=3QM3LBGNHO5FUVQH7DNXHMVQON7DOYEABGIXWVICTTPSPUIYJQYA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.849839742 (607h18m19.733253378s) [noise=true , address=104.36.82.3:4767 , piece_id=XPYEQ25OR24YS2LO7PUSHL2KFYYRAONBRJOZBYJP77HMM7ONWJ2Q, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:20.472725878 (607h18m20.356139509s) [upload=485547]: Next returned nil segment Jul 18 16:29:20.472752679 (607h18m20.35616632s) [upload=485547]: Waiting for error group managing segments... Jul 18 16:29:20.596980167 (607h18m20.480393796s) [noise=true , address=45.91.116.10:4081 , piece_id=ZP3GV7W7B2L2M45V4Y7OOWX6Z4Q4MI5PEX2MHMMY7D4L2PJWR3SQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.600142338 (607h18m20.483555961s) [noise=true , address=45.91.116.14:4189 , piece_id=EDZFMKYC63ABE7V5AE6E4FELMGIWPPPANRNSOFN3GAYE6X6UNUMQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.602409866 (607h18m20.485823483s) [noise=true , address=77.90.31.14:4661 , piece_id=SU2WXUKNOBMJ5HBPOVU44MD2T4PPMUDHTUSHFBQR6PXQNV5KCCIQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.605640089 (607h18m20.489053709s) [noise=true , address=45.91.116.9:4262 , piece_id=5MJOYGUIGLNE24MZCQ2PLX75TT4X4XSSVVVBN3WJH352MT5W64GQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.613715192 (607h18m20.497128805s) [noise=true , address=77.90.31.19:4843 , piece_id=YYCC2RDU5LFZV5HE2BCQGPOMUAPK3BPFLU4KR65D652IQEENQTFA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.653906544 (607h18m20.537320177s) [noise=true , address=104.36.82.5:4839 , piece_id=NWXJQEAAM2TZKJPIJMRB73K6OCI5A4SLHKY74XLPWXFFMW4RO3CQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654343289 (607h18m20.537756909s) [noise=true , address=104.36.82.3:4767 , piece_id=XPYEQ25OR24YS2LO7PUSHL2KFYYRAONBRJOZBYJP77HMM7ONWJ2Q, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654633188 (607h18m20.538046806s) [noise=true , address=104.36.82.14:4222 , piece_id=2TAGHXI3NADLHJQXCRIRY5CM6VMI2BHRPT7C6S7Y7TSPC2MGTXSQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654929539 (607h18m20.538343156s) [noise=true , address=104.36.82.14:4223 , piece_id=34IMOFVRIMVWGSZ2LJE5PYORVNQD4H3BQQNL5I2QTHD6LNB2MEPQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655173635 (607h18m20.538587262s) [noise=true , address=104.36.82.7:4911 , piece_id=INO4Q54WEULMMEAEG73QKVN4A6LLN4Z7CRJZTSLRBUDZ4NR4YHZA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655659265 (607h18m20.539072881s) [noise=true , address=45.91.116.5:4297 , piece_id=XFZUDFYDRRBBVSW6SS75DZHJHT7GS2DKPXJZCLIF7IDSYZWLSYVQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655717213 (607h18m20.53913084s) [noise=true , address=104.36.82.12:4151 , piece_id=4I5GEGRSL7PVJE55J5EC3MPDL2ZII74ZFO6KFOCBUZ4WS7KU3F5A, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:49.451525389 (607h18m49.334939016s) [noise=true , address=77.90.31.18:4806 , piece_id=6SFVUK4MQG27S35PSQXZHPWUVEYXQG2QHI7TIVSKVL7M4BGO5KSA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:35536->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451613960 (607h18m49.335027587s) [noise=true , address=77.90.31.11:4553 , piece_id=KYYATSWOYX5R7SKM67PJBA2QZC7PRRNJJ7GKPYLZBOTOZ2J6WFYQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:36256->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451690640 (607h18m49.335104266s) [noise=true , address=77.90.31.19:4841 , piece_id=IERSGG2GKGB5UD3QH3ETCKSJZJT5A6ELFHDLFLJFJ6VB3Y6QIFHA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 139.59.144.22:54400->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451746289 (607h18m49.335159915s) [noise=true , address=77.90.31.10:4519 , piece_id=3QM3LBGNHO5FUVQH7DNXHMVQON7DOYEABGIXWVICTTPSPUIYJQYA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451790375 (607h18m49.335204001s) [seg_pos={0 0}, upload=485547]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 16:29:49.569242253 (607h18m49.452655873s) [upload=485547]: Done uploading segments... Jul 18 16:29:49.569295294 (607h18m49.45270894s) [upload=485547]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.42533424s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 178.974813ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 17:05:14 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 178.998075ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 176.060818ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 17:04:35 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 176.084329ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 170.824995ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 17:04:32 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 170.861517ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........X.......X...........................................] duration: 2.470572121s FAIL (2025-07-18 17:04:24 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 9]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: context canceled; context canceled Jul 18 16:28:06.573617088 (607h17m6.457030715s) [seg_pos={0 0}, upload=485529]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 16:28:06.573709829 (607h17m6.457123445s) [upload=485529]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 16:28:06.573716041 (607h17m6.457129658s) [upload=485529]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 16:28:06.724230331 (607h17m6.607643945s) [upload=485529]: Done uploading segments... Jul 18 16:28:06.724250141 (607h17m6.607663755s) [upload=485529]: Upload finished. err: <nil> Jul 18 16:29:19.727355583 (607h18m19.610769209s) [upload=485547]: Starting upload Jul 18 16:29:19.727427333 (607h18m19.61084097s) [upload=485547]: Done starting upload Jul 18 16:29:19.727775544 (607h18m19.611189161s) [upload=485547]: Uploading segments... Jul 18 16:29:19.727840340 (607h18m19.611253975s) [upload=485547]: Got next segment. Inline: false Jul 18 16:29:19.727875280 (607h18m19.611288906s) [upload=485547]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 16:29:19.727880962 (607h18m19.611294582s) [upload=485547]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 16:29:19.840919734 (607h18m19.724333363s) [seg_pos={0 0}, upload=485547]: Begin upload segment... Jul 18 16:29:19.840972492 (607h18m19.72438611s) [seg_pos={0 0}, upload=485547]: Done begin upload segment. Jul 18 16:29:19.841004794 (607h18m19.724418434s) [noise=true , address=77.90.31.19:4841 , piece_id=IERSGG2GKGB5UD3QH3ETCKSJZJT5A6ELFHDLFLJFJ6VB3Y6QIFHA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841022981 (607h18m19.724436614s) [noise=true , address=77.90.31.11:4553 , piece_id=KYYATSWOYX5R7SKM67PJBA2QZC7PRRNJJ7GKPYLZBOTOZ2J6WFYQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841446916 (607h18m19.724860553s) [noise=true , address=45.91.116.9:4262 , piece_id=5MJOYGUIGLNE24MZCQ2PLX75TT4X4XSSVVVBN3WJH352MT5W64GQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841470680 (607h18m19.724884316s) [noise=true , address=104.36.82.12:4151 , piece_id=4I5GEGRSL7PVJE55J5EC3MPDL2ZII74ZFO6KFOCBUZ4WS7KU3F5A, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841825001 (607h18m19.725238633s) [noise=true , address=104.36.82.14:4222 , piece_id=2TAGHXI3NADLHJQXCRIRY5CM6VMI2BHRPT7C6S7Y7TSPC2MGTXSQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841863759 (607h18m19.725277377s) [noise=true , address=104.36.82.7:4911 , piece_id=INO4Q54WEULMMEAEG73QKVN4A6LLN4Z7CRJZTSLRBUDZ4NR4YHZA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842376953 (607h18m19.725790586s) [noise=true , address=45.91.116.14:4189 , piece_id=EDZFMKYC63ABE7V5AE6E4FELMGIWPPPANRNSOFN3GAYE6X6UNUMQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842942175 (607h18m19.726355788s) [noise=true , address=104.36.82.5:4839 , piece_id=NWXJQEAAM2TZKJPIJMRB73K6OCI5A4SLHKY74XLPWXFFMW4RO3CQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842951799 (607h18m19.726365432s) [noise=true , address=77.90.31.14:4661 , piece_id=SU2WXUKNOBMJ5HBPOVU44MD2T4PPMUDHTUSHFBQR6PXQNV5KCCIQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.843362766 (607h18m19.726776393s) [noise=true , address=104.36.82.14:4223 , piece_id=34IMOFVRIMVWGSZ2LJE5PYORVNQD4H3BQQNL5I2QTHD6LNB2MEPQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.844357136 (607h18m19.727770773s) [noise=true , address=45.91.116.10:4081 , piece_id=ZP3GV7W7B2L2M45V4Y7OOWX6Z4Q4MI5PEX2MHMMY7D4L2PJWR3SQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.845591955 (607h18m19.729005589s) [noise=true , address=45.91.116.5:4297 , piece_id=XFZUDFYDRRBBVSW6SS75DZHJHT7GS2DKPXJZCLIF7IDSYZWLSYVQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.846972534 (607h18m19.730386169s) [noise=true , address=77.90.31.19:4843 , piece_id=YYCC2RDU5LFZV5HE2BCQGPOMUAPK3BPFLU4KR65D652IQEENQTFA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.848301465 (607h18m19.731715101s) [noise=true , address=77.90.31.18:4806 , piece_id=6SFVUK4MQG27S35PSQXZHPWUVEYXQG2QHI7TIVSKVL7M4BGO5KSA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.849163101 (607h18m19.732576738s) [noise=true , address=77.90.31.10:4519 , piece_id=3QM3LBGNHO5FUVQH7DNXHMVQON7DOYEABGIXWVICTTPSPUIYJQYA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.849839742 (607h18m19.733253378s) [noise=true , address=104.36.82.3:4767 , piece_id=XPYEQ25OR24YS2LO7PUSHL2KFYYRAONBRJOZBYJP77HMM7ONWJ2Q, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:20.472725878 (607h18m20.356139509s) [upload=485547]: Next returned nil segment Jul 18 16:29:20.472752679 (607h18m20.35616632s) [upload=485547]: Waiting for error group managing segments... Jul 18 16:29:20.596980167 (607h18m20.480393796s) [noise=true , address=45.91.116.10:4081 , piece_id=ZP3GV7W7B2L2M45V4Y7OOWX6Z4Q4MI5PEX2MHMMY7D4L2PJWR3SQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.600142338 (607h18m20.483555961s) [noise=true , address=45.91.116.14:4189 , piece_id=EDZFMKYC63ABE7V5AE6E4FELMGIWPPPANRNSOFN3GAYE6X6UNUMQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.602409866 (607h18m20.485823483s) [noise=true , address=77.90.31.14:4661 , piece_id=SU2WXUKNOBMJ5HBPOVU44MD2T4PPMUDHTUSHFBQR6PXQNV5KCCIQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.605640089 (607h18m20.489053709s) [noise=true , address=45.91.116.9:4262 , piece_id=5MJOYGUIGLNE24MZCQ2PLX75TT4X4XSSVVVBN3WJH352MT5W64GQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.613715192 (607h18m20.497128805s) [noise=true , address=77.90.31.19:4843 , piece_id=YYCC2RDU5LFZV5HE2BCQGPOMUAPK3BPFLU4KR65D652IQEENQTFA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.653906544 (607h18m20.537320177s) [noise=true , address=104.36.82.5:4839 , piece_id=NWXJQEAAM2TZKJPIJMRB73K6OCI5A4SLHKY74XLPWXFFMW4RO3CQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654343289 (607h18m20.537756909s) [noise=true , address=104.36.82.3:4767 , piece_id=XPYEQ25OR24YS2LO7PUSHL2KFYYRAONBRJOZBYJP77HMM7ONWJ2Q, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654633188 (607h18m20.538046806s) [noise=true , address=104.36.82.14:4222 , piece_id=2TAGHXI3NADLHJQXCRIRY5CM6VMI2BHRPT7C6S7Y7TSPC2MGTXSQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654929539 (607h18m20.538343156s) [noise=true , address=104.36.82.14:4223 , piece_id=34IMOFVRIMVWGSZ2LJE5PYORVNQD4H3BQQNL5I2QTHD6LNB2MEPQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655173635 (607h18m20.538587262s) [noise=true , address=104.36.82.7:4911 , piece_id=INO4Q54WEULMMEAEG73QKVN4A6LLN4Z7CRJZTSLRBUDZ4NR4YHZA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655659265 (607h18m20.539072881s) [noise=true , address=45.91.116.5:4297 , piece_id=XFZUDFYDRRBBVSW6SS75DZHJHT7GS2DKPXJZCLIF7IDSYZWLSYVQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655717213 (607h18m20.53913084s) [noise=true , address=104.36.82.12:4151 , piece_id=4I5GEGRSL7PVJE55J5EC3MPDL2ZII74ZFO6KFOCBUZ4WS7KU3F5A, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:49.451525389 (607h18m49.334939016s) [noise=true , address=77.90.31.18:4806 , piece_id=6SFVUK4MQG27S35PSQXZHPWUVEYXQG2QHI7TIVSKVL7M4BGO5KSA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:35536->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451613960 (607h18m49.335027587s) [noise=true , address=77.90.31.11:4553 , piece_id=KYYATSWOYX5R7SKM67PJBA2QZC7PRRNJJ7GKPYLZBOTOZ2J6WFYQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:36256->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451690640 (607h18m49.335104266s) [noise=true , address=77.90.31.19:4841 , piece_id=IERSGG2GKGB5UD3QH3ETCKSJZJT5A6ELFHDLFLJFJ6VB3Y6QIFHA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 139.59.144.22:54400->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451746289 (607h18m49.335159915s) [noise=true , address=77.90.31.10:4519 , piece_id=3QM3LBGNHO5FUVQH7DNXHMVQON7DOYEABGIXWVICTTPSPUIYJQYA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451790375 (607h18m49.335204001s) [seg_pos={0 0}, upload=485547]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 16:29:49.569242253 (607h18m49.452655873s) [upload=485547]: Done uploading segments... Jul 18 16:29:49.569295294 (607h18m49.45270894s) [upload=485547]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.470601721s core uplink us-select-1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 178.213417ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 17:03:54 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 178.249579ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........X.......X..........................................] duration: 2.297541869s FAIL (2025-07-18 17:03:41 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 9]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: context canceled; context canceled Jul 18 16:28:06.573617088 (607h17m6.457030715s) [seg_pos={0 0}, upload=485529]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 16:28:06.573709829 (607h17m6.457123445s) [upload=485529]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 16:28:06.573716041 (607h17m6.457129658s) [upload=485529]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 16:28:06.724230331 (607h17m6.607643945s) [upload=485529]: Done uploading segments... Jul 18 16:28:06.724250141 (607h17m6.607663755s) [upload=485529]: Upload finished. err: <nil> Jul 18 16:29:19.727355583 (607h18m19.610769209s) [upload=485547]: Starting upload Jul 18 16:29:19.727427333 (607h18m19.61084097s) [upload=485547]: Done starting upload Jul 18 16:29:19.727775544 (607h18m19.611189161s) [upload=485547]: Uploading segments... Jul 18 16:29:19.727840340 (607h18m19.611253975s) [upload=485547]: Got next segment. Inline: false Jul 18 16:29:19.727875280 (607h18m19.611288906s) [upload=485547]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 16:29:19.727880962 (607h18m19.611294582s) [upload=485547]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 16:29:19.840919734 (607h18m19.724333363s) [seg_pos={0 0}, upload=485547]: Begin upload segment... Jul 18 16:29:19.840972492 (607h18m19.72438611s) [seg_pos={0 0}, upload=485547]: Done begin upload segment. Jul 18 16:29:19.841004794 (607h18m19.724418434s) [noise=true , address=77.90.31.19:4841 , piece_id=IERSGG2GKGB5UD3QH3ETCKSJZJT5A6ELFHDLFLJFJ6VB3Y6QIFHA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841022981 (607h18m19.724436614s) [noise=true , address=77.90.31.11:4553 , piece_id=KYYATSWOYX5R7SKM67PJBA2QZC7PRRNJJ7GKPYLZBOTOZ2J6WFYQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841446916 (607h18m19.724860553s) [noise=true , address=45.91.116.9:4262 , piece_id=5MJOYGUIGLNE24MZCQ2PLX75TT4X4XSSVVVBN3WJH352MT5W64GQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841470680 (607h18m19.724884316s) [noise=true , address=104.36.82.12:4151 , piece_id=4I5GEGRSL7PVJE55J5EC3MPDL2ZII74ZFO6KFOCBUZ4WS7KU3F5A, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841825001 (607h18m19.725238633s) [noise=true , address=104.36.82.14:4222 , piece_id=2TAGHXI3NADLHJQXCRIRY5CM6VMI2BHRPT7C6S7Y7TSPC2MGTXSQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841863759 (607h18m19.725277377s) [noise=true , address=104.36.82.7:4911 , piece_id=INO4Q54WEULMMEAEG73QKVN4A6LLN4Z7CRJZTSLRBUDZ4NR4YHZA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842376953 (607h18m19.725790586s) [noise=true , address=45.91.116.14:4189 , piece_id=EDZFMKYC63ABE7V5AE6E4FELMGIWPPPANRNSOFN3GAYE6X6UNUMQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842942175 (607h18m19.726355788s) [noise=true , address=104.36.82.5:4839 , piece_id=NWXJQEAAM2TZKJPIJMRB73K6OCI5A4SLHKY74XLPWXFFMW4RO3CQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842951799 (607h18m19.726365432s) [noise=true , address=77.90.31.14:4661 , piece_id=SU2WXUKNOBMJ5HBPOVU44MD2T4PPMUDHTUSHFBQR6PXQNV5KCCIQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.843362766 (607h18m19.726776393s) [noise=true , address=104.36.82.14:4223 , piece_id=34IMOFVRIMVWGSZ2LJE5PYORVNQD4H3BQQNL5I2QTHD6LNB2MEPQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.844357136 (607h18m19.727770773s) [noise=true , address=45.91.116.10:4081 , piece_id=ZP3GV7W7B2L2M45V4Y7OOWX6Z4Q4MI5PEX2MHMMY7D4L2PJWR3SQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.845591955 (607h18m19.729005589s) [noise=true , address=45.91.116.5:4297 , piece_id=XFZUDFYDRRBBVSW6SS75DZHJHT7GS2DKPXJZCLIF7IDSYZWLSYVQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.846972534 (607h18m19.730386169s) [noise=true , address=77.90.31.19:4843 , piece_id=YYCC2RDU5LFZV5HE2BCQGPOMUAPK3BPFLU4KR65D652IQEENQTFA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.848301465 (607h18m19.731715101s) [noise=true , address=77.90.31.18:4806 , piece_id=6SFVUK4MQG27S35PSQXZHPWUVEYXQG2QHI7TIVSKVL7M4BGO5KSA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.849163101 (607h18m19.732576738s) [noise=true , address=77.90.31.10:4519 , piece_id=3QM3LBGNHO5FUVQH7DNXHMVQON7DOYEABGIXWVICTTPSPUIYJQYA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.849839742 (607h18m19.733253378s) [noise=true , address=104.36.82.3:4767 , piece_id=XPYEQ25OR24YS2LO7PUSHL2KFYYRAONBRJOZBYJP77HMM7ONWJ2Q, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:20.472725878 (607h18m20.356139509s) [upload=485547]: Next returned nil segment Jul 18 16:29:20.472752679 (607h18m20.35616632s) [upload=485547]: Waiting for error group managing segments... Jul 18 16:29:20.596980167 (607h18m20.480393796s) [noise=true , address=45.91.116.10:4081 , piece_id=ZP3GV7W7B2L2M45V4Y7OOWX6Z4Q4MI5PEX2MHMMY7D4L2PJWR3SQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.600142338 (607h18m20.483555961s) [noise=true , address=45.91.116.14:4189 , piece_id=EDZFMKYC63ABE7V5AE6E4FELMGIWPPPANRNSOFN3GAYE6X6UNUMQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.602409866 (607h18m20.485823483s) [noise=true , address=77.90.31.14:4661 , piece_id=SU2WXUKNOBMJ5HBPOVU44MD2T4PPMUDHTUSHFBQR6PXQNV5KCCIQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.605640089 (607h18m20.489053709s) [noise=true , address=45.91.116.9:4262 , piece_id=5MJOYGUIGLNE24MZCQ2PLX75TT4X4XSSVVVBN3WJH352MT5W64GQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.613715192 (607h18m20.497128805s) [noise=true , address=77.90.31.19:4843 , piece_id=YYCC2RDU5LFZV5HE2BCQGPOMUAPK3BPFLU4KR65D652IQEENQTFA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.653906544 (607h18m20.537320177s) [noise=true , address=104.36.82.5:4839 , piece_id=NWXJQEAAM2TZKJPIJMRB73K6OCI5A4SLHKY74XLPWXFFMW4RO3CQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654343289 (607h18m20.537756909s) [noise=true , address=104.36.82.3:4767 , piece_id=XPYEQ25OR24YS2LO7PUSHL2KFYYRAONBRJOZBYJP77HMM7ONWJ2Q, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654633188 (607h18m20.538046806s) [noise=true , address=104.36.82.14:4222 , piece_id=2TAGHXI3NADLHJQXCRIRY5CM6VMI2BHRPT7C6S7Y7TSPC2MGTXSQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654929539 (607h18m20.538343156s) [noise=true , address=104.36.82.14:4223 , piece_id=34IMOFVRIMVWGSZ2LJE5PYORVNQD4H3BQQNL5I2QTHD6LNB2MEPQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655173635 (607h18m20.538587262s) [noise=true , address=104.36.82.7:4911 , piece_id=INO4Q54WEULMMEAEG73QKVN4A6LLN4Z7CRJZTSLRBUDZ4NR4YHZA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655659265 (607h18m20.539072881s) [noise=true , address=45.91.116.5:4297 , piece_id=XFZUDFYDRRBBVSW6SS75DZHJHT7GS2DKPXJZCLIF7IDSYZWLSYVQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655717213 (607h18m20.53913084s) [noise=true , address=104.36.82.12:4151 , piece_id=4I5GEGRSL7PVJE55J5EC3MPDL2ZII74ZFO6KFOCBUZ4WS7KU3F5A, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:49.451525389 (607h18m49.334939016s) [noise=true , address=77.90.31.18:4806 , piece_id=6SFVUK4MQG27S35PSQXZHPWUVEYXQG2QHI7TIVSKVL7M4BGO5KSA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:35536->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451613960 (607h18m49.335027587s) [noise=true , address=77.90.31.11:4553 , piece_id=KYYATSWOYX5R7SKM67PJBA2QZC7PRRNJJ7GKPYLZBOTOZ2J6WFYQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:36256->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451690640 (607h18m49.335104266s) [noise=true , address=77.90.31.19:4841 , piece_id=IERSGG2GKGB5UD3QH3ETCKSJZJT5A6ELFHDLFLJFJ6VB3Y6QIFHA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 139.59.144.22:54400->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451746289 (607h18m49.335159915s) [noise=true , address=77.90.31.10:4519 , piece_id=3QM3LBGNHO5FUVQH7DNXHMVQON7DOYEABGIXWVICTTPSPUIYJQYA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451790375 (607h18m49.335204001s) [seg_pos={0 0}, upload=485547]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 16:29:49.569242253 (607h18m49.452655873s) [upload=485547]: Done uploading segments... Jul 18 16:29:49.569295294 (607h18m49.45270894s) [upload=485547]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.297576428s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 177.633987ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 17:03:40 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 177.673493ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........X.......X.........................................] duration: 2.347277674s FAIL (2025-07-18 17:03:07 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 9]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: context canceled; context canceled Jul 18 16:28:06.573617088 (607h17m6.457030715s) [seg_pos={0 0}, upload=485529]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 16:28:06.573709829 (607h17m6.457123445s) [upload=485529]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 16:28:06.573716041 (607h17m6.457129658s) [upload=485529]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 16:28:06.724230331 (607h17m6.607643945s) [upload=485529]: Done uploading segments... Jul 18 16:28:06.724250141 (607h17m6.607663755s) [upload=485529]: Upload finished. err: <nil> Jul 18 16:29:19.727355583 (607h18m19.610769209s) [upload=485547]: Starting upload Jul 18 16:29:19.727427333 (607h18m19.61084097s) [upload=485547]: Done starting upload Jul 18 16:29:19.727775544 (607h18m19.611189161s) [upload=485547]: Uploading segments... Jul 18 16:29:19.727840340 (607h18m19.611253975s) [upload=485547]: Got next segment. Inline: false Jul 18 16:29:19.727875280 (607h18m19.611288906s) [upload=485547]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 16:29:19.727880962 (607h18m19.611294582s) [upload=485547]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 16:29:19.840919734 (607h18m19.724333363s) [seg_pos={0 0}, upload=485547]: Begin upload segment... Jul 18 16:29:19.840972492 (607h18m19.72438611s) [seg_pos={0 0}, upload=485547]: Done begin upload segment. Jul 18 16:29:19.841004794 (607h18m19.724418434s) [noise=true , address=77.90.31.19:4841 , piece_id=IERSGG2GKGB5UD3QH3ETCKSJZJT5A6ELFHDLFLJFJ6VB3Y6QIFHA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841022981 (607h18m19.724436614s) [noise=true , address=77.90.31.11:4553 , piece_id=KYYATSWOYX5R7SKM67PJBA2QZC7PRRNJJ7GKPYLZBOTOZ2J6WFYQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841446916 (607h18m19.724860553s) [noise=true , address=45.91.116.9:4262 , piece_id=5MJOYGUIGLNE24MZCQ2PLX75TT4X4XSSVVVBN3WJH352MT5W64GQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841470680 (607h18m19.724884316s) [noise=true , address=104.36.82.12:4151 , piece_id=4I5GEGRSL7PVJE55J5EC3MPDL2ZII74ZFO6KFOCBUZ4WS7KU3F5A, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841825001 (607h18m19.725238633s) [noise=true , address=104.36.82.14:4222 , piece_id=2TAGHXI3NADLHJQXCRIRY5CM6VMI2BHRPT7C6S7Y7TSPC2MGTXSQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841863759 (607h18m19.725277377s) [noise=true , address=104.36.82.7:4911 , piece_id=INO4Q54WEULMMEAEG73QKVN4A6LLN4Z7CRJZTSLRBUDZ4NR4YHZA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842376953 (607h18m19.725790586s) [noise=true , address=45.91.116.14:4189 , piece_id=EDZFMKYC63ABE7V5AE6E4FELMGIWPPPANRNSOFN3GAYE6X6UNUMQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842942175 (607h18m19.726355788s) [noise=true , address=104.36.82.5:4839 , piece_id=NWXJQEAAM2TZKJPIJMRB73K6OCI5A4SLHKY74XLPWXFFMW4RO3CQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842951799 (607h18m19.726365432s) [noise=true , address=77.90.31.14:4661 , piece_id=SU2WXUKNOBMJ5HBPOVU44MD2T4PPMUDHTUSHFBQR6PXQNV5KCCIQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.843362766 (607h18m19.726776393s) [noise=true , address=104.36.82.14:4223 , piece_id=34IMOFVRIMVWGSZ2LJE5PYORVNQD4H3BQQNL5I2QTHD6LNB2MEPQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.844357136 (607h18m19.727770773s) [noise=true , address=45.91.116.10:4081 , piece_id=ZP3GV7W7B2L2M45V4Y7OOWX6Z4Q4MI5PEX2MHMMY7D4L2PJWR3SQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.845591955 (607h18m19.729005589s) [noise=true , address=45.91.116.5:4297 , piece_id=XFZUDFYDRRBBVSW6SS75DZHJHT7GS2DKPXJZCLIF7IDSYZWLSYVQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.846972534 (607h18m19.730386169s) [noise=true , address=77.90.31.19:4843 , piece_id=YYCC2RDU5LFZV5HE2BCQGPOMUAPK3BPFLU4KR65D652IQEENQTFA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.848301465 (607h18m19.731715101s) [noise=true , address=77.90.31.18:4806 , piece_id=6SFVUK4MQG27S35PSQXZHPWUVEYXQG2QHI7TIVSKVL7M4BGO5KSA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.849163101 (607h18m19.732576738s) [noise=true , address=77.90.31.10:4519 , piece_id=3QM3LBGNHO5FUVQH7DNXHMVQON7DOYEABGIXWVICTTPSPUIYJQYA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.849839742 (607h18m19.733253378s) [noise=true , address=104.36.82.3:4767 , piece_id=XPYEQ25OR24YS2LO7PUSHL2KFYYRAONBRJOZBYJP77HMM7ONWJ2Q, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:20.472725878 (607h18m20.356139509s) [upload=485547]: Next returned nil segment Jul 18 16:29:20.472752679 (607h18m20.35616632s) [upload=485547]: Waiting for error group managing segments... Jul 18 16:29:20.596980167 (607h18m20.480393796s) [noise=true , address=45.91.116.10:4081 , piece_id=ZP3GV7W7B2L2M45V4Y7OOWX6Z4Q4MI5PEX2MHMMY7D4L2PJWR3SQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.600142338 (607h18m20.483555961s) [noise=true , address=45.91.116.14:4189 , piece_id=EDZFMKYC63ABE7V5AE6E4FELMGIWPPPANRNSOFN3GAYE6X6UNUMQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.602409866 (607h18m20.485823483s) [noise=true , address=77.90.31.14:4661 , piece_id=SU2WXUKNOBMJ5HBPOVU44MD2T4PPMUDHTUSHFBQR6PXQNV5KCCIQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.605640089 (607h18m20.489053709s) [noise=true , address=45.91.116.9:4262 , piece_id=5MJOYGUIGLNE24MZCQ2PLX75TT4X4XSSVVVBN3WJH352MT5W64GQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.613715192 (607h18m20.497128805s) [noise=true , address=77.90.31.19:4843 , piece_id=YYCC2RDU5LFZV5HE2BCQGPOMUAPK3BPFLU4KR65D652IQEENQTFA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.653906544 (607h18m20.537320177s) [noise=true , address=104.36.82.5:4839 , piece_id=NWXJQEAAM2TZKJPIJMRB73K6OCI5A4SLHKY74XLPWXFFMW4RO3CQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654343289 (607h18m20.537756909s) [noise=true , address=104.36.82.3:4767 , piece_id=XPYEQ25OR24YS2LO7PUSHL2KFYYRAONBRJOZBYJP77HMM7ONWJ2Q, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654633188 (607h18m20.538046806s) [noise=true , address=104.36.82.14:4222 , piece_id=2TAGHXI3NADLHJQXCRIRY5CM6VMI2BHRPT7C6S7Y7TSPC2MGTXSQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654929539 (607h18m20.538343156s) [noise=true , address=104.36.82.14:4223 , piece_id=34IMOFVRIMVWGSZ2LJE5PYORVNQD4H3BQQNL5I2QTHD6LNB2MEPQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655173635 (607h18m20.538587262s) [noise=true , address=104.36.82.7:4911 , piece_id=INO4Q54WEULMMEAEG73QKVN4A6LLN4Z7CRJZTSLRBUDZ4NR4YHZA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655659265 (607h18m20.539072881s) [noise=true , address=45.91.116.5:4297 , piece_id=XFZUDFYDRRBBVSW6SS75DZHJHT7GS2DKPXJZCLIF7IDSYZWLSYVQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655717213 (607h18m20.53913084s) [noise=true , address=104.36.82.12:4151 , piece_id=4I5GEGRSL7PVJE55J5EC3MPDL2ZII74ZFO6KFOCBUZ4WS7KU3F5A, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:49.451525389 (607h18m49.334939016s) [noise=true , address=77.90.31.18:4806 , piece_id=6SFVUK4MQG27S35PSQXZHPWUVEYXQG2QHI7TIVSKVL7M4BGO5KSA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:35536->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451613960 (607h18m49.335027587s) [noise=true , address=77.90.31.11:4553 , piece_id=KYYATSWOYX5R7SKM67PJBA2QZC7PRRNJJ7GKPYLZBOTOZ2J6WFYQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:36256->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451690640 (607h18m49.335104266s) [noise=true , address=77.90.31.19:4841 , piece_id=IERSGG2GKGB5UD3QH3ETCKSJZJT5A6ELFHDLFLJFJ6VB3Y6QIFHA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 139.59.144.22:54400->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451746289 (607h18m49.335159915s) [noise=true , address=77.90.31.10:4519 , piece_id=3QM3LBGNHO5FUVQH7DNXHMVQON7DOYEABGIXWVICTTPSPUIYJQYA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451790375 (607h18m49.335204001s) [seg_pos={0 0}, upload=485547]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 16:29:49.569242253 (607h18m49.452655873s) [upload=485547]: Done uploading segments... Jul 18 16:29:49.569295294 (607h18m49.45270894s) [upload=485547]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.34730981s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 178.193049ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 17:02:43 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 178.220255ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 173.512317ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 17:02:39 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 173.541613ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........X.......X........................................] duration: 2.390916585s FAIL (2025-07-18 17:02:30 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 9]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: context canceled; context canceled Jul 18 16:28:06.573617088 (607h17m6.457030715s) [seg_pos={0 0}, upload=485529]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 16:28:06.573709829 (607h17m6.457123445s) [upload=485529]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 16:28:06.573716041 (607h17m6.457129658s) [upload=485529]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 16:28:06.724230331 (607h17m6.607643945s) [upload=485529]: Done uploading segments... Jul 18 16:28:06.724250141 (607h17m6.607663755s) [upload=485529]: Upload finished. err: <nil> Jul 18 16:29:19.727355583 (607h18m19.610769209s) [upload=485547]: Starting upload Jul 18 16:29:19.727427333 (607h18m19.61084097s) [upload=485547]: Done starting upload Jul 18 16:29:19.727775544 (607h18m19.611189161s) [upload=485547]: Uploading segments... Jul 18 16:29:19.727840340 (607h18m19.611253975s) [upload=485547]: Got next segment. Inline: false Jul 18 16:29:19.727875280 (607h18m19.611288906s) [upload=485547]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 16:29:19.727880962 (607h18m19.611294582s) [upload=485547]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 16:29:19.840919734 (607h18m19.724333363s) [seg_pos={0 0}, upload=485547]: Begin upload segment... Jul 18 16:29:19.840972492 (607h18m19.72438611s) [seg_pos={0 0}, upload=485547]: Done begin upload segment. Jul 18 16:29:19.841004794 (607h18m19.724418434s) [noise=true , address=77.90.31.19:4841 , piece_id=IERSGG2GKGB5UD3QH3ETCKSJZJT5A6ELFHDLFLJFJ6VB3Y6QIFHA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841022981 (607h18m19.724436614s) [noise=true , address=77.90.31.11:4553 , piece_id=KYYATSWOYX5R7SKM67PJBA2QZC7PRRNJJ7GKPYLZBOTOZ2J6WFYQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841446916 (607h18m19.724860553s) [noise=true , address=45.91.116.9:4262 , piece_id=5MJOYGUIGLNE24MZCQ2PLX75TT4X4XSSVVVBN3WJH352MT5W64GQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841470680 (607h18m19.724884316s) [noise=true , address=104.36.82.12:4151 , piece_id=4I5GEGRSL7PVJE55J5EC3MPDL2ZII74ZFO6KFOCBUZ4WS7KU3F5A, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841825001 (607h18m19.725238633s) [noise=true , address=104.36.82.14:4222 , piece_id=2TAGHXI3NADLHJQXCRIRY5CM6VMI2BHRPT7C6S7Y7TSPC2MGTXSQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841863759 (607h18m19.725277377s) [noise=true , address=104.36.82.7:4911 , piece_id=INO4Q54WEULMMEAEG73QKVN4A6LLN4Z7CRJZTSLRBUDZ4NR4YHZA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842376953 (607h18m19.725790586s) [noise=true , address=45.91.116.14:4189 , piece_id=EDZFMKYC63ABE7V5AE6E4FELMGIWPPPANRNSOFN3GAYE6X6UNUMQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842942175 (607h18m19.726355788s) [noise=true , address=104.36.82.5:4839 , piece_id=NWXJQEAAM2TZKJPIJMRB73K6OCI5A4SLHKY74XLPWXFFMW4RO3CQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842951799 (607h18m19.726365432s) [noise=true , address=77.90.31.14:4661 , piece_id=SU2WXUKNOBMJ5HBPOVU44MD2T4PPMUDHTUSHFBQR6PXQNV5KCCIQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.843362766 (607h18m19.726776393s) [noise=true , address=104.36.82.14:4223 , piece_id=34IMOFVRIMVWGSZ2LJE5PYORVNQD4H3BQQNL5I2QTHD6LNB2MEPQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.844357136 (607h18m19.727770773s) [noise=true , address=45.91.116.10:4081 , piece_id=ZP3GV7W7B2L2M45V4Y7OOWX6Z4Q4MI5PEX2MHMMY7D4L2PJWR3SQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.845591955 (607h18m19.729005589s) [noise=true , address=45.91.116.5:4297 , piece_id=XFZUDFYDRRBBVSW6SS75DZHJHT7GS2DKPXJZCLIF7IDSYZWLSYVQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.846972534 (607h18m19.730386169s) [noise=true , address=77.90.31.19:4843 , piece_id=YYCC2RDU5LFZV5HE2BCQGPOMUAPK3BPFLU4KR65D652IQEENQTFA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.848301465 (607h18m19.731715101s) [noise=true , address=77.90.31.18:4806 , piece_id=6SFVUK4MQG27S35PSQXZHPWUVEYXQG2QHI7TIVSKVL7M4BGO5KSA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.849163101 (607h18m19.732576738s) [noise=true , address=77.90.31.10:4519 , piece_id=3QM3LBGNHO5FUVQH7DNXHMVQON7DOYEABGIXWVICTTPSPUIYJQYA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.849839742 (607h18m19.733253378s) [noise=true , address=104.36.82.3:4767 , piece_id=XPYEQ25OR24YS2LO7PUSHL2KFYYRAONBRJOZBYJP77HMM7ONWJ2Q, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:20.472725878 (607h18m20.356139509s) [upload=485547]: Next returned nil segment Jul 18 16:29:20.472752679 (607h18m20.35616632s) [upload=485547]: Waiting for error group managing segments... Jul 18 16:29:20.596980167 (607h18m20.480393796s) [noise=true , address=45.91.116.10:4081 , piece_id=ZP3GV7W7B2L2M45V4Y7OOWX6Z4Q4MI5PEX2MHMMY7D4L2PJWR3SQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.600142338 (607h18m20.483555961s) [noise=true , address=45.91.116.14:4189 , piece_id=EDZFMKYC63ABE7V5AE6E4FELMGIWPPPANRNSOFN3GAYE6X6UNUMQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.602409866 (607h18m20.485823483s) [noise=true , address=77.90.31.14:4661 , piece_id=SU2WXUKNOBMJ5HBPOVU44MD2T4PPMUDHTUSHFBQR6PXQNV5KCCIQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.605640089 (607h18m20.489053709s) [noise=true , address=45.91.116.9:4262 , piece_id=5MJOYGUIGLNE24MZCQ2PLX75TT4X4XSSVVVBN3WJH352MT5W64GQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.613715192 (607h18m20.497128805s) [noise=true , address=77.90.31.19:4843 , piece_id=YYCC2RDU5LFZV5HE2BCQGPOMUAPK3BPFLU4KR65D652IQEENQTFA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.653906544 (607h18m20.537320177s) [noise=true , address=104.36.82.5:4839 , piece_id=NWXJQEAAM2TZKJPIJMRB73K6OCI5A4SLHKY74XLPWXFFMW4RO3CQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654343289 (607h18m20.537756909s) [noise=true , address=104.36.82.3:4767 , piece_id=XPYEQ25OR24YS2LO7PUSHL2KFYYRAONBRJOZBYJP77HMM7ONWJ2Q, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654633188 (607h18m20.538046806s) [noise=true , address=104.36.82.14:4222 , piece_id=2TAGHXI3NADLHJQXCRIRY5CM6VMI2BHRPT7C6S7Y7TSPC2MGTXSQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654929539 (607h18m20.538343156s) [noise=true , address=104.36.82.14:4223 , piece_id=34IMOFVRIMVWGSZ2LJE5PYORVNQD4H3BQQNL5I2QTHD6LNB2MEPQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655173635 (607h18m20.538587262s) [noise=true , address=104.36.82.7:4911 , piece_id=INO4Q54WEULMMEAEG73QKVN4A6LLN4Z7CRJZTSLRBUDZ4NR4YHZA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655659265 (607h18m20.539072881s) [noise=true , address=45.91.116.5:4297 , piece_id=XFZUDFYDRRBBVSW6SS75DZHJHT7GS2DKPXJZCLIF7IDSYZWLSYVQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655717213 (607h18m20.53913084s) [noise=true , address=104.36.82.12:4151 , piece_id=4I5GEGRSL7PVJE55J5EC3MPDL2ZII74ZFO6KFOCBUZ4WS7KU3F5A, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:49.451525389 (607h18m49.334939016s) [noise=true , address=77.90.31.18:4806 , piece_id=6SFVUK4MQG27S35PSQXZHPWUVEYXQG2QHI7TIVSKVL7M4BGO5KSA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:35536->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451613960 (607h18m49.335027587s) [noise=true , address=77.90.31.11:4553 , piece_id=KYYATSWOYX5R7SKM67PJBA2QZC7PRRNJJ7GKPYLZBOTOZ2J6WFYQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:36256->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451690640 (607h18m49.335104266s) [noise=true , address=77.90.31.19:4841 , piece_id=IERSGG2GKGB5UD3QH3ETCKSJZJT5A6ELFHDLFLJFJ6VB3Y6QIFHA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 139.59.144.22:54400->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451746289 (607h18m49.335159915s) [noise=true , address=77.90.31.10:4519 , piece_id=3QM3LBGNHO5FUVQH7DNXHMVQON7DOYEABGIXWVICTTPSPUIYJQYA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451790375 (607h18m49.335204001s) [seg_pos={0 0}, upload=485547]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 16:29:49.569242253 (607h18m49.452655873s) [upload=485547]: Done uploading segments... Jul 18 16:29:49.569295294 (607h18m49.45270894s) [upload=485547]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.390947866s core uplink us-select-1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 167.114949ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 17:01:52 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 167.206171ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 188.372969ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 17:01:33 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 188.402113ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............X.......X.......................................] duration: 2.411434873s FAIL (2025-07-18 17:01:32 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 9]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: context canceled; context canceled Jul 18 16:28:06.573617088 (607h17m6.457030715s) [seg_pos={0 0}, upload=485529]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 16:28:06.573709829 (607h17m6.457123445s) [upload=485529]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 16:28:06.573716041 (607h17m6.457129658s) [upload=485529]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 16:28:06.724230331 (607h17m6.607643945s) [upload=485529]: Done uploading segments... Jul 18 16:28:06.724250141 (607h17m6.607663755s) [upload=485529]: Upload finished. err: <nil> Jul 18 16:29:19.727355583 (607h18m19.610769209s) [upload=485547]: Starting upload Jul 18 16:29:19.727427333 (607h18m19.61084097s) [upload=485547]: Done starting upload Jul 18 16:29:19.727775544 (607h18m19.611189161s) [upload=485547]: Uploading segments... Jul 18 16:29:19.727840340 (607h18m19.611253975s) [upload=485547]: Got next segment. Inline: false Jul 18 16:29:19.727875280 (607h18m19.611288906s) [upload=485547]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 16:29:19.727880962 (607h18m19.611294582s) [upload=485547]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 16:29:19.840919734 (607h18m19.724333363s) [seg_pos={0 0}, upload=485547]: Begin upload segment... Jul 18 16:29:19.840972492 (607h18m19.72438611s) [seg_pos={0 0}, upload=485547]: Done begin upload segment. Jul 18 16:29:19.841004794 (607h18m19.724418434s) [noise=true , address=77.90.31.19:4841 , piece_id=IERSGG2GKGB5UD3QH3ETCKSJZJT5A6ELFHDLFLJFJ6VB3Y6QIFHA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841022981 (607h18m19.724436614s) [noise=true , address=77.90.31.11:4553 , piece_id=KYYATSWOYX5R7SKM67PJBA2QZC7PRRNJJ7GKPYLZBOTOZ2J6WFYQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841446916 (607h18m19.724860553s) [noise=true , address=45.91.116.9:4262 , piece_id=5MJOYGUIGLNE24MZCQ2PLX75TT4X4XSSVVVBN3WJH352MT5W64GQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841470680 (607h18m19.724884316s) [noise=true , address=104.36.82.12:4151 , piece_id=4I5GEGRSL7PVJE55J5EC3MPDL2ZII74ZFO6KFOCBUZ4WS7KU3F5A, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841825001 (607h18m19.725238633s) [noise=true , address=104.36.82.14:4222 , piece_id=2TAGHXI3NADLHJQXCRIRY5CM6VMI2BHRPT7C6S7Y7TSPC2MGTXSQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841863759 (607h18m19.725277377s) [noise=true , address=104.36.82.7:4911 , piece_id=INO4Q54WEULMMEAEG73QKVN4A6LLN4Z7CRJZTSLRBUDZ4NR4YHZA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842376953 (607h18m19.725790586s) [noise=true , address=45.91.116.14:4189 , piece_id=EDZFMKYC63ABE7V5AE6E4FELMGIWPPPANRNSOFN3GAYE6X6UNUMQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842942175 (607h18m19.726355788s) [noise=true , address=104.36.82.5:4839 , piece_id=NWXJQEAAM2TZKJPIJMRB73K6OCI5A4SLHKY74XLPWXFFMW4RO3CQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842951799 (607h18m19.726365432s) [noise=true , address=77.90.31.14:4661 , piece_id=SU2WXUKNOBMJ5HBPOVU44MD2T4PPMUDHTUSHFBQR6PXQNV5KCCIQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.843362766 (607h18m19.726776393s) [noise=true , address=104.36.82.14:4223 , piece_id=34IMOFVRIMVWGSZ2LJE5PYORVNQD4H3BQQNL5I2QTHD6LNB2MEPQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.844357136 (607h18m19.727770773s) [noise=true , address=45.91.116.10:4081 , piece_id=ZP3GV7W7B2L2M45V4Y7OOWX6Z4Q4MI5PEX2MHMMY7D4L2PJWR3SQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.845591955 (607h18m19.729005589s) [noise=true , address=45.91.116.5:4297 , piece_id=XFZUDFYDRRBBVSW6SS75DZHJHT7GS2DKPXJZCLIF7IDSYZWLSYVQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.846972534 (607h18m19.730386169s) [noise=true , address=77.90.31.19:4843 , piece_id=YYCC2RDU5LFZV5HE2BCQGPOMUAPK3BPFLU4KR65D652IQEENQTFA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.848301465 (607h18m19.731715101s) [noise=true , address=77.90.31.18:4806 , piece_id=6SFVUK4MQG27S35PSQXZHPWUVEYXQG2QHI7TIVSKVL7M4BGO5KSA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.849163101 (607h18m19.732576738s) [noise=true , address=77.90.31.10:4519 , piece_id=3QM3LBGNHO5FUVQH7DNXHMVQON7DOYEABGIXWVICTTPSPUIYJQYA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.849839742 (607h18m19.733253378s) [noise=true , address=104.36.82.3:4767 , piece_id=XPYEQ25OR24YS2LO7PUSHL2KFYYRAONBRJOZBYJP77HMM7ONWJ2Q, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:20.472725878 (607h18m20.356139509s) [upload=485547]: Next returned nil segment Jul 18 16:29:20.472752679 (607h18m20.35616632s) [upload=485547]: Waiting for error group managing segments... Jul 18 16:29:20.596980167 (607h18m20.480393796s) [noise=true , address=45.91.116.10:4081 , piece_id=ZP3GV7W7B2L2M45V4Y7OOWX6Z4Q4MI5PEX2MHMMY7D4L2PJWR3SQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.600142338 (607h18m20.483555961s) [noise=true , address=45.91.116.14:4189 , piece_id=EDZFMKYC63ABE7V5AE6E4FELMGIWPPPANRNSOFN3GAYE6X6UNUMQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.602409866 (607h18m20.485823483s) [noise=true , address=77.90.31.14:4661 , piece_id=SU2WXUKNOBMJ5HBPOVU44MD2T4PPMUDHTUSHFBQR6PXQNV5KCCIQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.605640089 (607h18m20.489053709s) [noise=true , address=45.91.116.9:4262 , piece_id=5MJOYGUIGLNE24MZCQ2PLX75TT4X4XSSVVVBN3WJH352MT5W64GQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.613715192 (607h18m20.497128805s) [noise=true , address=77.90.31.19:4843 , piece_id=YYCC2RDU5LFZV5HE2BCQGPOMUAPK3BPFLU4KR65D652IQEENQTFA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.653906544 (607h18m20.537320177s) [noise=true , address=104.36.82.5:4839 , piece_id=NWXJQEAAM2TZKJPIJMRB73K6OCI5A4SLHKY74XLPWXFFMW4RO3CQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654343289 (607h18m20.537756909s) [noise=true , address=104.36.82.3:4767 , piece_id=XPYEQ25OR24YS2LO7PUSHL2KFYYRAONBRJOZBYJP77HMM7ONWJ2Q, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654633188 (607h18m20.538046806s) [noise=true , address=104.36.82.14:4222 , piece_id=2TAGHXI3NADLHJQXCRIRY5CM6VMI2BHRPT7C6S7Y7TSPC2MGTXSQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654929539 (607h18m20.538343156s) [noise=true , address=104.36.82.14:4223 , piece_id=34IMOFVRIMVWGSZ2LJE5PYORVNQD4H3BQQNL5I2QTHD6LNB2MEPQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655173635 (607h18m20.538587262s) [noise=true , address=104.36.82.7:4911 , piece_id=INO4Q54WEULMMEAEG73QKVN4A6LLN4Z7CRJZTSLRBUDZ4NR4YHZA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655659265 (607h18m20.539072881s) [noise=true , address=45.91.116.5:4297 , piece_id=XFZUDFYDRRBBVSW6SS75DZHJHT7GS2DKPXJZCLIF7IDSYZWLSYVQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655717213 (607h18m20.53913084s) [noise=true , address=104.36.82.12:4151 , piece_id=4I5GEGRSL7PVJE55J5EC3MPDL2ZII74ZFO6KFOCBUZ4WS7KU3F5A, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:49.451525389 (607h18m49.334939016s) [noise=true , address=77.90.31.18:4806 , piece_id=6SFVUK4MQG27S35PSQXZHPWUVEYXQG2QHI7TIVSKVL7M4BGO5KSA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:35536->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451613960 (607h18m49.335027587s) [noise=true , address=77.90.31.11:4553 , piece_id=KYYATSWOYX5R7SKM67PJBA2QZC7PRRNJJ7GKPYLZBOTOZ2J6WFYQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:36256->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451690640 (607h18m49.335104266s) [noise=true , address=77.90.31.19:4841 , piece_id=IERSGG2GKGB5UD3QH3ETCKSJZJT5A6ELFHDLFLJFJ6VB3Y6QIFHA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 139.59.144.22:54400->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451746289 (607h18m49.335159915s) [noise=true , address=77.90.31.10:4519 , piece_id=3QM3LBGNHO5FUVQH7DNXHMVQON7DOYEABGIXWVICTTPSPUIYJQYA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451790375 (607h18m49.335204001s) [seg_pos={0 0}, upload=485547]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 16:29:49.569242253 (607h18m49.452655873s) [upload=485547]: Done uploading segments... Jul 18 16:29:49.569295294 (607h18m49.45270894s) [upload=485547]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.411460141s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............X.......X......................................] duration: 2.546060757s FAIL (2025-07-18 17:00:43 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 9]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: context canceled; context canceled Jul 18 16:28:06.573617088 (607h17m6.457030715s) [seg_pos={0 0}, upload=485529]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 16:28:06.573709829 (607h17m6.457123445s) [upload=485529]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 16:28:06.573716041 (607h17m6.457129658s) [upload=485529]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 16:28:06.724230331 (607h17m6.607643945s) [upload=485529]: Done uploading segments... Jul 18 16:28:06.724250141 (607h17m6.607663755s) [upload=485529]: Upload finished. err: <nil> Jul 18 16:29:19.727355583 (607h18m19.610769209s) [upload=485547]: Starting upload Jul 18 16:29:19.727427333 (607h18m19.61084097s) [upload=485547]: Done starting upload Jul 18 16:29:19.727775544 (607h18m19.611189161s) [upload=485547]: Uploading segments... Jul 18 16:29:19.727840340 (607h18m19.611253975s) [upload=485547]: Got next segment. Inline: false Jul 18 16:29:19.727875280 (607h18m19.611288906s) [upload=485547]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 16:29:19.727880962 (607h18m19.611294582s) [upload=485547]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 16:29:19.840919734 (607h18m19.724333363s) [seg_pos={0 0}, upload=485547]: Begin upload segment... Jul 18 16:29:19.840972492 (607h18m19.72438611s) [seg_pos={0 0}, upload=485547]: Done begin upload segment. Jul 18 16:29:19.841004794 (607h18m19.724418434s) [noise=true , address=77.90.31.19:4841 , piece_id=IERSGG2GKGB5UD3QH3ETCKSJZJT5A6ELFHDLFLJFJ6VB3Y6QIFHA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841022981 (607h18m19.724436614s) [noise=true , address=77.90.31.11:4553 , piece_id=KYYATSWOYX5R7SKM67PJBA2QZC7PRRNJJ7GKPYLZBOTOZ2J6WFYQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841446916 (607h18m19.724860553s) [noise=true , address=45.91.116.9:4262 , piece_id=5MJOYGUIGLNE24MZCQ2PLX75TT4X4XSSVVVBN3WJH352MT5W64GQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841470680 (607h18m19.724884316s) [noise=true , address=104.36.82.12:4151 , piece_id=4I5GEGRSL7PVJE55J5EC3MPDL2ZII74ZFO6KFOCBUZ4WS7KU3F5A, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841825001 (607h18m19.725238633s) [noise=true , address=104.36.82.14:4222 , piece_id=2TAGHXI3NADLHJQXCRIRY5CM6VMI2BHRPT7C6S7Y7TSPC2MGTXSQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841863759 (607h18m19.725277377s) [noise=true , address=104.36.82.7:4911 , piece_id=INO4Q54WEULMMEAEG73QKVN4A6LLN4Z7CRJZTSLRBUDZ4NR4YHZA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842376953 (607h18m19.725790586s) [noise=true , address=45.91.116.14:4189 , piece_id=EDZFMKYC63ABE7V5AE6E4FELMGIWPPPANRNSOFN3GAYE6X6UNUMQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842942175 (607h18m19.726355788s) [noise=true , address=104.36.82.5:4839 , piece_id=NWXJQEAAM2TZKJPIJMRB73K6OCI5A4SLHKY74XLPWXFFMW4RO3CQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842951799 (607h18m19.726365432s) [noise=true , address=77.90.31.14:4661 , piece_id=SU2WXUKNOBMJ5HBPOVU44MD2T4PPMUDHTUSHFBQR6PXQNV5KCCIQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.843362766 (607h18m19.726776393s) [noise=true , address=104.36.82.14:4223 , piece_id=34IMOFVRIMVWGSZ2LJE5PYORVNQD4H3BQQNL5I2QTHD6LNB2MEPQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.844357136 (607h18m19.727770773s) [noise=true , address=45.91.116.10:4081 , piece_id=ZP3GV7W7B2L2M45V4Y7OOWX6Z4Q4MI5PEX2MHMMY7D4L2PJWR3SQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.845591955 (607h18m19.729005589s) [noise=true , address=45.91.116.5:4297 , piece_id=XFZUDFYDRRBBVSW6SS75DZHJHT7GS2DKPXJZCLIF7IDSYZWLSYVQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.846972534 (607h18m19.730386169s) [noise=true , address=77.90.31.19:4843 , piece_id=YYCC2RDU5LFZV5HE2BCQGPOMUAPK3BPFLU4KR65D652IQEENQTFA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.848301465 (607h18m19.731715101s) [noise=true , address=77.90.31.18:4806 , piece_id=6SFVUK4MQG27S35PSQXZHPWUVEYXQG2QHI7TIVSKVL7M4BGO5KSA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.849163101 (607h18m19.732576738s) [noise=true , address=77.90.31.10:4519 , piece_id=3QM3LBGNHO5FUVQH7DNXHMVQON7DOYEABGIXWVICTTPSPUIYJQYA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.849839742 (607h18m19.733253378s) [noise=true , address=104.36.82.3:4767 , piece_id=XPYEQ25OR24YS2LO7PUSHL2KFYYRAONBRJOZBYJP77HMM7ONWJ2Q, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:20.472725878 (607h18m20.356139509s) [upload=485547]: Next returned nil segment Jul 18 16:29:20.472752679 (607h18m20.35616632s) [upload=485547]: Waiting for error group managing segments... Jul 18 16:29:20.596980167 (607h18m20.480393796s) [noise=true , address=45.91.116.10:4081 , piece_id=ZP3GV7W7B2L2M45V4Y7OOWX6Z4Q4MI5PEX2MHMMY7D4L2PJWR3SQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.600142338 (607h18m20.483555961s) [noise=true , address=45.91.116.14:4189 , piece_id=EDZFMKYC63ABE7V5AE6E4FELMGIWPPPANRNSOFN3GAYE6X6UNUMQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.602409866 (607h18m20.485823483s) [noise=true , address=77.90.31.14:4661 , piece_id=SU2WXUKNOBMJ5HBPOVU44MD2T4PPMUDHTUSHFBQR6PXQNV5KCCIQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.605640089 (607h18m20.489053709s) [noise=true , address=45.91.116.9:4262 , piece_id=5MJOYGUIGLNE24MZCQ2PLX75TT4X4XSSVVVBN3WJH352MT5W64GQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.613715192 (607h18m20.497128805s) [noise=true , address=77.90.31.19:4843 , piece_id=YYCC2RDU5LFZV5HE2BCQGPOMUAPK3BPFLU4KR65D652IQEENQTFA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.653906544 (607h18m20.537320177s) [noise=true , address=104.36.82.5:4839 , piece_id=NWXJQEAAM2TZKJPIJMRB73K6OCI5A4SLHKY74XLPWXFFMW4RO3CQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654343289 (607h18m20.537756909s) [noise=true , address=104.36.82.3:4767 , piece_id=XPYEQ25OR24YS2LO7PUSHL2KFYYRAONBRJOZBYJP77HMM7ONWJ2Q, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654633188 (607h18m20.538046806s) [noise=true , address=104.36.82.14:4222 , piece_id=2TAGHXI3NADLHJQXCRIRY5CM6VMI2BHRPT7C6S7Y7TSPC2MGTXSQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654929539 (607h18m20.538343156s) [noise=true , address=104.36.82.14:4223 , piece_id=34IMOFVRIMVWGSZ2LJE5PYORVNQD4H3BQQNL5I2QTHD6LNB2MEPQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655173635 (607h18m20.538587262s) [noise=true , address=104.36.82.7:4911 , piece_id=INO4Q54WEULMMEAEG73QKVN4A6LLN4Z7CRJZTSLRBUDZ4NR4YHZA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655659265 (607h18m20.539072881s) [noise=true , address=45.91.116.5:4297 , piece_id=XFZUDFYDRRBBVSW6SS75DZHJHT7GS2DKPXJZCLIF7IDSYZWLSYVQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655717213 (607h18m20.53913084s) [noise=true , address=104.36.82.12:4151 , piece_id=4I5GEGRSL7PVJE55J5EC3MPDL2ZII74ZFO6KFOCBUZ4WS7KU3F5A, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:49.451525389 (607h18m49.334939016s) [noise=true , address=77.90.31.18:4806 , piece_id=6SFVUK4MQG27S35PSQXZHPWUVEYXQG2QHI7TIVSKVL7M4BGO5KSA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:35536->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451613960 (607h18m49.335027587s) [noise=true , address=77.90.31.11:4553 , piece_id=KYYATSWOYX5R7SKM67PJBA2QZC7PRRNJJ7GKPYLZBOTOZ2J6WFYQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:36256->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451690640 (607h18m49.335104266s) [noise=true , address=77.90.31.19:4841 , piece_id=IERSGG2GKGB5UD3QH3ETCKSJZJT5A6ELFHDLFLJFJ6VB3Y6QIFHA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 139.59.144.22:54400->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451746289 (607h18m49.335159915s) [noise=true , address=77.90.31.10:4519 , piece_id=3QM3LBGNHO5FUVQH7DNXHMVQON7DOYEABGIXWVICTTPSPUIYJQYA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451790375 (607h18m49.335204001s) [seg_pos={0 0}, upload=485547]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 16:29:49.569242253 (607h18m49.452655873s) [upload=485547]: Done uploading segments... Jul 18 16:29:49.569295294 (607h18m49.45270894s) [upload=485547]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.546094559s core uplink us-select-1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 173.03937ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 17:00:39 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 173.070664ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 176.346091ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 17:00:34 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 176.403221ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............X.......X.....................................] duration: 2.410405358s FAIL (2025-07-18 17:00:24 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 9]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: context canceled; context canceled Jul 18 16:28:06.573617088 (607h17m6.457030715s) [seg_pos={0 0}, upload=485529]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 16:28:06.573709829 (607h17m6.457123445s) [upload=485529]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 16:28:06.573716041 (607h17m6.457129658s) [upload=485529]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 16:28:06.724230331 (607h17m6.607643945s) [upload=485529]: Done uploading segments... Jul 18 16:28:06.724250141 (607h17m6.607663755s) [upload=485529]: Upload finished. err: <nil> Jul 18 16:29:19.727355583 (607h18m19.610769209s) [upload=485547]: Starting upload Jul 18 16:29:19.727427333 (607h18m19.61084097s) [upload=485547]: Done starting upload Jul 18 16:29:19.727775544 (607h18m19.611189161s) [upload=485547]: Uploading segments... Jul 18 16:29:19.727840340 (607h18m19.611253975s) [upload=485547]: Got next segment. Inline: false Jul 18 16:29:19.727875280 (607h18m19.611288906s) [upload=485547]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 16:29:19.727880962 (607h18m19.611294582s) [upload=485547]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 16:29:19.840919734 (607h18m19.724333363s) [seg_pos={0 0}, upload=485547]: Begin upload segment... Jul 18 16:29:19.840972492 (607h18m19.72438611s) [seg_pos={0 0}, upload=485547]: Done begin upload segment. Jul 18 16:29:19.841004794 (607h18m19.724418434s) [noise=true , address=77.90.31.19:4841 , piece_id=IERSGG2GKGB5UD3QH3ETCKSJZJT5A6ELFHDLFLJFJ6VB3Y6QIFHA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841022981 (607h18m19.724436614s) [noise=true , address=77.90.31.11:4553 , piece_id=KYYATSWOYX5R7SKM67PJBA2QZC7PRRNJJ7GKPYLZBOTOZ2J6WFYQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841446916 (607h18m19.724860553s) [noise=true , address=45.91.116.9:4262 , piece_id=5MJOYGUIGLNE24MZCQ2PLX75TT4X4XSSVVVBN3WJH352MT5W64GQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841470680 (607h18m19.724884316s) [noise=true , address=104.36.82.12:4151 , piece_id=4I5GEGRSL7PVJE55J5EC3MPDL2ZII74ZFO6KFOCBUZ4WS7KU3F5A, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841825001 (607h18m19.725238633s) [noise=true , address=104.36.82.14:4222 , piece_id=2TAGHXI3NADLHJQXCRIRY5CM6VMI2BHRPT7C6S7Y7TSPC2MGTXSQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841863759 (607h18m19.725277377s) [noise=true , address=104.36.82.7:4911 , piece_id=INO4Q54WEULMMEAEG73QKVN4A6LLN4Z7CRJZTSLRBUDZ4NR4YHZA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842376953 (607h18m19.725790586s) [noise=true , address=45.91.116.14:4189 , piece_id=EDZFMKYC63ABE7V5AE6E4FELMGIWPPPANRNSOFN3GAYE6X6UNUMQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842942175 (607h18m19.726355788s) [noise=true , address=104.36.82.5:4839 , piece_id=NWXJQEAAM2TZKJPIJMRB73K6OCI5A4SLHKY74XLPWXFFMW4RO3CQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842951799 (607h18m19.726365432s) [noise=true , address=77.90.31.14:4661 , piece_id=SU2WXUKNOBMJ5HBPOVU44MD2T4PPMUDHTUSHFBQR6PXQNV5KCCIQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.843362766 (607h18m19.726776393s) [noise=true , address=104.36.82.14:4223 , piece_id=34IMOFVRIMVWGSZ2LJE5PYORVNQD4H3BQQNL5I2QTHD6LNB2MEPQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.844357136 (607h18m19.727770773s) [noise=true , address=45.91.116.10:4081 , piece_id=ZP3GV7W7B2L2M45V4Y7OOWX6Z4Q4MI5PEX2MHMMY7D4L2PJWR3SQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.845591955 (607h18m19.729005589s) [noise=true , address=45.91.116.5:4297 , piece_id=XFZUDFYDRRBBVSW6SS75DZHJHT7GS2DKPXJZCLIF7IDSYZWLSYVQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.846972534 (607h18m19.730386169s) [noise=true , address=77.90.31.19:4843 , piece_id=YYCC2RDU5LFZV5HE2BCQGPOMUAPK3BPFLU4KR65D652IQEENQTFA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.848301465 (607h18m19.731715101s) [noise=true , address=77.90.31.18:4806 , piece_id=6SFVUK4MQG27S35PSQXZHPWUVEYXQG2QHI7TIVSKVL7M4BGO5KSA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.849163101 (607h18m19.732576738s) [noise=true , address=77.90.31.10:4519 , piece_id=3QM3LBGNHO5FUVQH7DNXHMVQON7DOYEABGIXWVICTTPSPUIYJQYA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.849839742 (607h18m19.733253378s) [noise=true , address=104.36.82.3:4767 , piece_id=XPYEQ25OR24YS2LO7PUSHL2KFYYRAONBRJOZBYJP77HMM7ONWJ2Q, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:20.472725878 (607h18m20.356139509s) [upload=485547]: Next returned nil segment Jul 18 16:29:20.472752679 (607h18m20.35616632s) [upload=485547]: Waiting for error group managing segments... Jul 18 16:29:20.596980167 (607h18m20.480393796s) [noise=true , address=45.91.116.10:4081 , piece_id=ZP3GV7W7B2L2M45V4Y7OOWX6Z4Q4MI5PEX2MHMMY7D4L2PJWR3SQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.600142338 (607h18m20.483555961s) [noise=true , address=45.91.116.14:4189 , piece_id=EDZFMKYC63ABE7V5AE6E4FELMGIWPPPANRNSOFN3GAYE6X6UNUMQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.602409866 (607h18m20.485823483s) [noise=true , address=77.90.31.14:4661 , piece_id=SU2WXUKNOBMJ5HBPOVU44MD2T4PPMUDHTUSHFBQR6PXQNV5KCCIQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.605640089 (607h18m20.489053709s) [noise=true , address=45.91.116.9:4262 , piece_id=5MJOYGUIGLNE24MZCQ2PLX75TT4X4XSSVVVBN3WJH352MT5W64GQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.613715192 (607h18m20.497128805s) [noise=true , address=77.90.31.19:4843 , piece_id=YYCC2RDU5LFZV5HE2BCQGPOMUAPK3BPFLU4KR65D652IQEENQTFA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.653906544 (607h18m20.537320177s) [noise=true , address=104.36.82.5:4839 , piece_id=NWXJQEAAM2TZKJPIJMRB73K6OCI5A4SLHKY74XLPWXFFMW4RO3CQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654343289 (607h18m20.537756909s) [noise=true , address=104.36.82.3:4767 , piece_id=XPYEQ25OR24YS2LO7PUSHL2KFYYRAONBRJOZBYJP77HMM7ONWJ2Q, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654633188 (607h18m20.538046806s) [noise=true , address=104.36.82.14:4222 , piece_id=2TAGHXI3NADLHJQXCRIRY5CM6VMI2BHRPT7C6S7Y7TSPC2MGTXSQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654929539 (607h18m20.538343156s) [noise=true , address=104.36.82.14:4223 , piece_id=34IMOFVRIMVWGSZ2LJE5PYORVNQD4H3BQQNL5I2QTHD6LNB2MEPQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655173635 (607h18m20.538587262s) [noise=true , address=104.36.82.7:4911 , piece_id=INO4Q54WEULMMEAEG73QKVN4A6LLN4Z7CRJZTSLRBUDZ4NR4YHZA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655659265 (607h18m20.539072881s) [noise=true , address=45.91.116.5:4297 , piece_id=XFZUDFYDRRBBVSW6SS75DZHJHT7GS2DKPXJZCLIF7IDSYZWLSYVQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655717213 (607h18m20.53913084s) [noise=true , address=104.36.82.12:4151 , piece_id=4I5GEGRSL7PVJE55J5EC3MPDL2ZII74ZFO6KFOCBUZ4WS7KU3F5A, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:49.451525389 (607h18m49.334939016s) [noise=true , address=77.90.31.18:4806 , piece_id=6SFVUK4MQG27S35PSQXZHPWUVEYXQG2QHI7TIVSKVL7M4BGO5KSA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:35536->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451613960 (607h18m49.335027587s) [noise=true , address=77.90.31.11:4553 , piece_id=KYYATSWOYX5R7SKM67PJBA2QZC7PRRNJJ7GKPYLZBOTOZ2J6WFYQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:36256->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451690640 (607h18m49.335104266s) [noise=true , address=77.90.31.19:4841 , piece_id=IERSGG2GKGB5UD3QH3ETCKSJZJT5A6ELFHDLFLJFJ6VB3Y6QIFHA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 139.59.144.22:54400->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451746289 (607h18m49.335159915s) [noise=true , address=77.90.31.10:4519 , piece_id=3QM3LBGNHO5FUVQH7DNXHMVQON7DOYEABGIXWVICTTPSPUIYJQYA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451790375 (607h18m49.335204001s) [seg_pos={0 0}, upload=485547]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 16:29:49.569242253 (607h18m49.452655873s) [upload=485547]: Done uploading segments... Jul 18 16:29:49.569295294 (607h18m49.45270894s) [upload=485547]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.410442062s core uplink us-select-1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 174.831596ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 17:00:23 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 174.856417ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 177.008203ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 16:59:40 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 177.048317ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............X.......X....................................] duration: 2.682464699s FAIL (2025-07-18 16:59:23 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 9]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: context canceled; context canceled Jul 18 16:28:06.573617088 (607h17m6.457030715s) [seg_pos={0 0}, upload=485529]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 16:28:06.573709829 (607h17m6.457123445s) [upload=485529]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 16:28:06.573716041 (607h17m6.457129658s) [upload=485529]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 16:28:06.724230331 (607h17m6.607643945s) [upload=485529]: Done uploading segments... Jul 18 16:28:06.724250141 (607h17m6.607663755s) [upload=485529]: Upload finished. err: <nil> Jul 18 16:29:19.727355583 (607h18m19.610769209s) [upload=485547]: Starting upload Jul 18 16:29:19.727427333 (607h18m19.61084097s) [upload=485547]: Done starting upload Jul 18 16:29:19.727775544 (607h18m19.611189161s) [upload=485547]: Uploading segments... Jul 18 16:29:19.727840340 (607h18m19.611253975s) [upload=485547]: Got next segment. Inline: false Jul 18 16:29:19.727875280 (607h18m19.611288906s) [upload=485547]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 16:29:19.727880962 (607h18m19.611294582s) [upload=485547]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 16:29:19.840919734 (607h18m19.724333363s) [seg_pos={0 0}, upload=485547]: Begin upload segment... Jul 18 16:29:19.840972492 (607h18m19.72438611s) [seg_pos={0 0}, upload=485547]: Done begin upload segment. Jul 18 16:29:19.841004794 (607h18m19.724418434s) [noise=true , address=77.90.31.19:4841 , piece_id=IERSGG2GKGB5UD3QH3ETCKSJZJT5A6ELFHDLFLJFJ6VB3Y6QIFHA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841022981 (607h18m19.724436614s) [noise=true , address=77.90.31.11:4553 , piece_id=KYYATSWOYX5R7SKM67PJBA2QZC7PRRNJJ7GKPYLZBOTOZ2J6WFYQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841446916 (607h18m19.724860553s) [noise=true , address=45.91.116.9:4262 , piece_id=5MJOYGUIGLNE24MZCQ2PLX75TT4X4XSSVVVBN3WJH352MT5W64GQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841470680 (607h18m19.724884316s) [noise=true , address=104.36.82.12:4151 , piece_id=4I5GEGRSL7PVJE55J5EC3MPDL2ZII74ZFO6KFOCBUZ4WS7KU3F5A, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841825001 (607h18m19.725238633s) [noise=true , address=104.36.82.14:4222 , piece_id=2TAGHXI3NADLHJQXCRIRY5CM6VMI2BHRPT7C6S7Y7TSPC2MGTXSQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841863759 (607h18m19.725277377s) [noise=true , address=104.36.82.7:4911 , piece_id=INO4Q54WEULMMEAEG73QKVN4A6LLN4Z7CRJZTSLRBUDZ4NR4YHZA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842376953 (607h18m19.725790586s) [noise=true , address=45.91.116.14:4189 , piece_id=EDZFMKYC63ABE7V5AE6E4FELMGIWPPPANRNSOFN3GAYE6X6UNUMQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842942175 (607h18m19.726355788s) [noise=true , address=104.36.82.5:4839 , piece_id=NWXJQEAAM2TZKJPIJMRB73K6OCI5A4SLHKY74XLPWXFFMW4RO3CQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842951799 (607h18m19.726365432s) [noise=true , address=77.90.31.14:4661 , piece_id=SU2WXUKNOBMJ5HBPOVU44MD2T4PPMUDHTUSHFBQR6PXQNV5KCCIQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.843362766 (607h18m19.726776393s) [noise=true , address=104.36.82.14:4223 , piece_id=34IMOFVRIMVWGSZ2LJE5PYORVNQD4H3BQQNL5I2QTHD6LNB2MEPQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.844357136 (607h18m19.727770773s) [noise=true , address=45.91.116.10:4081 , piece_id=ZP3GV7W7B2L2M45V4Y7OOWX6Z4Q4MI5PEX2MHMMY7D4L2PJWR3SQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.845591955 (607h18m19.729005589s) [noise=true , address=45.91.116.5:4297 , piece_id=XFZUDFYDRRBBVSW6SS75DZHJHT7GS2DKPXJZCLIF7IDSYZWLSYVQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.846972534 (607h18m19.730386169s) [noise=true , address=77.90.31.19:4843 , piece_id=YYCC2RDU5LFZV5HE2BCQGPOMUAPK3BPFLU4KR65D652IQEENQTFA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.848301465 (607h18m19.731715101s) [noise=true , address=77.90.31.18:4806 , piece_id=6SFVUK4MQG27S35PSQXZHPWUVEYXQG2QHI7TIVSKVL7M4BGO5KSA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.849163101 (607h18m19.732576738s) [noise=true , address=77.90.31.10:4519 , piece_id=3QM3LBGNHO5FUVQH7DNXHMVQON7DOYEABGIXWVICTTPSPUIYJQYA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.849839742 (607h18m19.733253378s) [noise=true , address=104.36.82.3:4767 , piece_id=XPYEQ25OR24YS2LO7PUSHL2KFYYRAONBRJOZBYJP77HMM7ONWJ2Q, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:20.472725878 (607h18m20.356139509s) [upload=485547]: Next returned nil segment Jul 18 16:29:20.472752679 (607h18m20.35616632s) [upload=485547]: Waiting for error group managing segments... Jul 18 16:29:20.596980167 (607h18m20.480393796s) [noise=true , address=45.91.116.10:4081 , piece_id=ZP3GV7W7B2L2M45V4Y7OOWX6Z4Q4MI5PEX2MHMMY7D4L2PJWR3SQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.600142338 (607h18m20.483555961s) [noise=true , address=45.91.116.14:4189 , piece_id=EDZFMKYC63ABE7V5AE6E4FELMGIWPPPANRNSOFN3GAYE6X6UNUMQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.602409866 (607h18m20.485823483s) [noise=true , address=77.90.31.14:4661 , piece_id=SU2WXUKNOBMJ5HBPOVU44MD2T4PPMUDHTUSHFBQR6PXQNV5KCCIQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.605640089 (607h18m20.489053709s) [noise=true , address=45.91.116.9:4262 , piece_id=5MJOYGUIGLNE24MZCQ2PLX75TT4X4XSSVVVBN3WJH352MT5W64GQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.613715192 (607h18m20.497128805s) [noise=true , address=77.90.31.19:4843 , piece_id=YYCC2RDU5LFZV5HE2BCQGPOMUAPK3BPFLU4KR65D652IQEENQTFA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.653906544 (607h18m20.537320177s) [noise=true , address=104.36.82.5:4839 , piece_id=NWXJQEAAM2TZKJPIJMRB73K6OCI5A4SLHKY74XLPWXFFMW4RO3CQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654343289 (607h18m20.537756909s) [noise=true , address=104.36.82.3:4767 , piece_id=XPYEQ25OR24YS2LO7PUSHL2KFYYRAONBRJOZBYJP77HMM7ONWJ2Q, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654633188 (607h18m20.538046806s) [noise=true , address=104.36.82.14:4222 , piece_id=2TAGHXI3NADLHJQXCRIRY5CM6VMI2BHRPT7C6S7Y7TSPC2MGTXSQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654929539 (607h18m20.538343156s) [noise=true , address=104.36.82.14:4223 , piece_id=34IMOFVRIMVWGSZ2LJE5PYORVNQD4H3BQQNL5I2QTHD6LNB2MEPQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655173635 (607h18m20.538587262s) [noise=true , address=104.36.82.7:4911 , piece_id=INO4Q54WEULMMEAEG73QKVN4A6LLN4Z7CRJZTSLRBUDZ4NR4YHZA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655659265 (607h18m20.539072881s) [noise=true , address=45.91.116.5:4297 , piece_id=XFZUDFYDRRBBVSW6SS75DZHJHT7GS2DKPXJZCLIF7IDSYZWLSYVQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655717213 (607h18m20.53913084s) [noise=true , address=104.36.82.12:4151 , piece_id=4I5GEGRSL7PVJE55J5EC3MPDL2ZII74ZFO6KFOCBUZ4WS7KU3F5A, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:49.451525389 (607h18m49.334939016s) [noise=true , address=77.90.31.18:4806 , piece_id=6SFVUK4MQG27S35PSQXZHPWUVEYXQG2QHI7TIVSKVL7M4BGO5KSA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:35536->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451613960 (607h18m49.335027587s) [noise=true , address=77.90.31.11:4553 , piece_id=KYYATSWOYX5R7SKM67PJBA2QZC7PRRNJJ7GKPYLZBOTOZ2J6WFYQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:36256->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451690640 (607h18m49.335104266s) [noise=true , address=77.90.31.19:4841 , piece_id=IERSGG2GKGB5UD3QH3ETCKSJZJT5A6ELFHDLFLJFJ6VB3Y6QIFHA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 139.59.144.22:54400->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451746289 (607h18m49.335159915s) [noise=true , address=77.90.31.10:4519 , piece_id=3QM3LBGNHO5FUVQH7DNXHMVQON7DOYEABGIXWVICTTPSPUIYJQYA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451790375 (607h18m49.335204001s) [seg_pos={0 0}, upload=485547]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 16:29:49.569242253 (607h18m49.452655873s) [upload=485547]: Done uploading segments... Jul 18 16:29:49.569295294 (607h18m49.45270894s) [upload=485547]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.682502951s core uplink us-select-1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 177.653651ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 16:59:11 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 177.682467ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 171.899217ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 16:58:56 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 171.954233ms edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................X.......X...................................] duration: 2.499839995s FAIL (2025-07-18 16:58:08 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 9]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: context canceled; context canceled Jul 18 16:28:06.573617088 (607h17m6.457030715s) [seg_pos={0 0}, upload=485529]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 16:28:06.573709829 (607h17m6.457123445s) [upload=485529]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 16:28:06.573716041 (607h17m6.457129658s) [upload=485529]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 16:28:06.724230331 (607h17m6.607643945s) [upload=485529]: Done uploading segments... Jul 18 16:28:06.724250141 (607h17m6.607663755s) [upload=485529]: Upload finished. err: <nil> Jul 18 16:29:19.727355583 (607h18m19.610769209s) [upload=485547]: Starting upload Jul 18 16:29:19.727427333 (607h18m19.61084097s) [upload=485547]: Done starting upload Jul 18 16:29:19.727775544 (607h18m19.611189161s) [upload=485547]: Uploading segments... Jul 18 16:29:19.727840340 (607h18m19.611253975s) [upload=485547]: Got next segment. Inline: false Jul 18 16:29:19.727875280 (607h18m19.611288906s) [upload=485547]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 16:29:19.727880962 (607h18m19.611294582s) [upload=485547]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 16:29:19.840919734 (607h18m19.724333363s) [seg_pos={0 0}, upload=485547]: Begin upload segment... Jul 18 16:29:19.840972492 (607h18m19.72438611s) [seg_pos={0 0}, upload=485547]: Done begin upload segment. Jul 18 16:29:19.841004794 (607h18m19.724418434s) [noise=true , address=77.90.31.19:4841 , piece_id=IERSGG2GKGB5UD3QH3ETCKSJZJT5A6ELFHDLFLJFJ6VB3Y6QIFHA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841022981 (607h18m19.724436614s) [noise=true , address=77.90.31.11:4553 , piece_id=KYYATSWOYX5R7SKM67PJBA2QZC7PRRNJJ7GKPYLZBOTOZ2J6WFYQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841446916 (607h18m19.724860553s) [noise=true , address=45.91.116.9:4262 , piece_id=5MJOYGUIGLNE24MZCQ2PLX75TT4X4XSSVVVBN3WJH352MT5W64GQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841470680 (607h18m19.724884316s) [noise=true , address=104.36.82.12:4151 , piece_id=4I5GEGRSL7PVJE55J5EC3MPDL2ZII74ZFO6KFOCBUZ4WS7KU3F5A, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841825001 (607h18m19.725238633s) [noise=true , address=104.36.82.14:4222 , piece_id=2TAGHXI3NADLHJQXCRIRY5CM6VMI2BHRPT7C6S7Y7TSPC2MGTXSQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.841863759 (607h18m19.725277377s) [noise=true , address=104.36.82.7:4911 , piece_id=INO4Q54WEULMMEAEG73QKVN4A6LLN4Z7CRJZTSLRBUDZ4NR4YHZA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842376953 (607h18m19.725790586s) [noise=true , address=45.91.116.14:4189 , piece_id=EDZFMKYC63ABE7V5AE6E4FELMGIWPPPANRNSOFN3GAYE6X6UNUMQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842942175 (607h18m19.726355788s) [noise=true , address=104.36.82.5:4839 , piece_id=NWXJQEAAM2TZKJPIJMRB73K6OCI5A4SLHKY74XLPWXFFMW4RO3CQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.842951799 (607h18m19.726365432s) [noise=true , address=77.90.31.14:4661 , piece_id=SU2WXUKNOBMJ5HBPOVU44MD2T4PPMUDHTUSHFBQR6PXQNV5KCCIQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.843362766 (607h18m19.726776393s) [noise=true , address=104.36.82.14:4223 , piece_id=34IMOFVRIMVWGSZ2LJE5PYORVNQD4H3BQQNL5I2QTHD6LNB2MEPQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.844357136 (607h18m19.727770773s) [noise=true , address=45.91.116.10:4081 , piece_id=ZP3GV7W7B2L2M45V4Y7OOWX6Z4Q4MI5PEX2MHMMY7D4L2PJWR3SQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.845591955 (607h18m19.729005589s) [noise=true , address=45.91.116.5:4297 , piece_id=XFZUDFYDRRBBVSW6SS75DZHJHT7GS2DKPXJZCLIF7IDSYZWLSYVQ, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.846972534 (607h18m19.730386169s) [noise=true , address=77.90.31.19:4843 , piece_id=YYCC2RDU5LFZV5HE2BCQGPOMUAPK3BPFLU4KR65D652IQEENQTFA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.848301465 (607h18m19.731715101s) [noise=true , address=77.90.31.18:4806 , piece_id=6SFVUK4MQG27S35PSQXZHPWUVEYXQG2QHI7TIVSKVL7M4BGO5KSA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.849163101 (607h18m19.732576738s) [noise=true , address=77.90.31.10:4519 , piece_id=3QM3LBGNHO5FUVQH7DNXHMVQON7DOYEABGIXWVICTTPSPUIYJQYA, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:19.849839742 (607h18m19.733253378s) [noise=true , address=104.36.82.3:4767 , piece_id=XPYEQ25OR24YS2LO7PUSHL2KFYYRAONBRJOZBYJP77HMM7ONWJ2Q, seg_pos={0 0}, upload=485547]: Uploading piece... Jul 18 16:29:20.472725878 (607h18m20.356139509s) [upload=485547]: Next returned nil segment Jul 18 16:29:20.472752679 (607h18m20.35616632s) [upload=485547]: Waiting for error group managing segments... Jul 18 16:29:20.596980167 (607h18m20.480393796s) [noise=true , address=45.91.116.10:4081 , piece_id=ZP3GV7W7B2L2M45V4Y7OOWX6Z4Q4MI5PEX2MHMMY7D4L2PJWR3SQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.600142338 (607h18m20.483555961s) [noise=true , address=45.91.116.14:4189 , piece_id=EDZFMKYC63ABE7V5AE6E4FELMGIWPPPANRNSOFN3GAYE6X6UNUMQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.602409866 (607h18m20.485823483s) [noise=true , address=77.90.31.14:4661 , piece_id=SU2WXUKNOBMJ5HBPOVU44MD2T4PPMUDHTUSHFBQR6PXQNV5KCCIQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.605640089 (607h18m20.489053709s) [noise=true , address=45.91.116.9:4262 , piece_id=5MJOYGUIGLNE24MZCQ2PLX75TT4X4XSSVVVBN3WJH352MT5W64GQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.613715192 (607h18m20.497128805s) [noise=true , address=77.90.31.19:4843 , piece_id=YYCC2RDU5LFZV5HE2BCQGPOMUAPK3BPFLU4KR65D652IQEENQTFA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.653906544 (607h18m20.537320177s) [noise=true , address=104.36.82.5:4839 , piece_id=NWXJQEAAM2TZKJPIJMRB73K6OCI5A4SLHKY74XLPWXFFMW4RO3CQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654343289 (607h18m20.537756909s) [noise=true , address=104.36.82.3:4767 , piece_id=XPYEQ25OR24YS2LO7PUSHL2KFYYRAONBRJOZBYJP77HMM7ONWJ2Q, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654633188 (607h18m20.538046806s) [noise=true , address=104.36.82.14:4222 , piece_id=2TAGHXI3NADLHJQXCRIRY5CM6VMI2BHRPT7C6S7Y7TSPC2MGTXSQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.654929539 (607h18m20.538343156s) [noise=true , address=104.36.82.14:4223 , piece_id=34IMOFVRIMVWGSZ2LJE5PYORVNQD4H3BQQNL5I2QTHD6LNB2MEPQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655173635 (607h18m20.538587262s) [noise=true , address=104.36.82.7:4911 , piece_id=INO4Q54WEULMMEAEG73QKVN4A6LLN4Z7CRJZTSLRBUDZ4NR4YHZA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655659265 (607h18m20.539072881s) [noise=true , address=45.91.116.5:4297 , piece_id=XFZUDFYDRRBBVSW6SS75DZHJHT7GS2DKPXJZCLIF7IDSYZWLSYVQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:20.655717213 (607h18m20.53913084s) [noise=true , address=104.36.82.12:4151 , piece_id=4I5GEGRSL7PVJE55J5EC3MPDL2ZII74ZFO6KFOCBUZ4WS7KU3F5A, seg_pos={0 0}, upload=485547]: Done uploading piece. err: <nil> Jul 18 16:29:49.451525389 (607h18m49.334939016s) [noise=true , address=77.90.31.18:4806 , piece_id=6SFVUK4MQG27S35PSQXZHPWUVEYXQG2QHI7TIVSKVL7M4BGO5KSA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:35536->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451613960 (607h18m49.335027587s) [noise=true , address=77.90.31.11:4553 , piece_id=KYYATSWOYX5R7SKM67PJBA2QZC7PRRNJJ7GKPYLZBOTOZ2J6WFYQ, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:36256->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451690640 (607h18m49.335104266s) [noise=true , address=77.90.31.19:4841 , piece_id=IERSGG2GKGB5UD3QH3ETCKSJZJT5A6ELFHDLFLJFJ6VB3Y6QIFHA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 139.59.144.22:54400->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451746289 (607h18m49.335159915s) [noise=true , address=77.90.31.10:4519 , piece_id=3QM3LBGNHO5FUVQH7DNXHMVQON7DOYEABGIXWVICTTPSPUIYJQYA, seg_pos={0 0}, upload=485547]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 16:29:49.451790375 (607h18m49.335204001s) [seg_pos={0 0}, upload=485547]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 16:29:49.569242253 (607h18m49.452655873s) [upload=485547]: Done uploading segments... Jul 18 16:29:49.569295294 (607h18m49.45270894s) [upload=485547]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.499867623s core uplink us-select-1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 171.969041ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-18 16:58:04 UTC) orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": EOF Duration: 172.000211ms edge ipfs us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get) antiblip: 60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 171.223354ms remote: "tcp:5.161.37.179:443" target: https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi FAIL (2025-07-18 16:57:49 UTC) orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": EOF Duration: 171.250394ms edge ipfs us1

Orbiter Check History

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)